How to work with Data Scientists as a Product Manager

As PMs, we often work with data scientists. This trend is going to accelerate in a GenAI-led future.

In this article, I want to help PMs understand how to work with Data Science and help DS solve the right problems while balancing tradeoffs. 

1. Nail down the objective function. 

Any real-life business problem will have success metrics and counter metrics. For example, as a PM, you may set the objective function to maximise the conversion on your e-commerce funnel, and you may want to build a DS-based ranking model on your search results page. The model will likely rank lower ticket-size products on the top because conversion is inversely proportional to price. You are making a lower gross margin by selling items of lower ticket size. The right thing to do is maximise ARPU & this may have a downward impact on conversion. 

Setting the right goals and aligning them with the relevant stakeholders is crucial before goals are finalised. 

2. Nail down the user segment

As a PM, you are the expert on the user. Again, let's take the same example as above. To maximize ARPU, you need to balance b/w ticket size and conversion. A key component of this is demand elasticity. How much bumping the higher-priced products impacts conversion? Now, the granularity at which you break down your users and user segments and build the elasticity curves will affect how much value you get from your ranking model. If you serve an affluent customer with a cheaper item because you need to break down the segments sufficiently, you leave money on the table. 

You should ensure models cover the nuance and take advantage of these important details. 

3. Help your Data Science(DS) team with necessary domain context.

As a PM, you must understand the user's mental model, what impacts his decision-making, and what input works as a proxy. You should recommend high-quality inputs (or data) to the DS team. Again, taking the same example as above, the price bucket of a user's device is a great indicator of user cohort/segment, and the ranking model can improve its efficiency by including this quality input. You must ensure this attribute makes it to the feature list.

4. Help your team with feature (or inputs) coverage (in other words, more data)

Any model is as good as the quality of inputs (or data). Let's say gender is a good predictor of what kind of content a user consumes on an app, and then it becomes super important to have good coverage of this input. As a PM, it is your responsibility to build useful customer-facing features on your app that get users to reveal their gender and ensure good coverage of this input. For example, you could build a user preferences/settings feature on your app that increases coverage of this input.