Thoughtful Analytics: Know the Product

A few weeks back, I shared the outline for the Thoughtful Analytics framework. As promised, I will expand on the first pillar of the framework: Know the Product. This pillar is guided by the foundational value of Empathy.

Know the product to create plausible hypotheses

How often as an analyst have you been given just raw data, with zero context about it, and been asked to answer some key philosophical questions about what it all means? I hope that’s never happened to you, though if it has, ¯\_(ツ)_/¯ as a response is totally appropriate!

Data needs context and nuance for you to be able to see the full picture.

It helps to understand how it was generated, and what caveats exist around it. It takes time and asking a lot of questions to start getting familiar with this context. An effective way to keep yourself focused, is by getting intimate with your software/hardware product, or service oriented business. I use “product” as an all-encompassing term in this series.

As you spend time learning the product, you develop empathy towards the engineers that coded it, the designers who designed it, the marketers who need to market it, and ultimately the customers who will use it. This process improves the knowledge that is later required for creating successful hypotheses, and to make recommendations based on actual data.

Understand the customer perspective

To better appreciate the customer perspective about a product, we need to understand how the product works, and use it like a customer would. What are its strengths and limitations? How does it feel to use it? Which parts are intuitive, and which parts seem clunky? These are all great questions to begin understanding the product from the customer perspective. This is the first step in building empathy with the customer, by putting yourself in their shoes.

Not all customers use a given product in the same way. The way you use it might represent the 99% population, or the 1%. However, just going through that process can help you start building an understanding of what the flow looks like, and how that might affect the data that gets created through the product’s use.

If the product is not a consumer facing one, it might not be easy to familiarize yourself in this manner. The next best option is to listen to the voice of the customers. Analysts can do this by reading customer feedback, customer support emails or any forums where customers talk about the product. This will build an understanding of the most common compliments and complaints customers have about the product.

Sometimes, analysts will observe unexpected data anomalies. An analyst well-versed with the product can create more effective hypotheses about the probable cause. For example you may be surprised that the data says the user picked an option that doesn’t exist on the option list, until you remember that there are no guard-rails preventing the customer from entering their own value as an alternate option.

Learn about the intended use of the product

I often advocate analysts use the product (or any new features) before they have any bias of how it was intended to be used. The way a product is used can sometimes be wildly different from how it was intended. These gaps can occur due to a variety of reasons, but will consistently affect the analyst’s ability to form a relevant hypothesis. Being aware of these two sides, can help with the process of choosing the more likely hypothesis to test first, and thus shortening the decision making time.

Reading the documentation, or even watching a demo video is a great way to get started on this. The Product and Engineering teams can usually provide a ton of resources. I’ve also found it extremely helpful to build a good relationship with the teams that sell the product — they typically have a great understanding of what market needs are being addressed by the product. Bouncing off ideas about my hypotheses with such teammates has proved incredibly useful time and again.

Often, there can be some obvious signs in the data that the product is not being used as expected. For example, maybe you see bad data crop up for a field that is expected to be auto-populated, or you notice that a particular feature that is supposed to be higher up in the funnel than another feature, never gets used. The analyst is in a position to recommend such issues to be fixed, and such discoveries can really make a difference in the product experience for the eventual customer. Ultimately, a great product experience drives retention, which is a common difference between successful and failed products.

Build an empathetic mindset

Empathy can be practiced. When you find ways to explicitly and intentionally relate to the various ways people think about the product you are supposed to analyze, you deepen the impact of your craft. You are in a better position to advocate for the product and its customers through your analyses.

Typically you’ll never exhaust all the ways there are to use the product, but it’ll be much easier to understand odd things that show up in the data, when you have at least some degree of familiarity. At the end of the day, you are practicing curiosity, and letting it lead you to new ideas that might have been previously unexplained.

If you regularly spend time experiencing the products and processes that you have to analyze, the resulting empathetic mindset will pay big dividends in the quality of analysis. The key is to always keep learning!

Action item to practice: Show Curiosity!

In my company, I usually find a customer support or sales representative who will let me occasionally ride along as a fly on the wall during their customer interactions. Yes, I can dive into the data generated by these or other voice of customer interactions — but this is so much more fun! My observations during these interactions have been the key to producing a good analysis more than once.

Understanding the intensity of how important something is to the customer gives you a visceral feel of how your work impacts real people and their lives.

Relating with customers at an individual level is not meant to scale, but is a valuable way to connect and appreciate how products affect people. For example, your product may be initially built for the majority, but you can make great strides towards inclusion if you make recommendations that are validated with data to build out the niche use cases that are equally sought after.

I hope these recommendations to Know the Product to power Thoughtful Analytics are useful! Next up, I will discuss the other crucial pillar for the framework: Know the Data.

I respond to comments on my post here or on Twitter (@ShailviW). Feel free to share your thoughts so far! You can view my previously published content here: http://www.shailvi.com/analytics-knowledge.html

Originally published at https://www.linkedin.com.