How to effectively measure a digital product in 4 steps

"What is not defined cannot be measured. What is not measured, cannot be improved. What is not improved, is always degraded".

This phrase, coined by the mathematician and physicist William Thomson Kelvin, is the basis upon which any measurement strategy begins—or should begin—that allows us to analyze, evaluate, foresee and execute improvements to any process or product.

In this sense, it is crucial to define the strategies and tools we use to obtain quantitative and qualitative data (i.e., to measure), and that we understand what objectives are guiding us.

At Modyo, the Performance Analysis team is in charge of ensuring that all digital products are measurable, and, more importantly, we integrate ourselves into all of the different teams so that these same products are flexible and scalable once they go into production.

Table of contents

• How to effectively measure a digital product in 4 steps
  • Define objectives and key results
  • Create a Measurement Plan
    • What is a Digital Measurement Plan?
  • Implementation of the Measurement Plan
  • Visualize the data and Optimize your products (if necessary)
    • Why is data visualization important?
• Conclusion

These are the four steps we follow to effectively measure the digital products we develop::

1. Define objectives and key results

The Measurement Process starts with the definition of OKRs (Objectives and Key Results), a management methodology designed to define the goals of a digital product and to measure its effectiveness. 

The structure of the OKRs is divided into two parts: Objectives: the high-level definitions of the business. It answers the question, "What do we want to achieve?" Key results: the quantifiable outcomes that measure progress against the objectives. It answers the question, "How will we know if we have achieved them?" To establish them, we follow the SMART methodology (specific, measurable, achievable, realistic and time-bound).

To establish both the objectives and the relevant key deliverables and to know how to break them down, Modyo recommends working in several sessions with all the business partners involved. This gives us an overview of all the goals to consider and assures that no serious modifications arise during product development.

2. Create a Measurement Plan

Before knowing how to create it, we must define…

What is a Digital Product Measurement Plan?

It’s a document that breaks down and defines a digital product's relevant objectives. These objectives serve as success criteria that determine whether the product needs to be improved or whether it’s achieving its purpose.

To define and break down the OKRs, deciding how we are going to evaluate their fulfillment and what metrics and/or dimensions they will be based on, we must include the following in the Measurement Plan:

• The business objectives
• The key results into which these objectives are broken down
• The metrics and dimensions that we we’ll use to measure the key results
• The quantitative and qualitative goals (or targets) that we want to achieve and that will determine the success of the product in an established time frame
• The tentative language of the custom events to implement or scripts needed that will determine how we will visualize them in the analytics tools

This artifact needs to be visual, readable, and function as a working document that we’ll return to during implementation. In this table we break down a possible target for a digital product as an example:

The Measurement Plan will help us recognize which indicators (metrics and dimensions) we need to gauge the objectives’ success, as well as to know if we can obtain them natively from the analytics and measurement tools we use, or if we must make specific implementations that allow us to obtain them in a customized way.

IMPORTANT: The most recommendedoption when seeking to obtain specific and relevant metrics/dimensions of a product are custom events, which can measure interactions with any kind of digital product (clicking on a specific button, downloading a document, submitting a form, etc.).

3. Put the Measurement Plan into Practice

Once we have "planted" the bases of what we want to measure according to the objectives and key results, we’ll use the measurement and analytical tools to implement the Measurement Plan. 

The most common tools are the ones that the Google Marketing Platform makes available for free (Google Analytics and Google Tag Manager), but there are also other options such as Tealium, Signal, Adobe's Launch, etc., for the tag system, or Contentsquare, Adobe Analytics, Kissmetrics, etc., for web analytics.

As part of this implementation and as part of the product development, the tentative language we define in the Measurement Plan may vary. Even so, we can define parameters of the interaction events with the product, so that we can identify them with that nomenclature in the analytics tools. Let's see an example...

Site interaction: we can complete the following form on the Modyo.com website to create an account, start a free trial, and click on "Create" to submit it.

This could be reflected in the measurement tool as follows:

Event	Create an account
Section	Complete form
Button	Create

4. Visualize the data and optimize your product (if necessary)

Why is data visualization important?

According to an estimate by the World Economic Forum, by 2025 we will be creating 463 exabytes of data every day, equivalent to 212 million DVDs. For the human brain, all this data is difficult to understand if we don’t have a basis that gives them a context. Therefore, data visualization is a crucial resource to make sense of it.

Dashboards are a way to present this data in a coherent, quick and easy-to-interpret format. The process of capturing the information in these panels is based on selecting the relevant data to answer the questions about the level of achievement of the objectives we set out at the beginning. In other words, by means of this data visualization we must be able to answer: are we meeting the objectives?

Thus, this dashboard should have the following characteristics:

• Each graph on the dashboard must have a clear purpose and respond to a specific key result of the Measurement Plan,
• The dashboard and the information displayed must address a specific audience (business management, marketing team, etc.),
• It should show relevant information in a clear and efficient way,
• It is advisable to show trends and changes in data over time, 
• We must choose the right type of chart (bar charts, tables, treemaps, etc.) to show the data we have selected,
• We recommend organizing the graphs hierarchically according to the relevance of each one to facilitate reading for the user. We can consider resources such as color, size and position of the elements, as it can also help the understanding of the data.
• It is advisable to provide filters based, for example, on dimensions, date ranges, etc., in order to customize the way in which the data is displayed.

Based on the visualization and data analysis from this panel, and its comparison with the goals set in the first stage of the measurement process, we can evaluate the success and/or achievement of the objectives. 

If they are not being met as expected, optimizations or improvements can be made to the developed digital product, such as:

• Relocate menu items, 
• Modify the order of home sections,
• Rethink the design of a download button,
• Adjust the information architecture of a product and its by-products,
• Adapt the fields of a form,
• etc.

Conclusion

More and more companies are joining the necessary digital transformation process and integrating new technologies into their business models. These implementations put a large amount of data at their disposal, requiring a frame of reference and a comprehensive analysis to inform relevant decision making for these companies. It is necessary, then, to carry out a measurement strategy through this series of steps, which will allow us to identify the indicators that will determine a digital product’s success. 

Building and evolving a digital product is a long-term process during which it must be constantly monitored and optimized based on its performance and objectives. Developing measurable digital products allows us to adopt a strategic and precise view that contributes to this task of continuous evolution.

Photo by Jeremy Bezanger on Unsplash.