One of the key challenges in measuring user engagement is defining what constitutes too low or satisfactory engagement level. While you could create a benchmark using typical industry engagement metrics, this is not always straightforward. Each product is different not only in terms of positioning, but also in terms of maturity. This makes direct comparisons challenging. At Vespucci, we have developed relative engagement measurement tools to address this issue. These tools provide a simple yet flexible way to measure user engagement on your product.
Vespucci does not measure the absolute time spent on your product. Instead, we create statistics to measure the engagement of your installed base, and then position your users in relation to these statistics. For example, you can see which users are highly engaged, engaged, or not very engaged, relative to the rest of your installed base.
You might be wondering how to use these tools. The first step is to write a story based on engagement. In the first box of the story editor, select "🔥 User Engagement".
You will then be prompted to specify a "Target Level". Let's set it to 50% for this example. You will then be asked if you are interested in users who spend 50% more or less time than the rest of your installed base.
Next, you need to indicate which statistic you want to use to evaluate your installed base. Here, we will choose the average as it is the most intuitive measure. Keep in mind that other stories are possible. For example, you may decide to use measures such as deciles or the median to qualify your installed base.
Finally, you must specify whether the measurement is performed on your entire installed base or specifically on users of certain screens of your product.
And there you have it, you've told Vespucci that you want to understand what is driving the engagement of these users. Let's go to the most interesting step now. Start by selecting the "brain" icon in the toolbar.
The icon reveals the Insights Finder, which uses powerful statistical models to identify the drivers of your users' behavior. Specifically, it identifies the elements that increase (or decrease) the probability of your user adopting the behavior you have described. In this case, we will focus on the elements that increase the probability of adopting a high level of engagement. These "elements" are organized into four axes: your Segment or Amplitude properties, the socio-demographic characteristics of your users, and the actions they have taken (this last section shows how the actions they have taken in the past can affect the likelihood of exhibiting one or the other engagement profile). In this case, we will focus on the first axis.
Looking at the screenshot, we can see that the example tagging plan (based on a fitness application) includes various elements that qualify the sports session, such as the Difficulty, Rating, the duration of the Workout session or the musical genre associated with the session.
Out of all the elements mentioned, only the duration and musical genre appear to impact the probability of adopting the engagement profile we have described. Let's take a closer look. Clicking on the "genre" property reveals more details. We can see that the "Chillout" and "Rock" genres increase the probability of engagement. In fact, a user who followed a session with a "Chillout" musical background is 40% more likely to be highly engaged with the product than those who did not.
This is just the beginning of the exploration. Next week, we will discuss segmentation issues and help you define a more accurate persona.
