In our fast-paced digital world, where customer and user satisfaction can make or break a product or service, understanding the value of user experience (UX) is of paramount importance.
In our experience, companies and organisations often rely on gut feeling or subjective assessments when it comes to UX. The effectiveness of UX or the status quo is rarely recorded - i.e. measured. As a result, the value of UX is not really "tangible", processes are optimised based on gut feeling, and budgets for this resulting "spongy" construct called "user experience" are therefore understandably released less and less.
- Is measuring the user's experience important?
- UX ist messbar – auf das wie kommt es an
- Can you measure UX with KPIs?
- Was die Messung von UX so herausfordernd macht: User Experience ist ein latentes Konstrukt
- Recommendation: Continuously measure UX with a mix of methods
- Key takeaways: Measuring the UX means minimising risk
Is measuring the user's experience important?
Without taking stock, you don't know what should be optimised - and without measuring the success of the UX measures implemented, it remains unclear whether they are actually effective. This is because no one knows exactly or can specify what the adjustments made after the last user survey or usability test have achieved.
This often seems to be because companies and organisations don't quite know where to start or where to begin to measure or "how to do it right".
For the sake of simplicity, hard figures from the analysis tools are often used as indicators of whether the user's experience is "good" or "bad". However, as these (alone) often do not provide comprehensive insights, companies are quickly disappointed by the process and put the UX measurement on the back burner again.
However, there are various ways to gain good insights into the UX, to measure and evaluate it.
UX is measurable - on the like it depends
To measure UX, qualitative and quantitative usability tests and many standardised questionnaires for the user experience are available. One of these questionnaires is the UEQ (Laugwitz et al., 2008). The UEQ allows a quick assessment of various aspects of software quality based on six factors. The AttrakDiff (Hassenzahl et al., 2003) captures various qualities such as usefulness and usability (is it simple or complicated?) but also the so-called "hedonic" Quality (Does it feel valuable or inferior? Is it original or rather conventional?)
Usability tests also play a central role in recording the user experience. Through observation and direct feedback from users, as well as the opportunity to ask specific questions, we gain valuable insights into actual usage and potential areas for improvement. These tests help us to identify real problems and continuously optimise not only pure usability, but also the user experience.
Can you measure UX with KPIs?
The conversion rate or the abandonment rate are often used as indicators of a "good" or "bad" UX. However, in our opinion, UX cannot be measured using such business key figures - the so-called key performance indicators (KPIs) - but rather about them on the user experience close. However, this should be treated with caution and is therefore associated with the risk of misinterpretation. This is because it does not measure the actual user experience, but captures a snapshot and trends of key business figures. It is therefore primarily used to measure business success - and various factors can influence why this increases or decreases.
It may be possible to recognise a trend - particularly with the two key figures mentioned (conversion rate and abandonment rate) - as to whether the UX measures implemented are bringing about the desired business success. However, so many other variables (confounding variables) play a role here - making it difficult to make an improved user experience (partly) responsible for this.
The same applies to many quantitative variant tests such as A/B tests. We will only find out that variant A performs better than variant B - but we don't necessarily know why - and whether this is due to a positive user experience.
What makes the measurement of UX so challenging makes: User experience is a latent construct
In order to be able to measure UX, we first need to briefly clarify what we are actually dealing with "in terms of measurement": It is important to understand that the user experience is not a property of a product or an interface. A website or app does not "have" a UX. Rather, the experience "happens" in the mind of the individual user as a reaction to their interaction with a digital product or service.
In scientific terms, the user experience is therefore a so-called latent construct - This means that it can not direct can only be derived or inferred from other - again observable - factors. However, these other factors (which all together represent the user experience) must first be defined using special and complex statistical methods (factor analyses). This is the only way to ensure that what is then measured actually reflects the user experience and not just anything else. This is precisely where the standardised questionnaires listed above come into play, as these use factor analyses to determine and ultimately query the factors that reflect the user experience.
Recommendation: Continuously measure UX with a mix of methods
Through a good combination of various methods, we can usually gain good insights into user behaviour and then nothing stands in the way of continuous optimisation of the user experience.
Usability tests, for example, usually provide deep insights into how users actually interact with the product. By analysing task completion, user behaviour and ongoing user comments during these tests, specific problems and potential for improvement can be identified. The tests also provide good insights into user thinking.
Usability tests and standardised UX questionnaires should therefore always supplement the insights gained through pure key figures - e.g. from KPIs - because they help to gain a comprehensive picture of the user experience.
We also always recommend continuously measuring the UX in order to assess whether an implemented UX optimisation was successful or not. The methods mentioned above are also a great way to take an initial inventory to find out where you stand and what needs to be optimised. This allows you to make good strategic decisions - or avoid making the wrong ones.
Key takeaways: Measuring the UX means minimising risk
It is important that companies can evaluate the UX measures they have implemented to assess their effectiveness by measuring them or taking stock so that they can make informed decisions about where adjustments are needed and how they should be prioritised. This ensures that these services and products are well received by users and customers. It also serves to minimise the company's risk - as it avoids expensive bad investments in development based on gut feeling and assumptions.
The use of a mix of methods - i.e. the combination of qualitative and quantitative methods - allows companies to gain valuable insights into the user-friendliness, emotional impact and general effectiveness of the product or service. This allows insight-based and informed decisions to be made, which can improve the user experience and thus significantly increase the success of the business.
Sources
Hassenzahl, M., Burmester, M., & Koller, F. (2003). AttrakDiff: A questionnaire to measure perceived hedonic and pragmatic quality. In G. Szwillus & J. Ziegler (Eds.), Man & Computer 2003 (Vol. 57, pp. 187-196). Vieweg+Teubner Verlag. https://doi.org/10.1007/978-3-322-80058-9_19
Laugwitz, B., Held, T., & Schrepp, M. (2008). Construction and Evaluation of a User Experience Questionnaire. In A. Holzinger (Ed.), HCI and Usability for Education and Work (pp. 63-76). Springer. https://doi.org/10.1007/978-3-540-89350-9_6
About this article series
In our series "UX myths debunked", we clear up common misconceptions in user experience design and share our views on them. You can find the link to the first part below the article.
If you liked this article, you can read part 1 here: UX myths debunked: UX design is only relevant for websites and apps.
Would you like to improve the user experience of your service or product?
Write to us or give us a call! We support you with questions about measuring UX, usability tests, interviews and the development of ideas and concepts, and advise you on tricky UX challenges or frustrated users.
We are happy to offer a non-binding initial consultation to get to know each other and look forward to hearing from you and your concerns!
