Meet Goals by KeepSolid 3.0!
Get the updated version now for half the price
Also meet the new Google Calendar integration

A/B Test

What is A/B Test?

A/B Test is a data-driven decision-making process. This method is quite popular in product development, especially in case of limited information.

When a team wants to introduce changes to their product, they expect that the changes will increase the product’s attractiveness to their customers and, consequently, its value will rise.

Yet, before proceeding with implementation, it is essential to estimate the value increase or, in other words, the ROI (Return On Investment).

A/B testing is one of the tools that minimize possible risks by leading users to confirm or disprove the product team's hypothesis. Furthermore, the method collects first-hand information by studying the real user behavior.

Learn the definition of A/B Test

How it is run

A/B testing is also called split testing or A/B split testing. This is an experiment where users are split into random groups and each group is shown different facets of the solution: design, UX, text and so on. Group A is the control group that sees no changes and is used as a reference to measure the changes effectiveness. Group B sees a variation of the solution, the hypothesis that should be verified.

Example:

Two groups receive lists with different designs or with different call-to-action texts. Based on their impression and other metrics, the marketing team evaluates the best performing variant.

Helpful metrics

Pure A/B testing can be supported by additional popular metrics that are used for solution evaluation:

  1. Conversion: calculated as a proportion of the total number of visitors who took an expected action, for example filling out a form on a landing page, making a purchase in an online store, subscribing to news, or clicking a link.
  2. Economics Metrics: Total Revenue (TR), ARPU (average revenue per user), Number of customers (C) and others.
  3. User Behavior: assessment of the visitors’ interest in the resource. The key metrics are: pageview depth - the number of pages viewed, average session duration, bounce rate - Churn rate, Retention rate, etc.

Decision making based on A / B tests

By splitting users in groups, it is possible to repeat the same process for other elements of the product, most frequently website design, UI/UX improvements, or UX in applications.

Most importantly, if A/B tests are run regularly, they measure the team’s expectations against real-world data, leading to better estimations and ultimately better product.

 

Stay on top of Business Goals

Set Goals, Plan Strategy, Involve Your Team, and Achieve Big Results. Manage the future with our Leading Goal Management Software

All rights reserved