The Ultimate CRO Vocabulary & Dictionary

When it comes to having an online business, there are dozens of acronyms and terms to get your head around. This guide will be your go-to CRO Vocabulary relating to all things Conversion Optimisation.

• Conversion Rate
• Conversion Rate Optimisation
• Control
• Variant
• Hypothesis
• A/B Testing
• A/B/n Testing
• Isolation Testing
• Batch Testing
• Multivariate Testing (MVT)
• Statistical Significance
• Statistical Power
• Minimum Detectable Effect (MDE)
• Personalization
• Attribution
• Conversion Funnel
• Wireframing
• Macro-Conversion (primary conversion metric)
• Micro-Conversion (secondary conversion metric)
• UCD (in a digital context)
• Agility (in a CRO context)
• User Research
• P.E.T
• Quality Assurance
• Personas
• User Journey
• Heuristic Evaluation
• Uplift
• Data Science
• Persuasion
• Social Proof
• Iterative Test
• Innovative Test
• Call to Action (CTA)
• Visitor Segmentation
• Test Prioritisation

Conversion rate

Conversion rate is the number of visitors that complete the desired goal divided by the total number of visitors you have to the site. Depending on whether your website is a transactional Ecommerce site or a lead generation site you will have ‘conversion goal’ as they usually align with your business goals. For example a conversion goal on an Ecommerce site might be when a visitor purchases something, or for a lead generation site, when a visitor requests a call back.

Conversion Rate Optimisation  (CRO)

Ever wondered “what Does CRO mean”? – Conversion Rate Optimisation or Conversion Optimisation refers to the process of improving the percentage of visitors who complete the desired goal on your website. This means that your business can get higher returns on the traffic that is already being sent to your website. Read our full beginner’s guide to CRO here.

Control

When conducting an A/B test, ‘A’ (the original) is known as the control. When testing, a control is included in all experiments to prove that the results of a test are because of the changes made (the variant) and not other circumstances.

Variant

When conducting an A/B test, ‘B’ is known as the variant. This is a changed version of the part of your site you are testing (which is the control).

Hypothesis

When you have identified a pain-point or opportunity for improvement from your data and user research, you build a hypothesis. Essentially a hypothesis is an idea or solution on how to improve part of your website. You then observe the validity of your hypothesis with an A/B test. The stronger the hypothesis (strength being determined by how much user research, data analysis and other research sources have been used to develop the hypothesis) the more likely your ‘variant’ will outperform the ‘control’.

Here is a hypothesis kit on how to develop a ‘strong’ hypothesis.

A/B testing

In order to improve your conversion rate, you need to identify and improve various parts of your site.

A/B testing (sometimes referred to as split testing) works by first identifying what area of the site you want to improve. You can split test most things such as website copy, calls to action, processes, etc. When A/B split testing you will have an “A” and “B” variation. The ‘A’ version will usually be your control (the current version of the page/element) while ‘B’ will be a version of the page/element with the changes you have made When the test has finished, you can compare the results to see which version performed better against your defined test metrics and goals.

A/B/n testing

A/B/n testing is the same as A/B testing but rather than testing one variant (the ‘B’) against the control (the ‘A’) you will test multiple. When the test is live, you will split the traffic equally between the control, the first variant and then the other variables. If you have four variants, you will split the traffic to your site five ways and the ‘n’ will be variants ‘C, D, E’.

A/B/n testing is particularly effective for high-traffic sites whose tests can gain statistical significance quicker. This way they can test more variants in one test to determine a champion.

Isolation testing

This is your standard A/B test, where you change just one element of a page in one test variation (i.e. a headline, an image etc.). You may have more than one test variation (B, C, D) being tested alongside your control (A).

One of the main advantages of isolation testing is that you will know with clarity that the one change you have made has directly impacted your primary conversion metric. Isolation testing, however, can be quite a time consuming task for those with lower traffic volumes (as tests take longer to reach statistical significance) and require a lot of resources.

Batch testing

Rather than having one element variation, a batch test is an A/B test that features multiple element changes to a page. For example, instead of running two isolation tests that test a copy change against the control or a header image against the control, you run one A/B test which includes both variations against the control.

Batch testing is perfect for sites with lower traffic volumes and fewer resources. One issue with batch testing, however, is that it is harder to deduce which specific change you have introduced has had the most impact on conversion changes. To avoid this, ensure your analytics tool and tracking is set up correctly.

It is worth noting that many sites that have run a batch test to gain a positive net impact will later conduct isolation tests on the various elements to deduce what provided the positive result.

Multivariate testing (MVT)

Multivariate testing is a technique for testing a hypothesis in which multiple variables are modified. So instead of running one batch A/B test, multivariate testing runs multiple tests, each with a different combination of element variations. The ultimate goal is to see which combination delivers the biggest uplift against the control (rather than whether your one batch A/B test with multiple changes beats the control).

The number of tests will always be [# of Variations on Element A] X [# of Variations on Element B] (and so on) = [Total # of Variations]. So for example, if you have two variations of a header image (A) and two variations of a headline (B) you will have four separate tests running.

Due to a higher number of variations being run at one time, multivariate testing requires a high volume of site traffic (to compensate for the split of traffic to each variation) to ensure each variation in the test reaches validity.

Statistical significance

Statistical significance is a measure used to assess the probability of a fluke result when testing. The industry norm for statistical significance is at 95%. This means that when examining test results, you are 95% certain that the difference in conversion rate percentage accurately represents your site traffic, therefore making the results of the test valid.

Statistical power

Statistical power measures how often, when a variation is different from the original, the test data will pick up on it. If statistical power is high, the chance of a false result, or concluding there is no effect on the test when there is one, decreases.

If a test’s statistical power is 80% or higher (the industry norm) you know that when your test has reached Statistical significance the result is highly likely to be true and accurate.

You can read more about statistical significance and power here.

Minimum Detectable Effect (MDE)

When running a test, you need to decide how much how much of a change from your baseline conversion rate you want to be able to detect.

Using a tool such as Optimizely’s Sample Size Calculator, you can work out how much traffic your test (with its MDE) needs to reach statistical significance. For example, if you have a baseline conversion rate of 5% and wish to detect a 10% change (so a conversion rate of 4.5% or 5.5%) you will require 31000 users.

In general, smaller differences (such as 5%) take longer to detect because you need more data from users to confirm that a test reached validity and the small changes were not just random fluctuations. For some businesses, the MDE can be as little as 1% or 2%, which often means a test requires a sample size in the millions.

Personalisation

Personalization is the technique of tailoring an experience to a specific individual or group of similar individuals based on demographic or behaviour-based data, such as the number of site visits, page viewing behavior, purchase history, or social media interaction.

An example would be Amazon’s ‘recommended for you’ section. Amazon analyses the combination of products you’ve looked at or bought on their site and then suggests other products you might also be interested in. This is an example of ‘behavioral personalisation’ and is just one of a broad range of techniques used to improve the online experience.

Attribution

In the context of marketing, attribution refers to the process of identifying which events or touchpoints contributed towards a desired outcome or conversion and then awarding them appropriate credit for their role in the outcome. There are different types of attribution model that assign credit in different ways. The key to using any attribution model is that it is applied consistently.

Conversion funnel

The conversion funnel is the set of steps that a website visitor completes in order to achieve the desired outcome. We most often refer to the conversion funnel of the primary conversion goal whether that is a sale, a new lead generated, etc. In analytics tools, we can look at using funnel visualization reports to measure progressions through the conversion funnel.

Wireframing

A website wireframe is a simplified, lo-fi visual representation of a web page. Starting your design process with a simple wireframe allows you to arrange important functionality and content in a structured way and illicit feedback at this level before colour, graphics and styling are applied.

This process helps to focus on designing an experience that will work well for the user and help the business achieve its online goals without starting debates about minor styling issues

Macro-conversion (primary conversion metric)

Macro-conversion is typically your key overarching conversion goal. In terms of conversion rate, it is likely to be your broadest onsite conversion for your most important goal. For an eCommerce site it would be the conversion rate of all visitors to purchase complete. When we refer to macro-conversion we are not specifically interested in the steps or stages that users go through to achieve the conversion.

Micro-conversion (secondary conversion metric)

A micro-conversion is an action or conversion on your website that is important and worthy of measurement, but less significant than your overarching primary goals or perhaps a step in the journey toward your [macro-conversion].

For example, on an eCommerce site, you might decide that configuring a goal for ‘Add to Basket’ is a micro-conversion that indicates progress through the user-journey and may be a sign of increased intent to purchase, but it is not as important as your macro conversion which would be ‘Completed Transactions’.

UCD (in a digital context)

User-centered design (UCD) refers to a set of processes, which ensure that the needs, wants and limitations of the end user of our websites and interfaces are considered at each stage of the design process. This will include a wide range of research techniques with real-world users such as surveys, user interviews, user testing, and sometimes A/B testing. The aim of this approach is to reduce the number of assumptions made by designers and gather feedback throughout the process to tailor the website or interfaces for the end user.

Agility (in a CRO context)

In a CRO context, when people talk about agility they are usually referring to either the speed or flexibility with which a team is able to deploy A/B & MVT tests.

User Research

User Research focuses on understanding user behaviors, needs, and motivations through observation techniques, task analysis, and other feedback methodologies. This field of research aims at improving the usability of products by incorporating experimental and observational research methods to guide the design, development, and refinement of a product. User Researchers often work alongside designers, engineers, and programmers in all stages …

User research is an iterative, cyclical process in which observation identifies a problem space for which solutions are proposed. From these proposals, design solutions are prototyped and then tested with the target user group. This process is repeated as many times as necessary.

P.E.T

P.E.T. is a framework for user experience improvement that reviews a website or interface for Persuasion, Emotion, and Trust. It is a system developed by Human Factors Inc. to help support the developing information ecosystem.

QA (Quality Assurance)

Quality Assurance (QA) happens at a number of stages in our design and build process where we review progress and look to prevent issues. Before launching any A/B Test we go through an internal QA process that ensures that we have a robust experiment that will not have any negative impact on the users or their ability to complete their goals online.

Personas

Personas are fictional characters designed to allow a business to focus its marketing media and online experiences to a limited number of key audiences and with defined behaviours, goals, or objectives. Good personas should be based on market segmentation strategy and created based on quantitative data and qualitative insights.

User journey

A term used in user experience circles to describe either the entire website and the set of actions that a user can carry out or sometimes used in a more specific way to describe specific journeys, for example, the purchase user journey. They describe at a high level the steps that users take to navigate through your website.

Heuristic evaluation

A heuristic evaluation is when we use recognized usability principles and standards to evaluate the user experience of our websites. We have developed a 50+ set of usability guidelines that we use to evaluate websites. Best practice guidelines on their own, but in combination with analytics data and primary user research, a heuristic evaluation can highlight or validate insights from other sources.

Uplift

Uplift is simply used when a metric has improved over a defined period. For example, a winning A/B test could be said to have created an uplift of X%.

Data Science

Data Science is an interdisciplinary field about processes and systems to extract knowledge or insights from data in various forms, either structured or unstructured, which is a continuation of some of the data analysis fields such as statistics, data mining, and predictive analytics, similar to Knowledge Discovery in Databases (KDD).

Persuasion

Persuasion is a key tool in the toolbox of any serious CRO professional. We use persuasion to align our website experiences with and influence a person’s attitudes, behaviours, beliefs, intentions, or motivations. Persuasive techniques can be applied to appeal to the user’s logic & reason or habits & emotions. We use a wide range of persuasive techniques in order to create engaging and compelling websites that benefit the user and our business goals.

Social Proof

Social Proof is a psychological phenomenon where people are influenced by the behaviour of others in a given situation. In the context of CRO, it is often used as a persuasive technique to highlight the behaviour of other visitors who are not seen. For example, highlighting a large number of satisfied customers or subscribers may be influential in a user’s decision-making process about a product or service. Ratings and Reviews can also model broader user behaviour.

Iterative tests

Tests that do not include any major innovation, that often looks to refine or improve existing features may be referred to as iterative tests. When scheduling experiments, there may be benefits in terms of simplicity and speed to implement iterative changes, but their impact may be more limited or rare with iterative experiments.

Innovative experiments

Innovative experiments tend to be more radical in nature than iterative experiments. It is more likely that they introduce new elements or larger scale redesigns of a page or interface. They are more likely to have a significant impact, however, they tend to be more costly in terms of effort and time to deploy.

Call to Action (CTA)

The Call to Action is the main action that you want users to take on your page. This may be the ‘Add to Basket’ button or a ‘Form Submit’ button. The copy that you use on your Call to Action can be influential and is something that can be tested to good effect.

Landing Page Optimisation

Landing Page Optimisation refers to the specific activity of conducting A/B & MVT tests on landing pages, usually with a view to increasing conversions and ultimately increasing the efficiency of the advertising spend used to drive traffic to the page. This may include “testing call to action” copy and design, copy, content, promotions, relevancy of categories, imagery and many other factors.

Visitor segmentation

Visitor segmentation is the process of using data to define useful segments from your board visitor base into subsets of customers who may behave in similar or more predictable ways. It might be that mobile users behave differently to desktop visitors or perhaps free and premium subscribers use your website in distinct ways.

These segments can be used and in A/B testing and over time you may be able to optimise your experience for a number of different segments.

Test prioritisation

After reviewing user behaviour in analytics, conducting a conversion evaluation, carrying out user research, and collecting internal ideas, we typically end up with a long list of potential test hypotheses. It then becomes extremely important that we effectively priorities tests that are most likely to deliver a positive impact for a reasonable amount of effort. Using a rating system is a common way to prior itise tests.