Quote and Buy CRO testing
Client: EDF Energy
Jan - July 2021
Objectives
Problem
The Quote and Buy journey had been overlooked; to the extent pattern libraries were two versions out of date. The Sales and Marketing team weren’t sure whether improvements or a full re-design were required. I proactively reached out to offer a heuristic review alongside validating CRO tests, with the aim of identifying the main problems and providing data to offer insights to aid their decision.
Deliverables
To supporting the re-design with data, forming hypotheses from discovery and conducting CRO tests.
Team
Solo project with weekly reporting, input and updates to/from the Sales and Marketing team.
Discovery
Before CRO testing I undertook discovery work including:
1. A heuristic review to understand the current journey and how this follow best practice.
2. Analytics to understand user behaviour, journey funnels, returning customers and more.
3. Competitive benchmarking to understand conventions which should be followed.
4. Results from previous CRO tests to see if learnings could be carried over.
5. Data Verification experiment to understand CTRs on each page element, and learn the most popular selections during the journey when the user is selected with options.
Outcome
The Sales and Marketing team advised the two below tests saved the company ~£80k in drop-off per month.
Unfortunately I don’t have the financial gains of the further tariff results page testing.
Please note: all test results are to statistical significance.
#1 Software being confident
Background
Due to technical / business reasons, by this stage in the journey (page 3), users were asked three times, on three separate occasions to enter their address and check whether their address was correct.
Hypothesis
Displaying the address the third time as a statement rather than a question, and removing the explanatory paragraph, will help a user progress through the journey by employing Hicks Law and taking away a thought process for the user. Due to the current drop-off analytics we have on this step, we predict this will increase progress in the journey by ~40%.
Results
+51%
Increase onto the next question
The confidence the website has now given +51% of users confidence to proceed to the next question (with statistical significance).
-27%
Reduction in clicks to change the users address
More analytics are required to understand whether this is a positive or negative insight. By this stage in the journey the user has already been asked to enter their address and confirm their address on two separate pages so shouldn’t need to change their address by this stage.
#2 Identifying accessibility issues
Background
On the ‘details’ page of the Quote and Buy journey the first question a user is asked is, ‘what they would like a quote for’ - dual fuel, electricity or gas. This defaults to the fuel type the user has which is based on their address and an ECOES system which matches their information.
Hypothesis
From the heuristic review I conducted it was found that users with poor vision or colour blindness will find it difficult to differentiate between these colours (green and grey), particularly the similar shades. Testing this section in the ‘blue’ pattern library which appears further down the same page shows clearer highlighting will be more accessible to a larger audience. Due to the current drop-off analytics we have on this step, we predict this will increase progress in the journey by ~40%.
Results
+51%
Average increase onto the following 3 pages
Although the further away from the signal, the more noise needs to be considered within the results, there was an uplift of ~51% average increase on the following three pages of the journey.
-27%
Reduction in clicks on the defaulted selections across all traffic
Users can now see when an button has been pre-selected to their household needs and are not trying to click.
Tariff results page
Background
As part of the Quote and Buy project, particular focus was highlighted on the tariff page, where there was a 70% drop-off.
Re-design of the tariff page
The first step I took was a re-design of the tariff results page. Each element on the page was thought about in relation to the goal: helping a user choose a tariff, therefore decreasing drop-offs and increasing conversions. It was decided that a re-design was required rather than iterating the current design. The result was the tariff results being brought ‘above the fold’ on desktop, many elements being removed, and an 11.7% increase.
Further improvements - tariff CTA wording
Multiple CTA copy tests were undertaken on this page, it was found ‘exploratory CTA’s’ increased CTR more than ‘acquisition’ style CTA copy. These received statistical significance and has inspired further tests to take place on the Home Move page to which saw the same results.
The first test had ‘acquisition labels’:
Choose this tariff (control)
- Select this tariff (-44.27% CTR)
- Choose tariff (-39.4% CTR)
- Get this tariff (-41.93% CTR)
- Buy this tariff (-53.72% CTR)
The second tested exploratory labels:
Choose this tariff (control)
- More details (+13.15% CTR)
- Tell me more (-2.78% CTR)
- I’m interested (-26.17% CTR)
- Explore (-2% CTR)
- Explore this tariff (+0.97% CTR)
- Get full details (+5.38% CTR)
The third test had a mix of labels, from social belonging ‘Join EDF’, to acquisition and exploratory. This was tested on the Home Move journey to see if the change of audience for a similar product would yield different results. The results were the same.
Choose this tariff (control)
- More details (+13.91 CTR)
- Sign me up (-9.72% CTR)
- Get this tariff (-12.95% CTR)
- Join EDF (-5.78% CTR)
- Get full details (+17.88% CTR)
The fourth and final test was conducted when there was only one tariff available, due to the onset of the Energy Crisis. Due to this change in market circumstance, no further tests were made, soon after this test was conducted the digital journeys were turned off until the market re-opened.
Choose this tariff (control)
Next (+194.75% CTR)
Continue (+125.9% CTR)
Join EDF (+65.93% CTR)
Join (+44.07% CTR)