HPE · UX Designer · 2020
Four category boxes on hpe.com all opened the same six-item carousel. I rebuilt the section on the design system’s tabbed module and tested it against the original on live traffic.
Clicks into the section and into its CTAs, tabs against the live carousel.
+99.79%
rate lift · tabbed
99.92%
confidence
3.45%
conversion rate
A/B test · session 2 · 8 Jan 2021 – 8 Feb 2021
Context
HPE’s Machine Learning Ops page sold a container-based platform to enterprise engineering buyers, live on hpe.com and carrying real traffic. My week on the personalization team ran on a loop: pull the analytics for a slice of the domain, find a section underperforming, test a replacement against it.
Two constraints came with that. Anything I proposed had to be built from components the UX team had already shipped, and it had to beat the design already there in a split test.
The mess
Four boxes sat under the section heading. Click one and a six-item carousel opened: the same six items, in the same order, whichever box you clicked, always parked on the first.
Visitors were doing their half. The boxes got clicked and ranked well against everything else on the page. What nobody did was scroll. Somebody who wanted Model Training clicked Model Training, landed on Model Build, and stopped.
Nothing was missing from that page. Every video and description was already on it, one click and an unsignposted scroll away, behind a control whose shape promised the click was enough.
What I did
The UX team had just put a tabbed module into the design system and nothing on the site was using it. I rebuilt the section on it: six tabs, one panel, the panel changes when the tab does. No new component to get approved, no content rewritten, no scroll to guess at. The click finally returns what the label advertised. I drew every breakpoint, because the module had never been proven at any of them.
Laying everything out with no clicks had beaten hidden content on other HPE tests, and it is what the GreenLake page does. Run tabs against the carousel alone and tabs could win for the wrong reason, because anything beats a broken control. So the third variant expanded all six sections at once, and forced the test to choose between two real ideas.
Evidence
The hypothesis, written down before anything was designed: people click one box expecting that box’s content, get a carousel of everything instead, and give up. Make the selection do what its shape promises and click-through goes up.
An A/B preference test on live traffic. Three variants (the carousel, the tabs, everything expanded) scored on clicks into the components and into their CTAs.
| Session | Variant | Conversion rate | Rate lift | Confidence |
|---|---|---|---|---|
| 01 | Everything expanded | 0.57% | -87.83% | 100% |
| 01 won | Tabbed | 7.48% | +62.15% | 96.77% |
| 02 won | Tabbed | 3.45% | +99.79% | 99.92% |
A/B preference test · three variants cut to two · session 1 from 8 Dec 2020 · session 2 8 Jan 2021 – 8 Feb 2021 · 50% of traffic
What shipped
Tabs won both sessions, so I took the numbers to the UX and data teams rather than filing them with my own. The designer who owns the ML Ops page left the test running at 50% and pulled the tabbed module into the redesign he was already scoping.
The results went up to our managers, David and Matt, and to Gabie Boko, VP of the digital team. What came back was a request to make tabs the default treatment on newly designed content-heavy pages.
I did not want a domain-wide rule resting on one section of one page, where layout was not the only thing that differed between variants. So the next test moved the tabbed module onto another high-traffic page and ran it on its own, to find out whether the result was about tabs or about this carousel.
Still open
Two things I cannot claim. The tabbed variant’s own conversion rate fell from 7.48% to 3.45% between sessions. The bigger lift is measured against a different baseline in a different month, and I never ran a clean holdout to separate design from season. And at the last measurement the test was still live at 50%: tabs were winning on that page, not shipped as a site-wide pattern.
Nothing was missing from that page. The control just did not behave the way it looked.
