Unbounce - Behavior Sets

Behavior Sets provides a peek into high-performing (and low-performing) traffic segments. With data like this, marketers know where to focus thier efforts and drive the best results. 

Role UX, UI

Opportunity

On Unbounce’s mission to get our customers more conversions, we created Smart Traffic, an alternative to A/B testing for your landing pages. The Smart Traffic algorithm gets to know your visitors attributes, their device, browser, OS, location and time zone - then routes them to the page variant where people like them have converted in the past. We had observed that users were trying out Smart Traffic, but not continuing to use it. Our goal was to improve Smart Traffic retention as customer churn for Smart Traffic adopters is roughly half of what it is for our broader customer base. 

Audience

• Marketers, primarily working on in house teams
• Their skill level ranges from novice to expert marketer

Scope & Constraints

For this project, we worked with the data that was already being used by Smart Traffic, which included visitor attributes of device, browser, OS, location and time zone. We were also limited by our abilities to only report on certain time frames (i.e. since last page publish) as the Smart Traffic algorithm resets once a page change is made. 

Process

• We conducted a diary study to uncover the source or our user's pain
  • I partnered with a user researcher to conduct a diary study and learn more about the issues users were having. We discovered that a blocker for many was a lack of visibility into how Smart Traffic is making its decisions. Why did this matter? They use the learnings from each page to inform future campaigns, report to clients & inform the strategy of their business. Even if Smart Traffic was increasing their conversion rates, they needed to know where those conversion increases were coming from. “I need to know what’s working for whom. Not just "+49 conversions" but "version A is working best for mobile devices" - Unbounce Customer

• I facilitated a design sprint to hone in on possible design concepts
  • Our team of 10 spent 2 days digging into the problem space and learning about what was possible with our data systems. We developed a series of prototypes, which we then put in front of customers. 

• Conducted 15 Customer Value Testing sessions to refine the concept
  • Through 15 customer interviews we took the designs through 3 iterations landing on the concept that we ended up building in the end. In order to get accurate feedback, we used the real data of each customer we were interviewing. 

• Final UI Prototype

• Launched into Beta w/ Diary Study to gather live feedback and learned that:
  • Our report was taking too long to calculate, so users were bouncing before they had a chance to see the data - We adjusted when we calculate the report to improve load time
    
  • Customers were not finding the report organically - we added in notifications when there was new data so as to lead customers to the report
  • Some of the labels we used to describe locations were not clear enough - We adjusted the labels to better reflect what they were describing

• Usability Test
  • Post our learnings from the diary study & incoming feedback from our customer support teams we ran a usability test to validate that our improvements were taking us in the right direction. The study showed us that the usability of the feature was tracking well, and also hinted at some opportunities to broaden the kind of data that we use in our Smart Traffic algorithm. These findings will be incorporated into future iterations.

• Outcomes & Results
  • As of writing this case study, the project is still in beta, but early signals point to the promise of Behavior Sets encouraging even more users to give Smart Traffic a try.