Inbox Pirates

Inbox Pirates

How email testing tool Inbox Pirates increased onboarding completions to 82.05%

What is Inbox Pirates?

Inbox Pirates is an email preview testing tool. Bharadwaj Giridhar, Founder & CEO, believes that the current email preview tools add to the ego-centric bias of marketers, as they make it appear like the recipient has only one email in their inbox, or they are reading the email in full-screen with full attention. To fix this problem, he built Inbox Pirates.

Through Inbox Pirates, mail marketers and cold emailing sales teams can preview, test, and analyze their emails on different platforms, devices, and even notification bars to increase open rates and fix email bugs. 

Microsoft Clarity has been a vital tool for them in understanding their customer journey.

The Challenge

Inbox Pirates was facing several challenges and needed a way to gather insights into customer behavior to decide what changes should be made.

  1. Customers were signing up but not finishing onboarding.
  2. Visitors watched the demo video and took the live interactive demo but failed to signup.
  3. Signed up users completed onboarding but didn’t generate their first preview.

They were able to utilize Microsoft Clarity to gain insights into how to correct these issues.

Learn how your customers navigate your site!

Through session recordings, heatmaps and much more!

Problem #1: Customers signed up but didn’t finish onboarding

Like any other tool, onboarding a customer on a self-serve basis for a new SaaS product is a huge challenge. 

In the beginning, Inbox Pirates events were captured with several analytical tools, and they were surprised to see 1 out of every 2 customers had skipped the onboarding process.

This was crucial for two reasons. The onboarding flow was where they got to know the name and goals of customers, and without this information, even if the tool functions, it’s hard to improve it without customer feedback.

Solution #1

Bharadwaj emphasizes, “If you can’t measure it, you can’t improve it”. Inbox Pirates uses a wide variety of tools, including Google Analytics, Posthog, Splitbee, Amplitude, and Crewcharge’s internal tool to measure onboarding as well as homepage signups. However, it was Clarity that was able to provide a detailed picture of the problem.

The onboarding issue was initially dismissed as the lack of interest by certain users, but as the sample size grew larger than 100, they started to get concerned. 

Viewing the Session Recordings of users made it clear that since the onboarding flow was built as a popup that could be dismissed, many of users thought it was optional. 

Action Taken: 

Discovering this just one day before their Product Hunt launch, they temporarily fixed it by making the popup non-dismissible. A few days after the launch, moving the onboarding onto a separate screen increased onboarding by 42.05%, meaning every customer who signed up now completed this step.

With the help of Clarity, they understood that users get accustomed to patterns like popups on other sites and see them as obstructions, while they initially thought it was a way to reduce friction.

Problem #2: Visitors watched the demo video & took the live interactive demo but failed to signup

If a visitor spent more than 3-5 minutes on their site, played their demo video and took the live demo, they assumed they were likely to signup. 

However, after filtering visitors using Clarity to see who performed these clicks, it was clear to Inbox Pirates that the people who didn’t take the demo were more likely to signup. 

Solution #2

The hypothesis then was that the people who signed up from the homepage had social proof and trust badges, while the people who took the demo (and technically invested more time) had just a blank button. 

Action Taken: 

Inbox Pirates ran an A/B split test. They hid the social proof (Trustpilot, G2, & Chrome store reviews) for half and showed it for the remaining. 

This increased signup rates. It also revealed that people trusted long-form testimonials over short two line testimonials.  

Inbox Pirates was able to use Clarity recordings to see how users navigated through the customer journey given the split tests.

Problem #3: Signed up users completed onboarding but didn’t generate their first preview

After being #1 product of the day on Product Hunt, they thought the 150+ new signups they received from the 873+ visitors meant a lot. While the onboarding problem was fixed, they found out that people were uninstalling the tool shortly after. This was expected since launch sites like Product Hunt, and Hackernews have massive drop-off rates as high as 60%.

However, they couldn’t get over the fact that the drop-off rate was that massive. 

Solution #3

After viewing more than 100 of the 683 videos recorded by Clarity, one thing was clear: customers hated installing the Chrome extension right away. 

Action Taken: 

They are now slowly rolling out a new “how-to-use” flow that does not require visitors to install the extension right away to use the product. They strongly believe this increases adoption. 

“Clarity proved that closing the popup could have been tracked as a Google Analytics event or knowing certain things cause friction with certain demographic users (for example, people who held a technical role had no issues installing the extension, while several marketers shied away from it) is possible only in hindsight.”

However, by utilizing Clarity, Inbox Pirates could directly see what changes needed to be made.

Outcomes & Shareable Learnings

After analyzing the customer journey using Clarity, Inbox Pirates saw some considerable improvements to their customer onboarding process. Let’s see how things turned out.

They saw an increase in onboarding (goal selection, country, usage description) by 42.05% the day after mandatory popups rolled out.

As you can see, the results of making the onboarding popup non-dismissible when analyzed across a cohort of 2-week users before and after July-18 (The day the update was rolled out).

This didn’t increase drop-off rates as they suspected but actually increased onboarding completions to a whopping 82.05% (from 40%) giving them a 42.05% boost! 

Shareable learnings: 

  1. Users will see only patterns among websites to get to ideal actions. Most of the time, they don’t read or even skim the text on buttons.
  2. Ensure there’s only one ideal action on a page if you want them to complete it. 
  3. Assume most people aren’t interested in using the product while designing it. With this attitude, you’ll design products that excite them to complete their ideal path instead of assuming they will fall in love immediately.   

Overall, Clarity helped diagnose key problems which would otherwise have been elusive and hard to solve. Session Recordings not only made that process simple, but also allowed Inbox Pirates to quickly make changes at the right time to improve onboarding and customer journey.

While tools like Google Analytics & Amplitude can tell you what your users are doing, only Clarity can help you figure out what your users are not doing, which is extremely valuable information.

-Bharadwaj Giridhar, Founder & CEO


Love using Clarity? Share your story with us!