Innovative Thinking

Meet Wasabi, an Open Source A/B Testing Platform

Intuit is open sourcing Wasabi, a real-time, enterprise-grade A/B Testing platform, battle-tested in FinTech products, servicing over 120 apps across web, mobile, and desktop. Data Whether you’re a startup or a Fortune 500 company, data is an enabler, a forensics device, precious fuel for user experiences, and often a product in its own right. Without

Written by Lucian Lita
Published Sep 29, 2017 - [Updated Nov 9, 2022]
4 min read

Intuit is open sourcing Wasabi, a real-time, enterprise-grade A/B Testing platform, battle-tested in FinTech products, servicing over 120 apps across web, mobile, and desktop.

Data

Whether you’re a startup or a Fortune 500 company, data is an enabler, a forensics device, precious fuel for user experiences, and often a product in its own right. Without data, it’s easy to make uninformed business decisions, misinterpret your users’ needs, and miss many opportunities to improve your products.

A/B Testing Today

For all the talk around data and experimentation, most companies that I’ve interacted with, to this day don’t use A/B Testing, but rather rely solely on gut feel and intuition to make product decisions. The ones that do use A/B Testing, often either build home-grown solutions that do not receive the necessary love and attention, or rely on costly, less flexible commercial 3rd party services. Companies, especially in domains such as Healthcare or FinTech, must protect their customers’ data (e.g. electronic medical records, tax returns, etc.) with an eye towards privacy and compliance, but still need reliable, scalable A/B Testing.

Meet Wasabi

Wasabi is a real-time, enterprise-grade, 100% API driven, A/B Testing platform open sourced by Intuit. With Wasabi, you own your own data and experiment across web, mobile, and desktop; back-end and front-end; product and marketing. It’s fast, easy to use, cheap and scalable. It’s chock full of features and instrumentation is minimal.

Here are further details on Wasabi’s features:

Own your own data: Wasabi protects your users’ data and maintain your competitive advantage at the same time. You don’t have to ship your sensitive data to 3rd party services for decisions, segmentation, etc.
Easy to run and instrument: Wasabi is a breeze to operate, scale, deploy, and maintain. You can spin up your Wasabi docker in 5 minutes and deploy it in distributed production mode within a day. Once it’s up, you can instrument your code in any language and environment, since it is 100% API driven.
Easy to create and manage experiments: Wasabi comes with a flexible UI for experiment management, making it easy for engineers, product managers, or analysts to hit the ground running. It provides real-time analytics and metrics visualization out of the box.
Real-time for in-product use: Wasabi has a consistent performance of 30ms server side.
Enterprise-grade: Wasabi is HA and has been battle tested in production for 2 years in TurboTax, QuickBooks, Mint.com, etc., powering over 1,000 experiments. Can be deployed either in the cloud or on-premise.
Multi-environment: Wasabi gives you a uniform, consistent experience across web, mobile, desktop, and also front-end, back-end. You choose what type of IDs drive individual experiments: 1st party, cookie IDs, 3rd party, etc.
Active audience-only (i.e. not wasteful): most other tools and 3rd party services pre-divide your audience into buckets/segments. Wasabi assigns users into experiments in real-time, only when they are actively exposed to a product experience.
Streaming integration with analytics & product: Wasabi has extendable hooks for streaming integration with any rest-based service. At Intuit we’ve used Apache Kafka to integrate Wasabi with the larger analytics ecosystem: Hadoop, MPPs, BI through Tableau, data manipulation via R and Python.

Why Intuit?

A while back we realized we need an experimentation platform: real-time, secure, reliable, and 100% API driven. It had to be able to go across multiple apps in web, mobile, and desktop, it had to span entire ecosystems, from small business to personal finance, and it had to keep sensitive data in-house. Such a platform did not exist. At the time Intuit had many fragmented, low-volume, and over-specialized A/B Testing tools and services, which made it difficult to scale and achieve consistency .

We built Wasabi and never looked back. It has already made a huge difference within Intuit: we now have a solid A/B Testing platform has scaled to over 120 apps, with over 3BN API calls for the past tax season alone, and running over 850 active experiments. All in the service of making data-driven product decisions consistently and at scale.

Why open source?

At meetups and conferences we learned that we are not the only company that needs to own its data, keep costs under control, and power front-end and back-end in-product experiments. With this in mind, almost a year ago, we co-founded the A/B Testing meetup. Amazingly, it grew to over 1,000 members and a recent session, held at Intuit, was attended by close to 300 people . Clearly we needed to do more. We made a decision to open source our platform, Wasabi, so we contribute to the broader community and make a difference.

Get started!

Our team (data@intuit.com) will gladly give you a demo and if needed, even provide guidance on deploying Wasabi. We also hold both online and in-person office hours. Wasabi will continue to evolve, so become a contributorand grow with us!

I’m proud of the team for all the effort and passion they’ve put into this open source project. We loved building Wasabi and hope you’ll love using it and make data driven product and business decisions.

A/B Test All the Things!!