Lyft’s mission: “Improve peoples lives with the world’s best transportation”

Lyft’s rideshare ecosystem is a two-sided marketplace consisting of drivers and riders. To deliver on the company mission, we helped Growth Teams in acquiring and retaining riders and drivers on Lyft’s platform.

Incentives, what’s the point?

One of the most fundamental methods to grow our platform and introduce new users to our products was through incentives. We used a variety of diverse incentives to tackle acquisition, engagement, product discovery, and remediation. Let’s look at specific examples:

• A new rider who has never taken a ride: They may be dissuaded by an unfamiliar product that they’ve never used before. Sending a small ride credit incentive in conjunction with thoughtful communication to encourage them to try out our platform and feel comfortable while doing it. We firmly believe in our mission to build the world’s best transportation, and part of that is making our product accessible to everyone.
• A driver who had a bad experience on the app, like a no-show rider: We want to acknowledge the experience and be empathetic to the situation. We can send them a message apologizing for the situation, along with a bonus incentive — it’s important to let our drivers know that they are a tremendously valued part of what we are building.

What is an incentive campaign? What are the functional blocks?

For incentives to be effective — Timing and communication is important

For incentive programs to be impactful — Scale and reliability is key

Let’s consider a use case “Increase the # of riders taking their first ride(Activation)” — This entails motivating signed up users to take their first ride by leveraging incentives.

Hypothesis/Objective

The Growth team was focused on scaling the business with the right product fit. At Lyft, we constantly ideated and tested new hypotheses.

Coming up with a hypothesis can be challenging and isn’t an exact science. Usually we start with historical data and experiments that have been run — analyze the results and find trends that have worked or areas that need exploration.

Let’s assume “ride credit”, along with the right communication, can encourage new riders to try out Lyft. Let’s put together a hypothesis —

“Giving a credit incentive accompanied by welcome communication will motivate users to take their first ride with Lyft.”

Target Audience

Picking the right audience is crucial for the success of an incentive program. There are many dimensions on which you can split the audience but granularity depends on the objective — If we want to explore how an incentive performs with different types of people, we will cast a wider net.

In this particular case we are more objective driven and will target riders that are new to Lyft and haven’t taken their first ride. We will further narrow this down by region, to ensure we are spending money in the right markets, where there is a lack of demand.

Incentive Inventory

Besides picking the right audience, it is important to pick an incentive that will help achieve the objective — “Increase the # of riders taking their first ride(Activation)”. Based on historical data, we evaluate the performance of the incentive on metrics that affect the objective. We analyze cost curves to match the scale at which we want to run the campaign and configurations we want to run them in.

Incentives can be run in many different configurations:

Value: The amount of credit the user will receive

Redemption: The method of activating an incentive

Validity: The time period over which this will be valid

In this particular case, let’s use a $5 ride credit with a 10 day validity period.

Experimentation

Incrementality is a concept we want to measure — this helps us understand the effect of giving an incentive against not giving one. A/B testing is a way to measure incrementality. We randomly assign users into variants: Control (receive no incentive), and Treatment (receive an incentive). This helps us measure true incrementality by comparing the average behavior of each variant.

In this case the objective is clearly defined — “Increase the # of riders taking their first ride(Activation)” and we would still like to measure the impact of giving $5 ride credit. The objective can be measured using the metric incremental activations and efficiency with cost per incremental activation.

Campaign setup

Based on the objective we are trying to achieve, we determine a cadence at which we want to run the campaign(daily/weekly/monthly/one-time). The incentive is complemented with a clear and effective message to the user. There are multiple different channels that we can choose from — Email, SMS, Push Notifications, In-app communication.

In this case, since the users are new to Lyft we want to take a more traditional approach of leveraging email and SMS to communicate with them.

Why build a Platform?

In the first few years of running incentive campaigns, we took the path of least resistance and adopted a highly manual approach. This entailed creating an audience (usually through queries), training models, setting up communication channels, ensuring incentives were given for every single campaign. The results were great, we were now looking to test more hypotheses and onboard newer use cases, but with the current state this would become an operational overhead.

The only way to scale incentive programs was to automate the operations and generalize running incentive campaigns.

We decided to embark on a journey that would democratize all the technology, reduce operational costs and unlock high experiment velocity.

Thus, we decided to build Lyft’s incentives platform — a one stop shop to give users incentives, at Lyft.

What does the Platform look like?

Goals:

• Build a cohesive incentives platform that would allow teams to test and grow a hypothesis.
• Daily campaign runs, contiguous performance measurement, automation and tooling.
• Build Lyft’s repository of incentive ML algorithms to support optimizations.

The main difficulty with automation is that incentives and campaigns come in many shapes and sizes. Some teams were interested in using machine learning to predict and allocate incentives — so we had to build out capabilities to easily mix and match ML algorithms. Others were interested in testing out a completely new hypothesis with no prior data. One thing was certain — the ability to add new functionality and be flexible with data would have to be baked into the DNA of this architecture.

At first, our architecture consisted of a few integral pieces — each incentive campaign is a pipeline of Tasks, where each task is a computation jobwith defined inputsand outputs. These tasks involved gathering the eligible audience for each campaign, picking an allocation algorithm, and finally plugging into Lyft’s architecture to actually execute the incentive drop and send comms, essentially automating all the manual processes.

As stakeholders started heavily using our platform to drop incentives, we stumbled onto a few more challenges related to scale.

• Audience Overlap: Multiple incentive campaigns would target the same user, in some cases risking double or triple incentivizing the same user.
• Spend Budget: Managing spend across multiple incentive campaigns was difficult.
• Modular Execution: Every campaign had different requirements and needed manual intervention.

We realized that we had to centralize some of these core functions, and create some choke points which could allow us to control spend and prevent overlaps between audiences. We introduced 3 new components to our platform:

• Population management
• Campaign execution
• Campaign decision engine

These components are tied together with numerous datastores, microservices, and workflow orchestration engines which move mountains of data for the purpose of taking the goal of any incentive campaign and executing it.

Population Management

The first step of an automated incentive campaign is figuring out the audience for the incentive drop. We deployed an audience definition config, to describe eligible users, with extensive filtering capabilities on facts like “region” or aggregations like “rides in the last 10 days”. In addition to filtering we added the ability to reserve users, for a specific time period, for each campaign to prevent collisions.

Campaign A incentivizes users in New York who haven’t taken a ride in the last month, whilst Campaign B is focused on users across the US whose payment method recently expired.

A ∩ B = Users in New York who haven’t taken a ride in a while w/ recently expired payment methods.

If both campaigns were allowed to give users benefits, this would lead to experimentation bias; users in both groups would end up getting two incentives as opposed to one, leading to unclean experimentation results.

Thus, via the population management infrastructure, each campaign reserves their users for a few days and only after these tags expire do those users become re-eligible for a new incentive.

Campaign Execution

Campaign execution is responsible for running a pipeline of modular tasks to ensure delivery of incentives to users. At a high level we can break down the campaign execution into 3 distinct parts:

• Specification:Generate and persist a campaign configuration
• Orchestration: Enable the execution of different configurations
• Execution: Run all the task in the configuration

Much like the audience configuration, Specification consists of a campaign config for each campaign that contains meta information and specifics on tasks that are expected to run, experiment configuration, essentially any customizable details of the incentive campaign.

The orchestrator continuously checks and schedules tasks as required, translating the campaign config into API calls and computation jobs which will execute the tasks required to drop incentives.

These jobs execute on in-house Lyft infrastructure, including our internal ML platform, workflow engine, and other custom microservices.

Campaign Decision Engine

The campaign decision engine is the last step of each incentive campaign. This engine is responsible for enforcing global constraints — deduplication of audience across campaigns and budget guardrails.

For deduplication, it maintains state of actively participating users and leverages a priority algorithm to ensure no audience overlap. It also enforces global and regional budgets across all campaigns in conjunction with spend pacing — spreading the budget over a time period.

Once the final allocation is complete, the list is sent to other internal microservices to drop the incentives and trigger the communications that are associated with the campaign.

Conclusion

Lyft’s incentives platform now manages tens of incentive campaigns, runs hundreds of concurrent experiments, and manages millions of dollars in spend — all aiming to grow our transportation network and figure out the best way to provide value to users.

We’ve built the beginnings of a platform that can automate and optimize the incentive process. All cross-functional stakeholders now have the ability to take an incentive idea and run with it. We’ve removed the need for stakeholders to plug into operational details, reducing the mechanical work it took to get an idea off the ground — stakeholders can now just worry about the top line impact. This saves thousands of hours of time across the board.

While we’ve come a long way, scaling from double to triple digits of incentive campaigns on this platform is an exciting challenge we’re ready to continue tackling.

Acknowledgements

Huge shout out to the Incentives Platform team, working tirelessly to help deliver the right incentives to the right people at the right time.

Special thanks to Chirag Toprani for helping put this together.