5 Product learnings from running test projects at Treatwell — inspired by #JamConf2018

Did you make #JAMConf2018 this year? If not, how could you? You can read a bit about it here and of course, buy your tickets for next year.

The Treatwell product team all attended for the 2nd year running (for some of us, it was the 4th year in a row) and as usual, we were not disappointed. In particular, we loved Tom May’s talk, entitled: “Test and Delivery: achieving speed and quality at Thread”.

“Tests are temporary. We are comfortable removing the code from a test project when it’s completed. It might sound extreme & it may cost more time in completing the delivery project, but that time is less than trying to control the scope of a test project.” @thjmay #JAMConf18

The talk summarised the difficult balance between speed and quality when it comes to product development. Using a recent discovery project to improve the onboarding registration flow, Tom described how the team at Thread re-thought their process for discovery and delivery projects. The main principle is that a project progresses to delivery only once it has demonstrated success during an initial test phase. The test code can be completely removed if the test is deemed unsuccessful.

So we wanted to expand upon Tom’s talk by sharing our own learnings from a recent B2B discovery project here at Treatwell: a brand new marketing tool for our salon partners to send tailored marketing campaign emails to their clients.

To gain the desired insights (more on this later), we decided to release this feature to all our partner users at once, meaning that we couldn’t compromise on quality (too much). But we also needed to optimise for speed (of delivery) in order to get the feature out with enough time to make it relevant (see why below).

Sound interesting? Then read on. This should be relevant for anyone working for a scaling organisation, looking to run test projects with quality at the core.

The project: a brand new salon marketing & CRM tool

The business context

Our Hair and Beauty salon partners are undoubtedly beauty experts but when it comes to marketing to their customers, by their own admission, they struggle. New salon businesses rely heavily on Treatwell to attract new customers. Once established, retaining and communicating to existing customers and getting them to rebook online becomes more challenging. Believe it or not, salons still rely heavily on the phone, SMS, Whatsapp and personal email. As consumers, think back to the days where you had to pick up a phone to book a taxi or manage a hotel room booking. In today’s world, this is hard to imagine, right?

Some partners do use more advanced marketing / CRM tools (including our own Connect CRM) but when they do, the content of the email is usually not relevant, not very timely and not very noticeable. It can even contain spelling mistakes. Sadly, it often ends up in spam folders.

In terms of metrics, we see low partner adoption/usage rates (with Connect CRM) and low customer engagement (measured by email open rates and click-through to conversion).

The hypothesis

We believed that by providing standardised, marketing campaign templates with content carefully curated for each campaign, smart audience segmentation and timing of send recommendation would drive up salon partner adoption and customer engagement.

The constraints

We knew that building a fully-fledged, fully automated, fire and forget marketing tool (like MailChimp, for example) would take a dedicated product team months (even years) to develop in-house. So before committing to such a large delivery project, we wanted to validate our hypothesis.

The test project

Hair and Beauty is very seasonal (hair colouring before summer, hair removal during summer and so on) so we knew that the Christmas holidays would be the perfect opportunity to try our first dedicated email campaign: a festive wishes campaign encouraging customers to check availability and book online, sent from Treatwell on our partner’s behalf. Partners would be given a limited time window before the end of the year to choose, edit (optionally) and send this campaign email.

The new marketing tool

The results

Fortunately, the experiment delivered successful results. 50% of our most engaged segment of salon partners interacted with the tool within a week, with around 30% of them converting (measured by email sent per partner). The overall resulting impact was double the number of partner emails sent from Connect CRM, year on year (YoY). On the consumer side, we saw 4 times the click-through rates resulting in a significant increase in orders.

In terms of effort and cost, the product team with limited (marketing) domain knowledge were able to deliver the test project iteratively in double quick time.

So here are 5 main learnings that we took away from this discovery project.

The learnings: 5 product lessons from this test

1. Maximise non-tech resources to launch faster

Wherever possible, look to utilise non-tech resources in order to develop and launch quickly. At project kick off, coming up with ways to utilise non-tech resources is one of the most important questions we ask ourselves for any of our projects.

In our example, we consulted and collaborated with our internal marketing team and used Intercom and Typeform (relatively new tools for us) to promote the new feature and gather feedback from users. We also spent time on education and internal PR so that our front-line teams would do a good job promoting the feature on our behalf.

2. Do your homework to increase chances of success

Calling a project a test project does not give you a free license to fail. In fact, with a strong discovery culture comes even more responsibility to pick good hypotheses and execute well.

This is why we did (and always strive to do) proper project inception before we commit to writing even 1 line of code. By analysing our survey data and cross-referencing it with 1 year’s worth of Connect CRM email send data, we were able to categorise emails into different use cases and prioritise these to ultimately come up with a list of most wanted campaigns. We also researched the market and competitors to develop an interactive product prototype to test with a few salon partners, to get some early validation on the problem and proposed solution.