SMS Testing

Timing

The purpose: This was a proactive initiative as timing had never been tested for SMS lifecycles. I started with Abandon lifecycles (Browse, Cart, Checkout) since they are the top-performing ones outside of Welcome which includes a coupon.

The strategy: Create an A/B test to test A (the existing timing, different for each lifecycle) against B (updated timing, 15min sooner for each lifecycle). The length of the test would be until we reached statistically significant results, or after 3 months.

The results:

• 19% increase in uCTR
• 19% increase in last click revenue

TryNow CTA

The purpose: TryNow was a recently implemented try-before-you-buy program. It was a great new offering to test in our Abandon lifecycles for eligible products.

The strategy: A/B test the value of try-before-you-buy language in the CTA. The length of the test would be until we reached statistically significant results, or after 3 months.

The results:

• 65.8% increase in CVR for TryNow products in Cart Abandon comms
• 97.7% increase in CVR in Browse Abandon comms

Media

The goal: In 2021, 56% of program spend was from MMS. The cost of including images was huge and I wanted to understand the value.

The strategy: Over the course of three months, implement A/B testing of Media vs. No Media. KPIs would focus on uCTR and CVR. This was divided into two categories: The Farm products and Fedora products.

The results: None of the tests reached statistically significant results! The use of imagery in texts was largely scaled back and by the end of 2022 the program had:

• 301X ROI (up 45% YoY)
• Reduced MMS cost by 59%