Test Usage Not Words

A framework for validating customer interest through real behavior testing, based on Instacart's early experiences.

Core Testing Philosophy

• Don't trust what customers say they'll do
• Test actual usage behavior instead of verbal commitments
• Be willing to try multiple approaches until finding what works

Failed Early Experiments

• Gave away free cookies with orders
  • Cost thousands of dollars
  • Poor ROI compared to just giving cash incentives
  • Showed importance of testing different approaches

Successful Customer Acquisition Strategies

• Free delivery promotion

  • Offered for extended period
  • Removed friction for first-time users
  • Proved more effective than product giveaways

• Referral Program Structure

  • $10 credit for referring friend
  • $10 credit for friend who accepts
  • Leveraged natural word-of-mouth behavior
  • Remains one of top performing acquisition channels
  • Tapped into customer pride in using service

Key Principles for Customer Testing

• Focus on solving real customer problems
• Make decisions that benefit paying customers first
• Listen to customers rather than making assumptions
• Gather "ground truth" data on customer needs
• Test and iterate product based on actual usage data
• Accept that first version will likely be wrong
• Use scalable ways to collect customer feedback

Framework Results

• Led to building largest grocery catalog system
• Created novel solutions to complex industry problems
• Enabled real-time pricing and availability tracking
• Developed sophisticated data processing capabilities
• Built foundation for sustainable customer growth