Platform Metrics Mislead

Shaan Puri and Sam Parr discuss their skepticism about platform-reported metrics, particularly in podcasting, newsletters, and social media. They believe most numbers are manipulated vanity metrics that don't reflect true engagement or value.

Key Points:

• Newsletter Metrics:

  • Substack's growth numbers are artificially inflated through forced recommendations
  • When users subscribe to one newsletter, they're prompted to subscribe to 5 more
  • This creates false growth that doesn't reflect real reader engagement

• Podcast Download Numbers:

  • Many podcasters claim unrealistic listener numbers
  • Download numbers can be manipulated by:
    • Releasing high volume of short episodes
    • Auto-downloading to subscribed devices
    • Counting total downloads across all episodes instead of per-episode engagement

• True Engagement Metrics:

  • Real metrics should measure actual user engagement:
    • Newsletter replies and interaction rates
    • Ability to mobilize audience for in-person events
    • Percentage of users who actively use the product daily

• Examples of Manipulation:

  • A business podcast claiming 2 million listeners per episode (unrealistic for the category)
  • Podcasters using high episode volume to inflate monthly download numbers
  • Mobile apps using pre-checked contact lists to force viral growth

• Better Measurement Approaches:

  • Look for "true metrics" that show real engagement
  • Test audience engagement through:
    • Direct response to calls-to-action
    • Real-world meetup attendance
    • Reply rates to emails
    • Actual purchase behavior

• Platform Issues:

  • Most platforms provide the "most large vanity useless number"
  • Difficult to measure true engagement across different platforms
  • Need to find metrics that actually show "the level of love that your product has"