Video-First Database Framework

A framework for building consumer apps using raw video as the primary data source, with AI extracting structured metadata on demand. This represents a fundamental shift from traditional database-driven applications.

Core Concept

• Traditional apps store canonical structured data (text, metadata) with media as attachments
• New approach: Store raw video/media as primary data, use AI to extract structured data on-demand
• AI can reprocess historical video content to extract new types of metadata without recollecting data

Key Benefits

• More flexible data model
  • No need to predefine all data fields
  • Can extract new types of metadata retroactively
  • Raw video contains richer information than structured text
• Real-time data extraction
  • AI can analyze videos to answer specific queries
  • Don't need to store pre-computed metadata
  • Can respond to novel data requests using existing content

Example Application: Restaurant Reviews

• Traditional (Yelp):

  • Structured database rows with facts about restaurants
  • Photos/videos attached as supplementary content
  • Fixed schema of metadata (hours, location, etc.)

• Video-First Approach:

  • Store raw video reviews of restaurant experiences
  • AI watches videos to extract relevant details
  • Can answer new types of queries (e.g. noise levels) by reanalyzing videos
  • More natural user experience starting with camera open

Implementation Considerations

• Experience should be camera-first like Snapchat/TikTok
• AI needs to deeply understand and index video content
• System must handle real-time AI analysis of video content
• Balance between pre-computed and on-demand metadata extraction

Potential Impact

• Could disrupt incumbent apps by making their structured data less valuable
• Enables more natural/authentic content creation
• Makes apps more adaptable to new types of queries/use cases
• May dramatically expand certain market segments