Revolutionizing Video Metadata Management with AI: A DPG Media Case Study
This post was co-written with Lucas Desard, Tom Lauwers, and Sam Landuydt from DPG Media.
DPG Media is a powerhouse media company dominating the Benelux region with its array of online platforms and TV channels. The crown jewel in DPG Media’s empire is VTM GO, a platform overflowing with over 500 days of uninterrupted content.
As the library of long-form video content grows exponentially, DPG Media realizes the critical need for effectively managing and enriching video metadata. This includes crucial information like actor details, genre classification, episode summaries, video mood, and more. Descriptive metadata plays a pivotal role in providing accurate TV guide descriptions, refining content recommendations, and empowering consumers to explore content that resonates with their interests and emotional state.
Enter DPG Media’s innovative solution – the integration of AI-powered processes using Amazon Bedrock and Amazon Transcribe into their video publication pipelines. In just 4 weeks, DPG Media revolutionized their approach towards automated annotation systems.
The Challenge: Scaling Metadata Extraction and Generation
DPG Media faced the challenge of dealing with a plethora of video productions accompanied by diverse marketing materials with varying levels of quality and standardization. This necessitated manual screening processes to understand the content and generate missing metadata like summaries and subtitles. As the volume and variety of content exploded, DPG Media sought a scalable solution to capture metadata at a vast scale, enhancing the online consumer experience and cataloging content characteristics effectively.
The initial hurdles in automation journey included:
- Language diversity – Juggling Dutch, English, and local shows with Flemish dialects
- Variability in content volume – Ranging from single-episode films to multi-season series
- Release frequency – Daily influx of new shows, episodes, and movies
- Data aggregation – Ensuring reliable metadata aggregation across various seasons
Solution Overview
To address the automation challenges, DPG Media opted for a blend of AI techniques and existing metadata to create fresh, precise content descriptions, moods, and categories. They narrowed their focus on audio processing due to its cost-effectiveness and rapid processing time, eschewing the need for intensive video data analysis.
The metadata generation pipeline architecture consisted of two primary steps:
- Generate transcriptions of audio tracks: Leveraging speech recognition models for accurate audio content transcripts
- Generate metadata: Using Large Language Models (LLMs) to extract detailed metadata from transcriptions
Subsequent sections delve deeper into these pipeline components.
Step 1: Generate Audio Transcriptions
To acquire essential audio transcripts for metadata extraction, DPG Media evaluated two transcription strategies – Whisper-v3-large and Amazon Transcribe. Following meticulous evaluation based on price-performance and transcription quality metrics, Amazon Transcribe emerged as the optimal choice due to its accuracy, convenience, and automatic model updates.
The experiments confirmed high-quality transcription results without video incorporation or extensive speaker diarization. DPG Media’s decision reflected a strategic choice to harness Amazon Transcribe’s managed service for streamlined transcription needs and to focus on extracting precise metadata effectively.
Step 2: Generate Metadata
Armed with audio transcripts, DPG Media utilized Large Language Models (LLMs) via Amazon Bedrock to produce a myriad of metadata categories such as summaries, genre classifications, and key events. The selection of Anthropic Claude 3 Sonnet model and Hugging Face LMSYS Chatbot Arena Leaderboard via Amazon Bedrock exemplified meticulous internal testing and alignment with Dutch language performance benchmarks.
Post-metadata generation at the video level, DPG Media seamlessly aggregated metadata across entire series by feeding structured summaries back through Amazon Bedrock, ensuring quality consistency and cost-efficiency for extended series runs.
The powerful metadata generation solution embraced flexibility by storing direct association between metadata types and prompts, facilitating effortless tuning and evolution to meet evolving business requirements.
Results and Lessons Learned
The integration of AI-powered metadata pipeline heralded a transformative era for DPG Media, significantly reducing metadata generation labor for TV series. Amazon Transcribe and Anthropic Claude 3 Sonnet on Amazon Bedrock emerged as cornerstone technologies, reflecting DPG Media’s strategic balance between AI automation and human validation to ensure consumer-facing accuracy.
By diving into AI-driven processes, DPG Media fosters enhanced user experiences, elevates content recommendations, and propels towards automated, efficient annotation systems, aligning perfectly with modern viewing habits and technological advancements.
Conclusion
The journey of implementing AI-powered processes for metadata management at DPG Media showcases the power of technology in enhancing content experiences and driving operational efficiencies. By leveraging Amazon Bedrock and forward-thinking AI solutions, DPG Media has set a new benchmark in the media landscape, offering a roadmap for others to follow.
If you’re curious about unlocking AI potential for your business, explore Amazon Bedrock and embark on your transformative journey today.
About the Authors
Lucas Desard is GenAI Engineer at DPG Media, driving generative AI integration across diverse company processes.
Tom Lauwers is a machine learning engineer spearheading video personalization at DPG Media, enhancing viewer experiences across leading platforms.
Sam Landuydt is the Area Manager Recommendation & Search at DPG Media, guiding the development of cutting-edge recommendation systems and AI solutions.
Irina Radu is a Prototyping Engagement Manager at AWS EMEA, championing tech innovation and acceleration for customers.
Fernanda Machado, AWS Prototyping Architect, empowers customers with modern application best practices.
Andrew Shved, Senior AWS Prototyping Architect, delivers innovative business solutions leveraging modern technology.