AI in Sound Design: Exploring Automatic Sound Recommendation and Generation for Film Production

Master Thesis

Start Date: Flexible

Supervisor: Christoph Weber (c.weber ät hff-muc.de) HFF Munich

Overview

This thesis project offers an exciting opportunity for students interested in artificial intelligence, film production, and audio design to contribute to cutting-edge research. Sound design plays a crucial role in shaping emotional engagement and narrative coherence in film, yet the process remains highly manual, experience-dependent, and time-consuming. The primary goal of this research is to explore how AI-driven systems can automatically classify selected film scenes and subsequently generate and/or suggest appropriate sound design elements from a database. By systematically examining scene classification and AI-based audio generation, this research aims to develop a practical solution that integrates seamlessly into professional audio workflows—reducing manual workload and enhancing creative possibilities for sound designers—while also investigating its impact on user experience, as well as key factors such as control and sense of ownership.

Objectives

• Perform a literature review
• Design, conduct, and analyze expert interviews
• Develop an AI-based prototype for automatic scene classification and sound design recommendation and/or generation
• Demonstrate the integration of your prototype within an established audio production workflow (e.g., Avid Pro Tools)
• Evaluate the effectiveness of your prototype and document your findings in a thesis
• Summarize your findings in a thesis and present them
• (Optional) Co-author a research paper based on your findings

Required Skills & Knowledge

• Good programming skills in Python
• Interest in film production and sound design
• Basic knowledge of audio and video editing (ideally Avid Pro Tools or similar)
• Knowledge of artificial intelligence, machine learning, and ideally deep learning
• Motivation to learn new skills
• You have reviewed and familiarized yourself with the provided references
• Optional, but beneficial) Experience with the JUCE Framework

References (selection)

• Kamath, Purnima, et al. "Sound designer-generative ai interactions: Towards designing creative support tools for professional sound designers." Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 2024.
• Cheng, Ho Kei, et al. "Taming multimodal joint training for high-quality video-to-audio synthesis." arXiv preprint arXiv:2412.15322 (2024).
• Zhang, Yiming, et al. "Foleycrafter: Bring silent videos to life with lifelike and synchronized sounds." arXiv preprint arXiv:2407.01494 (2024).

Please send a brief motivation letter, CV, and transcript of records if you are interested in this Master thesis to c.weber ät hff-muc.de.