Imagine you are a film director, but instead of hiring a composer to write a score for your movie, you have a very smart, automated robot assistant. Your movie is long, full of different scenes that jump from a quiet forest to a high-speed car chase, and then to a sad goodbye.

The problem with current AI music tools is that they are like a musician who only knows how to play one song at a time. If you ask them to score a whole movie, they might play a happy tune for the forest, then abruptly switch to a heavy metal song for the car chase, leaving you with a jarring, disjointed mess. They don't know how to bridge the gap between the scenes.

JenBridge is a new system designed to fix this. Think of it as a "musical bridge builder" that can handle an entire movie, ensuring the music flows naturally even when the visual scenes change drastically.

Here is how it works, broken down into three simple steps:

1. The Editor: Cutting the Movie into Scenes

First, JenBridge acts like a film editor. It looks at your long video and automatically cuts it into smaller, logical chunks (scenes). It doesn't just cut randomly; it looks for where the story changes, like when a character walks out of a room or the weather changes.

2. The Composer: Writing Music for Each Scene

Next, for each little chunk of video, JenBridge has a "Composer" AI. This composer is very talented. It looks at the video clip and asks, "What kind of music fits this?"

It uses a special translator (called VMPT) to turn a description of the video (e.g., "a sad robot in the rain") into a musical recipe (e.g., "slow, sad piano, minor key").
It then generates a high-quality piece of music specifically for that scene.

3. The Director: The "Glue" Between Scenes

This is the most important part. In the past, if you had two different songs, you would just tape them together, which sounds terrible. JenBridge has a Director (an AI agent acting like a human film director) who decides how to connect the songs.

The Director has a "toolkit" of four ways to connect scenes:

The Hard Cut: If the scene changes from a calm beach to a sudden explosion, the Director slams the music to a stop and starts the new song immediately. It's like a sharp "snap" in the story.
The Silence: If a character is shocked, the Director might cut the music to complete silence for a moment to let the tension build.
The Fade: If two scenes are similar (like a morning coffee turning into a morning walk), the Director gently fades one song out while fading the other in.
The Bridge (The Magic Trick): If the story shifts from "sad" to "hopeful," the Director doesn't just switch songs. It uses a special AI to compose a brand-new musical bridge that smoothly morphs the sad song into the hopeful one. It's like a musical translator that speaks both languages and creates a sentence that connects them perfectly.

The Director is smart enough to look at the story and say, "Okay, this scene needs a hard cut, but this next one needs a smooth bridge." It makes these choices intelligently, just like a human director would.

How They Tested It

The creators realized that no one had ever properly tested if an AI could do this for long movies. So, they built a new "exam" called the LVS Benchmark.

They took 120 movie trailers (which are full of fast scene changes).
They asked JenBridge and several other AI systems to score them.
They had human listeners rate the results.

The Result: JenBridge won by a landslide. The human listeners said the music sounded much more natural, the transitions were smoother, and the whole movie felt like it had a single, cohesive story, rather than a bunch of random songs glued together.

In Summary

JenBridge is a system that doesn't just make music for video clips; it understands the story. It breaks the video down, writes custom music for each part, and then uses a smart "Director" to decide exactly how to connect those parts so the music flows as naturally as the movie itself. It's a step toward AI that can act as a true creative partner for making movies.

Technical Summary: JenBridge

Problem Statement

Current AI music generation systems, particularly those advancing text-to-music (T2M) synthesis, excel at creating high-fidelity audio for short, isolated clips. However, they face critical limitations when applied to long-form video content characterized by dynamic scene transitions. Existing video-to-music (V2M) approaches typically treat videos as monolithic segments, generating a single piece of music that lacks mechanisms to handle semantic shifts between scenes. This results in a lack of narrative continuity and an inability to provide creative control over intermediate steps or transition naturalness, rendering them impractical for professional, long-form video soundtracking.

Methodology

The authors propose JenBridge, a modular and interpretable framework designed for adaptive long-form video soundtracking. The system operates through a three-stage pipeline:

1. Semantic Video Segmentation

The input video is partitioned into a temporally ordered sequence of semantically coherent clips ( $V_1, V_2, \dots, V_K$ ) using PySceneDetect. This preprocessing allows the framework to handle videos of arbitrary duration by operating on manageable segments.

2. Progressive Video-Aware Music Generation

The core generative model follows a two-stage training paradigm to achieve high-fidelity synthesis and precise cross-modal alignment:

Stage 1 (Text-to-Music Foundation): A foundational model is pre-trained on large-scale text–audio corpora. It utilizes a pre-trained neural audio codec (Encodec) to encode raw waveforms into a latent space. The generative backbone is a Multimodal Diffusion Transformer (MMDiT) trained with a flow-matching objective. The conditioning mechanism employs a multi-encoder text architecture (T5-large and T5-base) to generate sequence embeddings ( $y$ ) and global pooled embeddings ( $c$ ) from descriptive and attribute prompts (genre, instrument, mood).
Stage 2 (Video-Aware Adaptation): The model is adapted to the video domain via dual conditioning.
- Visual Conditioning: A dedicated visual encoder (SigLIP) extracts frame-level features, which are average-pooled to form a global visual feature vector. This is concatenated with the text embedding to form a fused global condition ( $c_{fused}$ ).
- Visual-to-Music Prompt Translator (VMPT): To address the lack of structured music prompts in video datasets, a fine-tuned LLM (Qwen3-8B) translates raw video captions into structured music prompts containing metadata (genre, instrument, mood, BPM).
- The model is trained to predict a ground-truth vector field based on these fused conditions.

3. Adaptive Transition Mechanism

To connect the generated audio clips ( $A_1, \dots, A_K$ ) seamlessly, JenBridge employs an adaptive transition system comprising:

Transition Toolkit: A versatile set of four styles: Abrupt Cut, Silent Gap, Fade In/Out, and a Generative Transition. The generative transition utilizes a ControlNet-based inpainting model trained to synthesize a musical bridge between two distinct audio segments. This model is guided by conditions created via spherical linear interpolation (slerp) of text embeddings and block-wise latent representations.
LLM Agent as Director: An LLM agent (Qwen3-8B) acts as a creative director. Given the visual and musical context of adjacent clips, the agent uses few-shot in-context learning to intelligently select the most appropriate transition style from the toolkit for each narrative shift.

Key Contributions

JenBridge Framework: An end-to-end, modular framework for long-form video soundtracking that produces coherent, high-fidelity music across diverse scene transitions. Its architecture is designed to be interpretable and controllable.
Adaptive Transition Mechanism: A novel system combining a versatile toolkit of transition styles (including a generative inpainting method) with an LLM agent that makes context-aware creative choices to direct transitions.
LVS Benchmark: The introduction of the Long-form Video Soundtrack (LVS) Benchmark, the first benchmark specifically designed for this task. It includes a curated dataset of 120 long-form trailer videos with rich multimodal annotations and proposes a holistic evaluation protocol that moves beyond simple segment-based metrics to assess transition quality and narrative coherence.

Experimental Results

The authors evaluated JenBridge on the LVS Benchmark against six baseline approaches (including CMT, LORIS, AudioX, and VidMuse in various configurations).

Objective Metrics: JenBridge achieved state-of-the-art performance, significantly outperforming baselines in video-music alignment (ImageBind Score: 0.224 vs. 0.162 for the next best) and audio aesthetics. Notably, it scored highest in Production Complexity (PC), indicating a richer musical structure compared to the simple concatenation or monolithic generation of baselines.
Subjective Metrics: In a user study with 10 participants, JenBridge received the highest ratings across all dimensions. The most significant improvement was in Transition Naturalness (4.2 vs. 2.5 for the best baseline), validating the effectiveness of the adaptive transition mechanism.
Ablation Studies: Removing the adaptive transition mechanism caused a catastrophic drop in transition scores (from 4.2 to 2.8), confirming its critical role. Disabling visual conditioning or the VMPT also led to significant performance degradation across all metrics.

Significance and Claims

The paper claims that JenBridge represents a significant step towards fully automated, professional-quality video soundtracking. By addressing the critical gap of maintaining narrative continuity across scene transitions, the framework bridges the divide between automated generation and the creative workflows required for long-form content. The authors position JenBridge not as a replacement for human creators, but as a collaborative tool to empower them.

The work also highlights the importance of the LVS Benchmark in establishing rigorous evaluation standards for long-form generation, moving beyond the limitations of existing short-clip datasets. While acknowledging limitations regarding the fidelity of public video-music datasets and the lack of global narrative planning in the current LLM agent, the authors assert that the framework provides a robust foundation for future advancements in collaborative AI tools for video production.

JenBridge: Adaptive Long-Form Video Soundtracking across Scene Transitions