MovieAgent: Automated Movie Generation via Multi-Agent CoT Planning

Show Lab, National University of Singapore

MovieAgent: Convert your idea to a Movie!


Pipeline for MovieAgent

Network Structure

The Overall Pipeline for MovieAgent. The proposed framework employs a hierarchical CoT reasoning process with director, scene plan, and shot plan agents to automate long-form movie generation.


Task 1: Script Synopsis to Keyframe/StoryBoard Generation

Text to Keyframe/Storyboard Generation

With the hierarchical data MovieAgent provide, existing methods can partially reconstruct a coherent storyline, maintaining consistent character identities and relationships.


Task 2: Script Synopsis to Movie Generation

Anna, Elsa, Kristoff, Olaf, and Mattias embark on a journey to uncover the truth behind the mysterious voice calling Elsa. As they travel to the enchanted forest, they discover that the past holds secrets about their kingdom and Elsa’s powers. Mattias, a loyal Arendelle soldier trapped in the forest for years, helps them navigate the tensions between Arendelle and the Northuldra people…

Script Synopsis/Raw Story

Elsa
Elsa
Anna
Anna
Kristoff
Kristoff
Mattias
Mattias
Input: Script Synopsis and Character Bank (Image, Name)
Process with MovieAgent (GPT4-o + ROICtrl + Kling 1.6)
Output: Multi-scene, multi-shot videos with a coherent narrative.
Sub-Script 1: The Call and the Journey Begins
Plot: Anna, Elsa, Kristoff, and Olaf embark on a journey toward a mysterious voice calling Elsa. Along the way, supernatural weather linked to Elsa’s emotions challenges them. In the enchanted forest, they encounter frozen relics of a past conflict between Arendelle and the Northuldra. Elsa’s magic calms elemental spirits blocking their path, and they uncover cryptic symbols tied to Elsa’s childhood memories.
Involving Characters: Elsa, Anna, Kristoff, and Olaf.
Emotional Tone: Curiosity and determination.
Key Themes: Adventure, discovery, and mystery.
Scene 1 - Shot 1
Plot: Elsa stands alone in her ice dress, listening to a mysterious, ethereal voice. Her expression is a mix of curiosity and concern as she looks towards the distant horizon.
Involving Characters: Elsa.
Shot Type: Close-up.
Camera Movement: Static shot.
Subtitle: {Elsa:I can hear it again... the voice. It's calling me.}
Scene 1 - Shot 3
Plot: A wide shot reveals the enchanted forest bathed in ethereal light. Glowing plants and shimmering air create a magical atmosphere, emphasizing the vastness and wonder of the setting.
Involving Characters:
Shot Type: Wide shot.
Camera Movement: Static shot.
Subtitle: {}
Scene 1 - Shot 4
Plot: Anna and Elsa continue their conversation, now focusing on the maps. They lean over a large map, pointing out key locations and discussing the route.
Involving Characters: Elsa, Anna.
Shot Type: Medium close-up.
Camera Movement: Dolly-in.
Subtitle: {Anna: This path should lead us to the mountains. Elsa: Let's mark it clearly.}
Scene 1 - Shot 6
Plot: Anna, Elsa and Kristoff set out through a dense forest, their travel gear visible. Wide shot emphasizes the vast and challenging landscape they face.
Involving Characters: Elsa, Anna, Kristoff.
Shot Type: Wide shot.
Camera Movement: Static shot.
Subtitle: {Anna: Here we go, everyone. Stick together. Kristoff: Stay close, it's easy to get lost here.}
Scene 1 - Shot 7
Plot: A tracking shot of Elsa and Kristoff leading the way into the forest. The forest is dense and mysterious, with tall trees and a thick canopy. The sky is partly cloudy, casting dappled sunlight on the ground.
Involving Characters: Elsa, Kristoff.
Shot Type: Tracking shot.
Camera Movement: Dolly-in.
Subtitle: {Elsa: Let‘s go, everyone. Stay close. Kristoff: Right behind you, Elsa.}
Scene 2 - Shot 1
Plot: Wind whips the hair of Anna across faces, prismatic storm light slicing diagonally through frame.
Involving Characters: Anna
Shot Type: Medium Shot.
Camera Movement: Handheld shaky pan following debris.
Subtitle: {}
Scene 2 - Shot 4
Plot: Wide shot of Anna and Elsa surrounded by the ancient trees and historical artifacts.
Involving Characters: Elsa, Anna.
Shot Type: Wide Shot.
Camera Movement: Pull away.
Subtitle: {Anna: Understanding the past is key to moving forward. Elsa: We must learn from these stories.}
Scene 2 - Shot 5
Plot: Elsa and Anna stand in awe as mystical lights dance around them, casting ethereal glows and revealing ancient symbols.
Involving Characters: Elsa, Anna.
Shot Type: Close-up.
Camera Movement: Slow-motion.
Subtitle: {Elsa: Do you see that? It's like they're speaking to us. Anna: I've never seen anything like it.}
Sub-Script 2: Unraveling the Past
Plot: Mattias, a long-trapped Arendelle soldier, explains the unresolved conflict between Arendelle and Northuldra. Elsa, driven by the voice, ventures alone to a glacial cavern, where she learns she is the Fifth Spirit, a bridge between humans and magic. Meanwhile, Anna leads the group to follow Elsa, proving her leadership despite Kristoff’s concerns.
Involving Characters: Elsa, Anna, Kristoff, and Olaf.
Emotional Tone: Tension and revelation.
Key Themes: History, responsibility, and leadership.
Scene 3 - Shot 1
Plot: Close-up of Mattias as he explains the historical tensions between Arendelle and the Northuldra, his expression serious and informative.
Involving Characters: Mattias.
Shot Type: Close-up.
Camera Movement: Static shot.
Subtitle: {Mattias: The roots of our conflict lie in misunderstandings and past decisions.}
Scene 3 - Shot 3
Plot: Close-up of Mattias and Elsa as they facilitate the dialogue, their expressions hopeful and focused, working towards reconciliation.
Involving Characters: Elsa, Mattias.
Shot Type: Close-up.
Camera Movement: Gentle push-in.
Subtitle: {Mattias: By acknowledging our past, we can build a better future. Elsa: Together, we can find common ground.}
Scene 3 - Shot 4
Plot: Elsa and Mattias stop to consult the map and compass. Elsa looks determined but concerned. Mattias points out the next part of the journey.
Involving Characters: Elsa, Mattias.
Shot Type: Medium shot.
Camera Movement: Slow zoom out.
Subtitle: {Elsa: What's the next step?, Mattias: We need to cross the river up ahead. It's going to be tough, but we can do it.}
Scene 4 - Shot 1
Plot: Elsa, preparing to part ways with Anna, follows the call of the voice and ventures alone into a glacial cavern.
Involving Characters: Elsa, Anna.
Shot Type: Wide shot.
Camera Movement: Static shot.
Subtitle: {Elsa: Maybe I should leave. I'm being summoned.}
Scene 5 - Shot 1
Plot: Close-up of Elsa's face as she holds the revealing artifact, then sets off alone to a glacial cavern.
Involving Characters: Elsa.
Shot Type: Close-up.
Camera Movement: Static shot.
Subtitle: {Elsa: I understand now. All of it makes sense.}
Scene 5 - Shot 2
Plot: Elsa standing in the clearing, surrounded by swirling elemental forces. The camera beautifully captures the dynamic movement of the elements.
Involving Characters: Elsa.
Shot Type: Aerial shot.
Camera Movement: Dynamic movement capturing the swirling elements.
Subtitle: {}
Scene 6 - Shot 2
Plot: Group shot of Anna and Kristoff sharing a meal with representatives, smiles and nods exchanged as they reach a moment of understanding.
Involving Characters: Kristoff, Anna.
Shot Type: Group Shot.
Camera Movement: Static Shot.
Subtitle: {Anna: We can learn so much from each other. Kristoff: This feels like a new beginning.}
Scene 7 - Shot 1
Plot: During the expedition, Anna proves her leadership skills, while Kristoff, despite his concerns, gradually comes to accept it.
Involving Characters: Kristoff, Anna.
Shot Type: Group Shot.
Camera Movement: Static shot.
Subtitle: {Anna: We must go find Elsa. Kristoff: You're right.}
Sub-Script 3: Truth and Resolution
Plot: Elsa freezes after sending an ice message to Anna, who realizes she must destroy the dam to restore balance. Elsa revives as the forest’s protector, and the sisters choose separate paths. Elsa with the Northuldra, Anna as Arendelle’s queen.
Involving Characters: Elsa, Anna, Kristoff.
Emotional Tone:Sacrifice and resolution.
Key Themes: Redemption, unity, and self-discovery.
Scene 8 - Shot 1
Plot: Sweeping shot of Elsa and Kristoff standing with the crowd, applauding Anna‘s recognition, their expressions filled with pride and joy.
Involving Characters: Elsa, Kristoff.
Shot Type: Sweeping shot.
Camera Movement: Static shot.
Subtitle: {Elsa: I'm so proud of you, Anna. Kristoff: You've truly earned this moment.}
Scene 8 - Shot 2
Plot: Close-up of Anna as she is crowned queen, her expression a mix of triumph and hope, wearing the royal regalia.
Involving Characters: Anna.
Shot Type: Close-up.
Camera Movement: Static shot.
Subtitle: {Anna: I vow to lead with wisdom and kindness.}
Scene 9 - Shot 1
Plot: Close-up on Elsa‘s face, her expression a mix of love and admiration for Anna.
Involving Characters: Elsa.
Shot Type: Close-up.
Camera Movement: Static shot.
Subtitle: {Elsa: You've shown me what true strength is.}
Scene 10 - Shot 1
Plot: Wide shot of Anna and Elsa standing together on a hill, overlooking the enchanted forest and Arendelle.
Involving Characters: Elsa, Anna.
Shot Type: Wide shot.
Camera Movement: Static shot.
Subtitle: {Anna: We've come so far, and there's so much more to achieve. Elsa: Together, we'll ensure harmony for both our homes.}

Task 3: Joint Generation of Movies with Audio from Script Synopses

Case 1: Frozen II

Anna, Elsa, Kristoff, Olaf, and Mattias embark on a journey to uncover the truth behind the mysterious voice calling Elsa. As they travel to the enchanted forest, they discover that the past holds secrets about their kingdom and Elsa’s powers. Mattias, a loyal Arendelle soldier trapped in the forest for years, helps them navigate the tensions between Arendelle and the Northuldra people…

Script Synopsis/Raw Story

Elsa
Elsa
Anna
Anna
Kristoff
Kristoff
Input: Script Synopsis and Character Bank (Image,Audio Sample, Name)
Process with MovieAgent (GPT4-o + ROICtrl + VALL-E X + Mango Talking Human)
Output: Multi-scene, multi-shot videos with a coherent narrative, synchronized audio.
Scene 1 - Shot 1
Plot: Elsa stands alone in her ice dress, listening to a mysterious, ethereal voice. Her expression is a mix of curiosity and concern as she looks towards the distant horizon.
Involving Characters: Elsa.
Subtitle:
Elsa: I can hear it again... the voice. It's calling me. I have to follow it... I have to know the truth.
Scene 6 - Shot 2
Plot: Group shot of Anna and Kristoff sharing a meal with representatives, smiles and nods exchanged as they reach a moment of understanding.
Involving Characters: Kristoff, Anna.
Anna: We can learn so much from each other. We just need to listen and work together.
Kristoff: Anna, this is dangerous. Are you sure about this?
Anna: I know it won’t be easy, but we can’t turn back now. Trust me, Kristoff.
Kristoff:You never stop surprising me... Okay, I trust you.
Scene 8 - Shot 1
Plot: Sweeping shot of Elsa and Kristoff standing with the crowd, applauding Anna‘s recognition, their expressions filled with pride and joy.
Involving Characters: Elsa, Kristoff.
Elsa: I'm so proud of you, Anna. You’ve always had the heart of a leader. Now, the world sees it too.
Kristoff: You've truly earned this moment. No one deserves it more.
Elsa: Because we’ve always known what you’re capable of. And now, so do you.
Scene 8 - Shot 2
Plot: Close-up of Anna as she is crowned queen, her expression a mix of triumph and hope, wearing the royal regalia.
Involving Characters: Anna.
Anna: I vow to lead with wisdom and kindness.
Anna: Arendelle has faced many challenges, but we have always stood strong—together.
Anna: As your queen, I promise to honor our past, protect our present, and guide us toward a bright future.
Anna: With courage, with compassion… and with all my heart.

Case 2: NeZha II


After the catastrophe, Nezha and Aobing’s souls survived, but their bodies were on the verge of shattering. Taiyi planned to use the seven-colored lotus to restore them. However, Shenggongbao led the dragon clan and other sea demons to besiege Chentang Pass, seeking revenge on Nezha. In the fierce battle, Nezha accidentally shattered Aobing’s body. To save him, Nezha decided to travel to Yuxu Palace and take the Immortal Ascension Exam to obtain a way to revive Aobing. Before his departure, Lijing and Ladyyin bid him farewell.

Script Synopsis/Raw Story

Ne Zha
Ne Zha
Ao Bing
Ao Bing
Shenggongbao
Sheng GongBao
Input: Script Synopsis and Character Bank (Image,Audio Sample, Name)
Process with MovieAgent (GPT4-o + ROICtrl + VALL-E X + Mango Talking Human)
Output: Multi-scene, multi-shot videos with a coherent narrative, synchronized audio.
Scene 1 - Shot 1
Plot: Ne Zha and Ao Bing discuss how to restore their bodies in a vibrant, colorful forest.
Involving Characters: Ne Zha, Ao Bing.
Ne Zha: Our bodies... we can’t stay like this forever. There must be a way to restore them.
Ao Bing: The Yuxu Palace holds countless ancient treasures. Perhaps one of them can heal us.
Ne Zha: Yuxu Palace? That place isn’t exactly welcoming.
Ao Bing: It’s our best chance. If we don’t try, we’ll never know.
Ne Zha: Fine. But if we get caught, you’re doing the talking.
Scene 2 - Shot 5
Plot: Sheng Gongbao appears in a confrontational stance, intent on attacking Chentang Pass and assassinating Nezha.
Involving Characters: Sheng Gongbao.
Subtitle:
Sheng Gongbao: I won't let you succeed! I've come too far to be stopped now.
Sheng Gongbao: Nezha, you are but a child playing with fire.
Sheng Gongbao: Step aside, or be burned by it.
Scene 4 - Shot 3
Plot: Li Jing offers words of encouragement to Ne Zha, believing that he can successfully retrieve the elixir from Yuxu Palace to save his friend.
Involving Characters: Ne Zha, Li Jing.
Subtitle:
Li Jing: Don't be too sad. Believe in yourself—you can definitely save your friend. Then return to us safely.
Ne Zha: Thank you, Dad. I think I need a bit more time to process my emotions.
Scene 5 - Shot 5
Plot: Ne Zha sneaks into Yuxu Palace, searching for the elixir. However, the palace is vast, and as he navigates its endless corridors, he unknowingly steps into a mysterious chamber.
Involving Characters: Ne Zha.
Subtitle:
Ne Zha: So this is Yuxu Palace… Hmph, not as scary as the legends say.
Ne Zha: Alright, time to find that elixir and get out before someone decides I’m an intruder.

BibTeX

@misc{wu2025movieagent,
      title={Automated Movie Generation via Multi-Agent CoT Planning}, 
      author={Weijia Wu, Zeyu Zhu, Mike Zheng Shou},
      year={2025},
      eprint={2503.07314},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}