Screenplay → Interactive Manga
Techniques For Transforming Story Text Into Multimedia Worlds with AI
Last summer, Lore Machine partnered with legendary science fiction writer Phil Gelatt, Modern Arts, and HP to produce the world’s first commercially distributed AI-augmented manga. The Way Forward: Into the Unknown was released IRL at Bilibili World in Beijing and made a lot of new friends online. So we got the band back together to make an interactive Volume 2!
Join us below on a sightseeing tour through the production process that made Volume 2: The Mirror Maze even more immersive. The techniques we invented directly inform tools that now power Lore Machine’s story visualization platform.
Lore Machine works with like-minded partners to transform text into multimedia adventures. Say hi at sup@loremachine.world 🖖
All Stories Start with Text
Like everything at Lore Machine, both The Way Forward manga volumes started their life as screenplay text.
In Volume 1, the main character slips through a portal in a video game, landing in a metaversal labyrinth patrolled by giant cyberpunk spiders. To take the ‘game within a game’ construct to the next level, we made Mirror Maze a choose your own adventure.
Phil Gelatt wrote event-based triggers into his foundational screenplay, inviting readers to forge their own path through the story.
Non-linear narrative design has historically presented a challenge for formats like manga. Producing enough visuals to keep up with multiplying story branches can become a losing race, fast. Generative technologies give superpowers to small teams crafting big interactive experiences.
Thumb Panel Illustrations
In the production of Volume 1, we built the final imagery straight from text. This led to a costly revisions process. No fun.
In Mirror maze, we gained consensus across teams by introducing an intermediate step between text and image generation: thumb panel illustrations. We worked with legendary comic illustrator Omar Francia to build a sketch of the action in each manga panel. This extra measure got all teams aligned and introduced more dexterity over the final art.
Here are some examples:
With thumb panel illustrations locked, the Lore Machine team moved on to image generation with more conviction, and control.
Final Art
The Lore Machine team crafted a style model that honored and elevated Volume 1’s aesthetic. We marked each figure’s joints on the thumb panel illustrations to guide the style model, which was layered over top.
Here are some examples of sketch → final art:
Multimedia Explosion
Generative enables small teams to build fleets of supporting assets that snap to the myriad formats and aspect ratios required of a fractured medisapshere. Damn you, 9:16! [shakes fist at sky]
Here’s a peek at the making of select 2.5D animated assets:
3D animated assets:
We even produced a 60-second Chinese national TV spot starring CGI’d out renditions of our characters.
Generative Lift
The hybrid human-machine production pipeline we stood up for Mirror Maze enabled Lore Machine’s small team to build an expansive bilingual story world in a breakneck 1.5 months.
You can stare into the Mirror Maze in English and Mandarin.
Team
Mirror Maze was brought to life by technologies being developed at Lore Machine and some absolutely stellar humans:
Original Story: Phil Gelatt
Story Adaptation: B. Earl
Illustration: Omar Francia
Synthography: Jumoke Fernandez
Creative Direction: Tillavision
Project Management: Jonathan Montaos
Layout & Graphic Design: Dersu Rhodes & Vitaly Kidon
Soundscape: Erik Hanson
2.5D Animation: Bijay Chudal
3D Animation: Paul Plastic
And the teams at HP Omen & Modern Arts
 | A guest post by
|
This is great. I subscribed to Lore and have been waiting on wait list for long time. Any update? Thanks.