The Quest to Rule Scene Density 🏰

ℑ𝔫 𝔚𝔥𝔦𝔠𝔥 𝔞 𝔐𝔢𝔯𝔯𝔶 𝔅𝔞𝔫𝔡 𝔬𝔣 𝔖𝔱𝔬𝔯𝔶𝔱𝔢𝔩𝔩𝔢𝔯𝔰 🧙‍♂️🧌🧝🏻‍♂️ 𝔈𝔮𝔲𝔦𝔭 𝔞𝔫 𝔏𝔏𝔐 𝔴𝔦𝔱𝔥 𝔈𝔶𝔢𝔰 👁️

Lore Machine is on a quest to build the planet’s most powerful story visualization system. To this end, we’re on 10 side quests at all times. One of these is maximizing Scene Density.

Scene Density is the number of words it takes to generate one image. It’s a crucial calculation:

Scene Density = StoryText Word Count / Total Images Generated

The relationship between a comprehensive visual narrative - like a film or graphic novel - and its constituent generated scenes is akin to a digital image and its pixels. When it comes to Scene Density more is better.

In utilitarian terms, more Scene Density gives you more multimedia blocks to work with when building an immersive story experience.

Here’s where Lore Machine started with Scene Density, where we’re at and where we’re going. 

A classic Lore Machine side quest rendered in DotMatrix

Humble Beginnings

When Lore Machine lurched into gear this March, Scene Density was an afterthought.

We initially rigged Lore Machine to generate an image every time our LLM detected one of three story occurrences:

• A character performed an action

• A location changed

• A physical object impacted its surroundings

Lots of these happen in every story, and so lots of images would come out the other side, so we thought. A perfect plan, so it seemed.

But alas, no. We drastically overestimated our LLM’s ability to detect these carefully-orchestrated scene triggers. As a result, swathes of beautiful storytext were going unvisualized. Three-thousand word masterpieces were yielding four images on the regular.

We needed a plan.

Round Two: Sight!

Our LLM was failing to see crucial scene triggers resulting in sub-optimal image generation and much unrest in the Discord. We needed to give our LLM eyes.

To begin, we told the LLM when to look and how often. Working backward from anecdotal user input, we established an ideal Scene Density of 200. In other words, we demanded the LLM see 1 image worth of scene details every 200 words before proceeding through a storytext.

Opening our LLM’s eyes up to this set of properties every 200 words began to work. The detected scene properties are then merged with libraries of metadata from each respective story using an assembler algorithm. The outcome is an explosion in the volume of modularly-built prompts, which turbocharges Scene Density and multimedia output for all users.

Here are some early results:

Scene Density All The Way Down

We are dangling our legs over the edge of a future in which writers are the commanders of content. This is a future in which the originators of our civilization’s stories can conjure their imagination into reality in an evening.

Lore Machine is here to help writers tap into these emergent story visualization powers. By cranking aspects of our system like Scene Density, we provide high-resolution visual representations of storytext. This volume of media means creative options and that composability provides writers with the surface area upon which to heighten individual expression and maybe help us all understand each other a little more.

If this gets you so excited you can’t sleep at night