Story Orcs Prototype

This prototype, was built over a long weekend. It takes Aesop’s fables from the Library of Congress, and creates a story-time experience around a campfire. At run-time, it takes the text, sends it to the Amazon Polly service, and combines the speech-mark meta data along with a bunch of custom scripts, to define everything from the facial animation, body poses, face emote, and the music track.

With such a small amount of time to create this demo, I didn’t have time to properly rig and animate the characters. Because of this, I opted to simply swap heads and pose the character in a more stop-motion style.

The prototype was written in JavaScript and uses the WebGL engine, Amazon Sumerian. I did the ui, code, and art. The was entirely composed by an AI, using the service, Jukedeck.

There is also a VR mode where guests can sit with the orcs, and listen to stories. Currently, it plays a new story for every day of the week.

If I HAD MORE TIME TO MAKE THIS A PROPER APPLICATION…

I love taking ideas and seeing how far we can go with it. In this case, I’d probably start by doing more research into the art of oral storytelling. The cadence, the pacing, the pausing, and communicating highs and lows. I’d also like to make the environment procedural as well to reflect the stories time of day. And of course we’d need to be able to have environmental sound effects for onomatopoeia.

When in VR, the one thing I found a bit strange, is that it’s really tough to express how yourself. Normally when I watch TV, I’m often doodling, folding paper, or expressing my fear (covering my eyes) or excitement (fist pump) while I watch the story. My gut tells me, that by giving guests orc arms/hands and props like a stick, rocks, and marshmallows. This would allow them to draw on the ground, roast marshmallows, throw things at the kids or the storyteller, and express their joy or fear.

Below is the live WebGL version. (I haven’t fully tested it in my Squarespace web-page environment).