What I Learned Building a Game Almost Entirely with AI

As you might remember from my previous blog post, I decided to build a game for mobile phones. I had two goals in mind.

First, I wanted to use it as an AI experiment. I basically wanted AI to do everything. It was part of a learning exercise where I wanted to test where the boundaries are. My first intention was simple: “Write one prompt and AI will do everything for you.” I couldn’t have been farther from reality.

My second intention was to create something calming and healing. With all the anxiety currently surrounding the use of Artificial Intelligence, creating something that brings the opposite emotions sounded like a nice idea.

The nature of the game ruled out some genres automatically — shooters or fighting games, for instance. I leaned toward an adventure game, but for my first experiment I wanted to make something simple. I settled on a story game presented through static pictures and text.

I started to build a visual novel without knowing there was a genre called visual novel.

I began with the Gemini access I had. The excitement was real. We set up the developer machine in a VM and I started to scaffold the code. Gemini guided me nicely, and for me it was a lot of copy-pasting generated code into VS Code, saving files, running the debugger, copy-pasting errors, and going back and forth. I was really excited, because the capability of Gemini Pro felt like something different from what I’d experienced with ChatGPT a year ago. It also explained a lot of coding principles to me, and I started to understand the structure of a Flutter app.

In fact, that was one of the biggest things this experiment brought me. Step by step, I started to understand how Flutter works. It made me dive deeper into the technology.

The next part was the graphics. And to be honest, this was a real challenge. After some discussion with Gemini, I bought a Midjourney subscription. The killer feature for me was the so-called omni reference. You can take a picture of a person, set the level of “sameness,” and Midjourney will generate an image with exactly (or more or less) the same person. That’s a great thing when you need to create a series of images of your hero in different situations and setups.

I started by creating a character reference sheet — sketches of Elara, my heroine, from different angles and in different positions. I used that as the omni reference for all my pictures. The key takeaway from working with Midjourney was that image generation is far less straightforward than code generation. Sometimes I simply couldn’t get what I wanted. I tried various prompts, different wording, even asked Gemini to write the prompts for me… but many times it was impossible to get a good image that fit the game.

In those scenarios I usually fell back to Sora, which creates different styles of images, and more than once it saved me from the void of desperation.

The next part of the story was… the story. I brainstormed it with Gemini a lot, and we settled on a story divided into three days of Elara’s life. The game presents low-stakes choices that shape the story but shouldn’t bring any stress or “wrong” choices. I let Gemini generate the story. It was quite a bit of text I had to proofread in both English and Czech.

One side note. I’m not a fan of AI-generated text. I accepted it for this game since it was an AI experiment, but for my next story game I’ll invest more time in a human-crafted story. AI is still not a good writer.

So I had code scaffolding, images, and a story for the game. The last piece was music and sound effects. Honestly, music was the piece I spent the least time on. Neither AI service was much good at it, and getting a decent beat out of them was almost impossible. Maybe I just didn’t invest enough time, or I chose the wrong tools.

And that was it. After two weeks of intensive work, I had a working game and a lot of new experience. I’ve been thinking about the main takeaways from my experiment, and here’s what I curated:

• A single-prompt game or piece of software is nonsense. AI can be extremely helpful, but it’s still just a tool, not a magic wand. There’s a lot of work to do when making an app.
• Image generation might seem cool until you need consistency, or until you have an exact idea of what you need.
• AI can enable you to do things you don’t know how to do, in an extremely short time.
• All in all, AI is far more powerful and useful than many people realise.

If you want to check out Aura of Gold, it’s available on Google Play. You’ll find the link on the App page here.