Posted On: 2023-10-02
Today marks 5 years since I started on this strange journey of full-time game development. That's a pretty big milestone, and it's prompted me to reflect on my current project - where I am, how I got here, and what was lost along the way. In particular, the code - and the engines underneath it - has gone through an incredible transformation in that time. Thus, for today's post, I thought I'd describe that transformation - divided up into 5 different phases.
The very first iteration of the project was the Magic Training Prototype. At the time, I knew there would be a lot of changes, but I wanted to get something playable in front of people, even if it meant creating a lot of code that I would throw away at the end. Importantly, this was my first attempt to use RexEngine a (now defunct) 2D physics/mechanics library that I'd planned to use for the larger project, so being able to experiment with it and generally make a mess of things was essential for learning the new tool.
Despite my intentions to the contrary, one part of the code base actually survived the prototype. The DialogueEngine (which controls both the conversation and the menu systems) was so complete that, when I sat down to rewrite it in my "final version" of my game project, I realized I could copy it in with only minimal changes, and it would be fully featured and future-proof.
Besides writing the code with the intention of discarding it later, there were also several features that I attempted but ultimately excluded from the playable prototype. In some ways this foreshadowed what would happen with the project longer-term, so I think it's important to include the cut features here:
Rewriting what I'd created in the Magic Training Prototype so that it fit into a larger, more structured project was supposed to be something that I would only need to do once. To that end, I put in a lot of effort to make sure everything was lined up correctly up-front, including working through how the save and load systems would work together with RexEngine.
During this time I also developed the Salience Engine, which was designed to remedy some of the structural issues that I had while working with Yarn in the Magic Training Prototype. Development on that gradually grew over time, as what started out as a couple extra commands (and a hand-written parser) gradually became a full-fledged language (with its own ANTLR-based parser-lexer). As it grew, so too did my need for tools to support it, and before long I was not only building this new language, but also a brand-new UI for writing Salience-Engine-based Yarn content.
Much of the work on the Salience Engine was superseded by my eventual switch to Ink, which happened late during this phase. Interestingly, the DialogueEngine, which handled connecting Yarn to custom UI elements in Unity (as well as handling user choices) proved to be a fine foundation on which I could build a new framework for connecting Ink and that same UI. As such, converting the dialogue over to Ink went fairly quickly - it only took about 2 months to make the switch, during which time I was migrating both the code and the underlying dialogue scripts themselves.
These two years of development included a lot of features - most of which began as "experimental" code, with only the most promising (and stable) being promoted to become part of the project proper. Among the experimental features that didn't make the cut are:
By this point, I'd accumulated several years worth of code, but I was still missing a fundamental system. Writing the movement of AI-controlled Agents was painstakingly slow, and extremely brittle as even small changes to the terrain would require a complete rework of everything I'd accomplished. On top of that, even the simplest variation in movement required breaking out of the existing AI framework - which itself was tightly integrated into RexEngine (and its extremely limited options for movement). Something as simple as causing a bird to move upwards when it flaps its wings was essentially impossible with my current tools.
Facing this problem head-on, I spent significant time focused just on the problem of Autonomous Agent Movement, making any and all changes necessary along the way to accomplish it. Some of this took the form of various experimental flight and navigation systems, but the biggest change by far was the careful and deliberate removal of RexEngine: by this point it had become clear that what was once a simple crutch to speed my development was now holding the project back.
By the end of this phase, however, it became clear that removing RexEngine required completely rewriting the project. Nearly every script - both in the "stable" codebase and the large pile of experimental code - touched Rex in one way or another. Scene loading had to tiptoe around Rex singletons, user input handling had to coordinate with Rex's own input system, and anything that affected movement had to account for Rex's physics and character controllers, lest Rex completely override the changes. Rex constrained what was possible in my project - not simply by being incompatible with certain features, but by requiring concessions and workarounds on every feature.
No matter how I proceeded, I knew I would have to rewrite my project. Things could be scavenged, certainly, but starting fresh would be faster and more stable than trying to keep building on broken systems. After exploring the possibility of starting over with a new Unity project, I finally decided to cut those ties as well - by developing the project as a library first, I would be manually testing less, and automatically testing far more - something that would be unrealistic to attempt with Unity.
This standalone library contains several successful rewrites of older systems, including a fully-functional dialogue and menu system based on what I'd built for Ink in Unity, but adjusted so it can work with any UI/game engine. Developed alongside that was an updated version of the save system - based on what I'd built for Unity, but freed from the convolutions that RexEngine introduced into its design. Both are completely covered by automated unit tests, and the freedom that's provided has allowed new features to be added with confidence (including the ability to save and load dialogues mid-conversation, without losing a beat.)
Like every phase of the project, there're some cut features here and there - but the main one is my work on a substitute physics engine. Replacing RexEngine means I need some kind of 2D physics system - but whatever replacement I use needs to be stable and predictable, since the physics will be a core part of the gameplay. For my part, I would prefer to eschew dynamics (ie. bouncing objects) for a more stable system, but dynamics is the selling point of most physics engines, so I haven't found anything that fits my needs. As part of my testing-focused approach, I made a set of tests that represents how I want the physics system to behave (ie. objects stopping on contact, etc.) but, so far, I haven't found (or created) anything that passes every single one of the tests.
The standalone library was never intended to be a complete solution: input and graphics are foundational parts of any interactive project, but I'd very much prefer that I don't have to code those myself. Rather, I wanted to create a standalone library so that it would be possible to move the game between engines with the confidence that the fundamentals would remain the same. Recently, I put this theory to the test, and have been delighted by the result.
When I started to learn Godot for an unrelated reason (a small personal project), I was stunned by how much better the C# support is in the current version (4.1) compared to earlier versions - and even compared to Unity. In light of that, Godot seemed like the perfect test for the library - to determine whether I truly could keep all the game's logic stable and isolated, while still using a fully-fledged game engine for display and input.
With a few configuration tweaks, I've managed to arrange the Godot project and the standalone library to coexist together in the same solution. The main benefit of this approach is that I can continue to develop the library with the same tools and workflow (IDE, automated tests, parallel development, REPL-based spot-debugging) and instantly get those changes included in the Godot project. This has been a huge boon as I work through an early attempt at a new feature - I used Godot to do the "experimental" work, and now that I'm happy with it, I can easily clean up and migrate the code into the library - all while adding the necessary automated tests to validate that it works correctly in every situation.
At this point I am quite optimistic about my project's future. Although the project is still missing several fundamentals, Godot's workflow allows me to work in earnest on that - complete with automated tests protecting against future changes and mistakes. Perhaps Godot will prove to be the right fit for the project long-term, but even if it doesn't, I am optimistic that my new approach will allow me to keep making progress, no matter what engine it uses in the end.