Posted On: 2020-11-09
Automated testing is a key part of sustainable development. As projects change over time, bugs inevitably creep in, and automated tests form the first line of defense by alerting developers to issues as soon as they occur. While automated testing is (generally) well supported in many frameworks, Unity 3D was not originally designed to support it. Fortunately, the Unity developers have (somewhat) recently* taken an interest in providing better support for automated testing. In this series of posts, I will describe some forms of automated tests, how they become more complex with Unity, and how I personally use them (with or without Unity's tools.)
Automated unit tests are the most common form of automated tests, and can cover a wide range of potential sources of bugs. At its core, a unit test is a test of a small piece of functionality - generally as small a piece as possible. This is primarily achieved by testing the code itself directly, in contrast with most kinds of manual testing (which test the application while it is running.) Unit tests are perhaps best understood with an example - if one wishes to unit test a feature where pushing a button changes the color of some text to red, that would likely be three separate unit tests:
SetColor) sets the color property of the text to the provided colorSetColor with "red" as its parameter.These three tests together represent the idea "clicking this button turns the text red", but by testing smaller units, the tests become more valuable as diagnostic tools.
If, for example, a change to the text (accidentally) caused it to always be green, then the automated tests would suddenly fail. Importantly, the unit test for SetColor would fail, while the test for the button click handler would continue to pass*. As a result, one will immediately know that something is wrong with how colors
behave, which will (hopefully) save time diagnosing and solving the issue. Thus, the more granular and precise a test is, the more it will help to narrow down
the source of unexpected changes.
Unfortunately, Unity is particularly troublesome when it comes to unit testing - in large part due to many of the conveniences of its design. Unity manages all object initialization, including assigning the designers' choices in the inspector to fields on the component. This often includes dependencies between components, which poses a significant challenge for unit testing, as isolating tests from one another normally requires substituting dependencies with test doubles that limit the impact of unrelated code. While it is possible to work around many of these difficulties, doing so often makes the code more complex, potentially affecting the usability of the inspector or making it more difficult for new developers to get up to speed with the architecture.
For my own work, I've worked around this by being a bit less thorough with my tests. Specifically, I've separated my code so that it belongs to one of two categories: code that is worth unit testing, and code that is not. Then, I take everything that is worth unit testing and isolate it so that it does not depend on Unity*. This empowers me to completely control the life-cycle of every part of the isolated code, which, in turn, makes it simple to unit test. In most cases, I also move this code into a completely separate library - though that is more for my own convenience** than anything else. Once properly isolated, I can write and execute tests effortlessly, making it possible to use patterns like Test-Driven Development to ensure the code always behaves exactly as intended.
Unfortunately, this approach comes with the downside that, at some point, I have to write code that can't have automated unit tests. Generally this occurs in areas that are the (metaphorical) glue between my own (well-tested) libraries and Unity itself. In an ideal world, this would be exclusively wrapper Monobehaviors, but in practice this also includes quite a bit of code that does actual work. Like many other gaps between the ideal and the actual, this is something I try to accept with an eye to the future: leaving it as-is for now, all while understanding that some portion of it will undoubtedly come back to bite me later*.
Since I inevitably will have gaps in my unit tests, other forms of automated tests are much more important than they otherwise would be. As such, I've been spent time developing my knowledge of automated integration testing in Unity. Tune in next week, when I describe what I've learned, and how I'm using it to catch UI bugs before they happen.