Posted On: 2020-11-16
Automated integration tests are the polar opposite of unit tests: rather than testing any individual piece of the code, an integration test aims to verify that all the pieces work together correctly*. If, for example, one wanted to test "pressing start on the title screen takes the player to the character select screen", that could easily be a dozen unit tests**, but it would be one single integration test. As such, integration tests are most effective when run after all other tests, as the other tests are (generally) both faster to run and better diagnostic tools (due to being more precisely focused.)
In my experience, integration tests are something of a last line of defense. Where a robust suite of unit tests will tell me that my code works, a solid set of integration tests tells me that the code works when a user actually uses it. As such, automated integration tests are my last chance to catch obvious problems before QA gets their hands on the application - and therefore a key part of making sure that QA is best able to use their unique skill set to identify and triage bugs that no one could have anticipated.
One special kind of integration test is a User Interface test (or UI test for short*). Automated UI tests are, as their name suggests, focused on validating the systems' behavior in the context of what the user can see or interact with in the UI. Thus, for a test like "clicking the help button opens the help menu", a UI test will actually simulate the act of clicking the button in the UI - rather than relying on method calls, or other, similarly direct, mechanisms.
UI tests are particularly important in Unity, as this is one of the places where Unity's Test Framework is most powerful. Unity offers first-class support for creating and executing automated UI tests, making it simple to perform actions using precise, in-engine timing*. What's more, Unity's Test Framework uses the same development patterns for both game code and automated tests, which means a variety of wait actions (such as WaitForSecondsRealtime and WaitForEndOfFrame) are built-in, and any custom ones that were developed for use in the game can potentially be shared with the tests themselves**.
Personally, I am using automated UI testing in Unity to verify that all menu screens can be navigated using the keyboard/gamepad. This is a particularly important test for me, as supporting seamless transitioning between keyboard and mouse has been a source of many bugs, and resolving those bugs often risks breaking that functionality on other screens*. Now, with the UI tests in place, I can code with confidence, knowing that if the keyboard/gamepad input on any screen breaks, I will be immediately alerted so that I can resolve it.
Unity does, however, suffer from a problem that is common to many UI testing frameworks. While Unity provides a number of useful low-level tools (such as simulating input), there don't seem to be any for common high-level tasks (like finding a specific button from among a list of buttons). As such, developers often have to take on those tasks themselves and, if they are not careful, they can create some very brittle tests in the process*.
The lack of tools to solve high-level UI testing problems is by no means an insurmountable issue. Developers can create their own tools, and I have no doubt that some of those will make their way onto Unity's Asset Store. For myself - I expect I'll approach it similarly to how I approach anything else I'm learning: rolling my own (naive) implementations as I go*, and only investing in third-party solutions when I am skilled/experienced enough to see my own approach's shortcomings.
Prior to automating Unity tests, I had believed that automated unit and integration tests were the only kinds of testing necessary for a fully covered system. Together, they answered the two key questions that a developer faces: "does this work in theory", and "does this work in practice." After studying up on best practices for testing in Unity, I've found one more kind - something that I'd never needed before, but is clearly crucial in Unity: Content Validation. Join me next week, where I dive into what Content Validation testing is, why I'd never heard of it before, and why it's so crucial in Unity.