Test Automation for Video Games: Things to Consider Before You Start
How do you feel about processing every added feature in WoW, Forza Horizon, or The Sims Online manually to make sure they work as intended? How long would it take to ensure there is no conflict with the other upgrades?
It would take ages. This is why automation is crucial in video game testing right now.
The gaming industry is not mature enough in test automation practices, yet their adoption is being brought into the mainstream by dev studios all over the world.
In this article, we’ll outline the basics of test automation, its potential advantages, factors to consider in advance, and game types highly suitable for automation.
Test Automation in a Nutshell
Software testing and test automation are different things, obviously.
Manual testing is about designing effective test cases able to detect as many bugs as possible in every aspect of the software, and then executing tests by hand.
Test automation, on the other hand, allows automating test execution (for example, for repetitive tasks) and, therefore, cutting costs and speeding up the whole process.
That’s exactly what game developers need these days, right? First of all, many game development studios are adopting CI/CD practices that require them to test and release faster. Secondly, modern video games have a long lifespan as some of them have been in production since the early 00’s – therefore, there is an ever-growing amount of patches and expansions to be tested and released. And, most importantly, gamers invest in games. They expect things to work!
Game Test Automation: Potential advantages and false expectations
Test automation advantages for the gaming industry? Lower development costs and faster time-to-market – if we put that simply. This goal can be achieved due to the ability to run more tests in less time (on different hardware combinations, OS’s, and program versions), reuse them, and optimize the overall use of resources.
However, nothing happens instantaneously. “Automagic” can impose unrealistic expectations, so it is imperative to set them accurately, according to the realities of test automation. Like, not every test should be automated – a great deal of what is being done manually will not be suitable for automation. Or one cannot buy all the tools off the shelf – proprietary game code requires custom-written automation scripts too.
Factors to consider before automation
There are five core things one needs to take into account for a successful game test automation, namely production costs, test automation infrastructure, reusability, human factor, and automation scale.
Cost of production
Hiring new developers to write scripts and other test utilities implies significant initial investment – even though it will pay off eventually. Additionally, automated test scripts are like any other code – they may come with their own set of bugs (if written by non-QA pros).
Test automation infrastructure
A solid infrastructure for automated tests to be placed onto should be included in the production costs – but this point needs extra attention. It is a starting point and essential condition to be able to adjust test code to the changes in a game build/functionality/features set, etc. The system should be really bulletproof to get integrated with game development.
Reusability
Reusability is one of the main advantages of test automation scripts, but there is a thing about the gaming industry that makes it a little more complicated. Unlike other software products, each game is built on new code. That is why game test automation requires a particular approach. Such as providing functions that hide the game code details from the automated tests to maximize test reusability.
The Human Factor
Test automation does not eliminate the need for human testers. At the end of the day, some tests cannot be automated (like ad hoc or playtesting. for example). However, it will definitely leave more time for the manual assessment of the playability, balance, and other issues.
Automation scale
The more aspects of testing the dev studio intends to automate, the more factors they have to take into account. First of all, it inevitably brings a fundamental shift in the way the company operates. Secondly, it is a long-term investment that will not pay off immediately. Finally, automation is not just simply purchasing a magic tool and throwing it into the development mix. Automation needs persistence.
That is why it is imperative to assign this task to experienced test automation engineers who would not imperil the whole thing. A trusted provider of independent software testing services will arrange a team of seasoned professionals on-demand, able to handle the automation process flawlessly.
What Games to Automate?
Most of the games would benefit from some degree of automation, but some of them require it full-scale.
- Massively Multiplayer Games: for automating the elements that have many repetitive actions or get synchronized among a large number of servers and clients simultaneously, and for simulating server load.
- First-Person Shooters: for imitating a player repeatedly shooting in a certain pattern, randomly testing paths through a level to detect clippings, sticky spots or invisible walls, and retesting the GUI of the game with every new build.
- Puzzle Games: for specifically testing what would happen if the gamer makes moves the game developer does not expect them to make.
- Mobile games: for automating the testing of game screen displays, press actions, and menu selections.
Summary
From increased efficiency, coverage, and cost savings to better game quality – there are clear potential advantages the gaming industry can gain from test automation. Even though it is a common practice these days to start automating tests in the later stages of development, game developers should start as early as possible and test throughout the development cycle.
Entrust test automation to the QA pros from QATestLab. We offer the testing services on-demand, work with projects of any scale, and are ready to start with a few days’ advance notice.
Learn more from QATestLab
Related Posts:
About Article Author
view more articles
Good post with a lot of solid information.
I had a bit of a smile at this quote:
“Additionally, automated test scripts are like any other code – they may come with their own set of bugs (if written by non-QA pros).”… code is code, when we become the developer, we are as prone to entering bugs as any other developer. I think that testers who become automation developers are more open to considering testability and are more likely to look for bugs in their own code, so we catch them before we “ship” our automated tests… but we still screw it up 🙂
The only thing I would question in your post is the note that “adopting CI/CD practices that REQUIRE them to test and release faster”. Rather I would say that companies or teams adopt CI/CD practices to ALLOW them to release faster.
There is a lot of development focused automated testing that can happen in a well defined CI/CD pipeline, but the rapidity of the flow limits the amount of end-user based game feature testing that can be done. That being said, your conclusion is still totally accurate, if a company is investing in CI/CD in order to release faster, automated testing is something they absolutely need to invest in.
If we differentiate between automated tests that follow a more development flow (unit, feature, function based. largely independent of the full game flow) and automated tests that follow a more test centric flow (uses the game and in-game functions and more closely follows a tester workflow), this will help us identify what is likely to be in a build pipeline.
Again though, great post.