Machine learning could create the perfect game bosses

Camping at the location of the orbs is a solid strategy: the player must collect the orbs to win (imagine if Pac-Man’s ghosts just lingered near the entrances to every corner of the map). It also makes the game less fun. Players no longer experience thrilling chases. Instead, the AI ​​can set up an unpredictable ambush. Trachel and Peiro say their goal is “not to create superhuman bots – that would be uninteresting and uninteresting for a novice gamer – but instead to find ways to incorporate machine learning into the AI ​​game tools that are already being used in production.”

This may seem boring for players who are hungry for better AI. However, the machine learning methods shown by Trachel and Peiro remain useful for setting difficulty even if the opponents players face in the finished game don’t use them. Julian Togeliusco-founder and director of research at Modl.ai, has spent nearly five years using AI to test games. Modl.ai uses bots to find graphical glitches, flaws in world geometry, and sniff out situations that make it impossible to win.

“You can tell us which failure state you are interested in. And then, basically, everything will work. You submit an assignment and it gets done depending on how much you want to learn,” says Togelius. “And, of course, we can group them for you and provide a report, pointing out where you seem to be having problems, and so on.”

Modl.ai testing bots use machine learning to adapt to each game being tested, although its current implementation limits these adaptations to each specific game. Togelius says the company is developing a prototype deep learning system that will teach bot behavior across multiple games. Once used, Modl.ai bots will learn to mimic the behavior of real players, which should be more effective at identifying issues that players might discover.

Game engines need a revolution for true machine learning

When it comes to challenges, machine learning can be both a problem and a solution. But creating a challenge that’s fair and fun isn’t the only hurdle that developers face when they want to use machine learning in games. The problems run deeper, so deep that they could force us to rethink how games are made.

Performance is one of the barriers. Machine learning requires a lot of training data for worthwhile results, and that data can only be obtained by playing the game thousands or tens of thousands of times (although bots can lighten the load, a tactic Thrashel and Peiro used when creating their demo). And once the training data is collected, the resulting model can become cumbersome to run in real time.

“Yes, performance is clearly an issue, especially with large machine learning models that process frames for every tick of the game clock,” Trachel and Peiro said in an email. “In our case, to avoid performance issues, we used a small neural network that only made inferences at certain points in the game.” Scaling up to the massive open world that today’s gamers expect is another matter entirely.

Togelius says the way modern game engines work exacerbates the problem. Machine learning, he says, “is necessarily going to be slow because game engines aren’t built for it. One of the many reasons we don’t see more interesting modern AI in games is because Unreal, Unity and all of them are basically terrible – anti-AI in many ways.”

Animation is a separate issue. Most modern game engines require animation to be strictly defined frame by frame. This works well when the animators know exactly how the game characters will behave, but machine learning-driven AI can behave in ways that the animators didn’t expect. Designers can work around this with a physics-based approach to animationbut this places an even greater strain on the performance of the game console or computer hardware, and comes with its own development challenges.

In short, the developers are faced with a monster created by them. Game engines are built to use behavior trees and prescribed actions to create AI-driven NPC worlds that work well even on meager hardware. But as machine learning gains momentum, these classic solutions will need to be revisited.

“If you talk to a machine learning researcher who doesn’t understand game design, he’ll say, ‘Why don’t you use new things and get NPCs that are more realistic and adapt to how you play?’ and so on,” says Togelius. “But you can’t just plug it into an existing game. You have to rethink what a game is.”