Machine learning models already master the games of chess, go, Atari, etc. .
“We wanted to build what we think is the most accessible ‘big challenge’ with this game. It won’t be solve AI, but it will open up avenues for better AI, ”said Edward Grefenstette of Facebook AI Research. “Games are a good place to find our assumptions about what makes machines smart and break them.”
You may not be familiar with Nethack, but it is one of the most influential games of all time. You are an adventurer in a fantasy world, diving into the increasingly dangerous depths of a different dungeon each time. You have to fight monsters, navigate traps and other dangers, and meanwhile stay on good terms with your god. This is the first “roguelike” (after Rogue, its immediate and much simpler predecessor) and arguably still the best – almost certainly the most difficult.
(By the way, it’s free, and you can download and play it on almost any platform.)
Its simple ASCII graphics, using ag for a goblin, an @ for the player, lines and dots for the level’s architecture, etc., belies its incredible complexity. Because Nethack, which debuted in 1987, has been in active development ever since, with its changing team of developers expanding its roster of objects and creatures, rules and the myriad, countless interactions between them all.
And that’s part of what makes Nethack such a difficult and interesting challenge for AI: it’s so open. Not only is the world different every time, but every object and creature can interact in new ways, most of which have been hand-coded over the decades to cover every possible choice for players.
“Atari, Dota 2, StarCraft 2… the solutions we have had to progress there are very interesting. Nethack simply presents different challenges. You have to rely on human knowledge to play the game as a human, ”said Grefenstette.
In these other games, there is a more or less obvious strategy to win. Of course, it’s more complex in a game like Dota 2 than in an Atari 800 game, but the idea is the same – there are parts the player controls, a game environment, and victory conditions to look for. . It is a bit the case in Nethack, but it is stranger than that. On the one hand, the game is different every time, and not just in the details.
“New dungeon, new world, new monsters and items, you don’t have a save point. If you make a mistake and die, you don’t have a second chance. It’s a bit like real life, ”said Grefenstette. “You have to learn from your mistakes and approach new situations with that knowledge. “
Drinking a corrosive potion is a bad idea, of course, but what about throwing it at a monster? Coat your gun with it? Pour it on the lock of a treasure chest? Dilute it with water? We have intuitive ideas about these actions, but an AI that plays the game doesn’t think like us.
The depth and complexity of Nethack’s systems is hard to explain, but this diversity and difficulty makes the game a perfect candidate for a competition, according to Grefenstette. “You have to rely on human knowledge to play the game,” he said.
People have been designing robots for playing Nethack for many years that do not rely on neural networks but on decision trees as complex as the game itself. The Facebook Research team hopes to spawn a new approach by creating a training environment in which people can test machine learning-based game algorithms.
The Nethack learning environment was created last year, but the Nethack challenge is only just beginning. The NLE is essentially a version of the game integrated into a dedicated computing environment that allows an AI to interact with it via text commands (directions, actions like attacking or eating)
It’s a tempting target for ambitious AI designers. While games like StarCraft 2 may benefit from a higher profile in some ways, Nethack is legendary, and the idea of building a model on completely different lines than those used to dominate other games is an interesting challenge.
It’s also, as Grefenstette explained, more accessible than many in the past. If you wanted to create AI for StarCraft 2, you needed a lot of computing power to run visual recognition engines on the in-game images. But in this case, the whole game is transmitted by text, which makes it extremely efficient to work. It can be played thousands of times faster than any human, even with the most basic computer setup. This leaves the challenge wide open to individuals and groups who don’t have access to the kind of high-powered setups needed to power other machine learning methods.
“We wanted to create a research environment that presented many challenges for the AI community, but not restrict it to only large university labs,” he said.
Over the next few months, NLE will be available for people to test out, and competitors can basically build their bot or AI any way they choose. But when the competition itself begins in earnest on October 15, they’ll be limited to interacting with the game in its controlled environment via standard commands – no special access, no RAM inspection, etc.
The goal of the competition will be to complete the game, and the Facebook team will track how many times the agent “rides”, as it is called in Nethack, within a set amount of time. But “we assume it will be zero for everyone,” admitted Grefenstette. After all, this is one of the toughest games ever made, and even humans who have been playing it for years struggle to win even once in their lifetime, let alone multiple times. after. There will be other scoring measures to judge the winners in a number of categories.
The hope is that this challenge provides the seed for a new approach to AI, which more fundamentally resembles actual human thought. Shortcuts, trial and error, score hacking and zerging won’t work here – the agent has to learn systems of logic and apply them flexibly and intelligently, or die horribly at the hands of a centaur or a rabid owl.
You can consult the rules and other specifics of the Nethack Challenge here. The results will be announced at the NeurIPS conference later this year.