Inventory Management with Attention-Based Meta Actions

Keisuke Izumiya and Edgar Simo-Serra

IEEE Conference on Games (CoG), 2021

Detail PDF GitHub

Abstract

Roguelike games are a challenging environment for Reinforcement Learning (RL) algorithms due to having to restart the game from the beginning when losing, randomized procedural generation, and proper use of in-game items being essential to success. While recent research has proposed roguelike environments for RL algorithms and proposed models to handle this challenging task, to the best of our knowledge, none have dealt with the elephant in the room, i.e., handling of items. Items play a fundamental role in roguelikes and are acquired during gameplay. However, being an unordered set with a non-fixed amount of elements which form part of the action space, it is not straightforward to incorporate them into an RL framework. In this work, we tackle the issue of having unordered sets be part of the action space and propose an attention-based mechanism that can select and deal with item-based actions. We also propose a model that can handle complex actions and items through a meta action framework and evaluate them on the challenging game of NetHack. Experimental results show that our approach is able to significantly outperform existing approaches.

BibTeX

@inproceedings{izumiya2021cog,
author = "Keisuke Izumiya and Edgar Simo-Serra",
title = "Inventory Management with Attention-Based Meta Actions",
booktitle = "IEEE Conference on Games (CoG)",
year = 2021,
}

Item Management Using Attention Mechanism and Meta Actions in Roguelike Games

Keisuke Izumiya and Edgar Simo-Serra

Visual Computing (VC), 2021

Abstract

Roguelike games are challenging environments for Reinforcement Learning (RL) algorithms due to having to restart the game from the beginning when losing, randomized procedural generation, and proper use of in-game items being essential to success. While recent research has proposed roguelike environments for RL algorithms and proposed models to handle this challenging task, to the best of our knowledge, none have dealt with the elephant in the room, i.e., handling of items. Items play a fundamental role in roguelikes, but it is not straightforward to incorporate them into an RL framework because of an unordered set with a non-fixed amount of elements which form part of the action space. In this work, we tackle the issue of having unordered sets be part of the action space and propose an attention-based mechanism that can select and deal with item-based actions. We also propose a model that can handle complex actions and items through a meta action framework and evaluate them on the challenging game of NetHack. Experimental results show that our approach is able to significantly outperform existing approaches.