Grounding natural language commands to StarCraft II game states for narration-guided reinforcement learning

Nicholas Waytowich; Sean L. Barton; Vernon Lawhern; Ethan Stump; Garrett Warnell

doi:10.1117/12.2519138

10 May 2019 Grounding natural language commands to StarCraft II game states for narration-guided reinforcement learning

Nicholas Waytowich, Sean L. Barton, Vernon Lawhern, Ethan Stump, Garrett Warnell

Proceedings Volume 11006, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications; 110060S (2019) https://doi.org/10.1117/12.2519138
Event: SPIE Defense + Commercial Sensing, 2019, Baltimore, MD, United States

Abstract

While deep reinforcement learning techniques have led to agents that are successfully able to learn to perform a number of tasks that had been previously unlearnable, these techniques are still susceptible to the longstanding problem of reward sparsity. This is especially true for tasks such as training an agent to play StarCraft II, a real-time strategy game where reward is only given at the end of a game which is usually very long. While this problem can be addressed through reward shaping, such approaches typically require a human expert with specialized knowledge. Inspired by the vision of enabling reward shaping through the more-accessible paradigm of natural-language narration, we investigate to what extent we can contextualize these narrations by grounding them to the goal-specific states. We present a mutual-embedding model using a multi-input deep-neural network that projects a sequence of natural language commands into the same high-dimensional representation space as corresponding goal states. We show that using this model we can learn an embedding space with separable and distinct clusters that accurately maps natural-language commands to corresponding game states . We also discuss how this model can allow for the use of narrations as a robust form of reward shaping to improve RL performance and efficiency.

Conference Presentation

Citation Download Citation

Nicholas Waytowich, Sean L. Barton, Vernon Lawhern, Ethan Stump, and Garrett Warnell "Grounding natural language commands to StarCraft II game states for narration-guided reinforcement learning", Proc. SPIE 11006, Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, 110060S (10 May 2019); https://doi.org/10.1117/12.2519138

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available