Custom client for AI/ML

Discussion around programming R'n'D, its source code and its tools.

Moderators: Zomis, Flumminator

Post Reply
tuero
Posts: 1
Joined: Fri Mar 13, 2020 2:19 am

Custom client for AI/ML

Post by tuero » Sat Mar 14, 2020 7:42 am

I would to first off like to thank everyone involved in the development and maintainment of RnD! I am a Masters Computer Science student focusing on tree search and machine learning methods. A popular research environment for machine learning is in games, as it provides hard challenges (large action/state spaces) and its easy to simulate to gather data or run experiments.

My thesis advisor played Boulderdash and Emerald Mine back in the day, and knew about the RnD client being open source with extensive levels (and custom levels readily available), and so thought that this would be an interesting research environment to create policies which can learn to play these games.

To allow use for my thesis project, I’ve done a lot of work to create an interface/hooks into the engine to allow input from a custom controller to send actions for the agent, and helper functions for common engine state information. I’ve also modified many parts of the game code to create a headless (no SDL image/sound) binary version of the game to allow for faster simulations of the game engine (helpful for simulation based agents such as MCTS which require faster than real-time engine simulation).

The project design has been based primarily around my thesis work and what I needed at the time, so the code is by no means the most elegant. It is at a point now where there shouldn't be major design changes, and I make no promises that it is bug free! However, there may be other people here which would like to play around with their own agents, or see the changes that I’ve made to inspire their own projects.

You can find the repo here: https://github.com/tuero/rocksndiamonds
It should compile fine on Linux/Mac with gcc/clang and cmake (install instructions/docs are on the repo page). It would be nice to get Windows support, but porting over has been non-trivial and I have been busy with my thesis work.

I hope this inspires others! :D

User avatar
Holger
Site Admin
Posts: 3268
Joined: Fri Jun 18, 2004 4:13 pm
Location: Germany
Contact:

Re: Custom client for AI/ML

Post by Holger » Sat Mar 14, 2020 8:11 am

Wow, this project looks extremely cool! Only had a first, very quick glance over the GitHub page so far, though.

I've already discussed the idea of having an AI client play R'n'D with some other people years ago (with no practical result), as I found it absolutely fascinating, but was always convinced that it would be totally impossible to come around with anything remotely usable. Apparently you mastered it, when I look at the animated GIF at your GitHub page! Wow! :shock:

Regarding "headless mode", there's something similar in R'n'D (check for the "program.headless" variable), which I use for automated tape tests with maximum engine speed (without frame delay) and without GUI (using the "autotest" command), but it still requires the SDL libraries. Will have a look at your implementation! :D

I will have a deeper look into your project, which indeed looks totally inspiring and awesome! :)

Post Reply