“Ores” Web Game Playing Bot in Python

Recently whilst looking up information on the internet I came across this tutorial on How to build a python bot that can play web games. I’ve written similar things myself before, but this looked like a nice simple method, so I thought I’d give it a go.

This is the game I picked as my target:

(You can play it here)

The goal - to write a program that will automatically play the game exactly the way a human player would play. That is, using only the information visible on screen with no behind the scenes fiddling or poking with the game’s code.

And here’s how things worked out …

I hit my first technical hitch almost immediately. The tutorial is written in Python 2.7 and uses the Python Imaging Library, whereas I only have Python 3.2 installed on my PC, which does not have a supported version of the PIL. (The huge incompatibility between Python 2 and Python 3 is something that really bugs me about the language - it makes looking up information especially difficult. As I rarely use Python I find that I often need to look things up, too.)

After some investigation I decided to use the Image processing capabilities of the PyGame library. It’s also another excuse to learn something new!

This still didn’t give me the handy ImageGrab function that the tutorial used, so I had to roll my own using win32 API calls. I then had to spend quite a bit of time working out how to get that data into the pygame Image format. After that, though, things went relatively smoothly.

I did things slightly differently to the tutorial example. Most importantly, I wrote some code (findGameOnScreen) to locate the top left corner of the game screen from within the bitmap grab of the browser window. This means that it doesn’t matter if I’ve moved the browser, or re-sized the window, or if the content has changed, say due to different sizes of advert above the game. In the tutorial code you’d have to change the x_pad and y_pad variables every time this happened.

Once the bot is running, it enters a loop where first it checks a single pixel to see if the help window has popped up. If it has then it clicks the Resume button and continues.

Then the bot samples (in readOres) a single pixel for each of the 16x10 possible ore locations to build up an internal representation of the game. It uses this representation to select the move that would clear the most blocks at once, then clicks that cell on screen.

And that’s all that is needed to create a functioning bot to play this game!

I’d thought the bot might be affected by the game’s moving graphics. For example if it grabs an image of the screen while the bricks are moving to the left, or while some of the bricks are falling down after being clicked. However, since there is no penalty for clicking in an invalid location, worst case is that the bot doesn’t make the move it thinks it is making, or possibly doesn’t see a better move. Neither of these are serious game play mistakes and the bot will recover once the screen stabilises.

Also, when the player can’t make a move he has the option to press the fast forward button in the lower right hand corner to advance some more blocks onto the screen. It wouldn’t be too tricky for the bot to detect the white pointing finger icon that appears to highlight this situation (although it should already know anyway, if it has read the screen correctly) but it is easier simply to wait for the timer to cause the blocks to advance automatically.

In practice, this version of the bot doesn’t perform all that well. On its test run it scored 135570 and reached level 9, which is actually less than my record (before I got bored) of 143675, level 9.

I think the reason for this is that it doesn’t yet handle the bombs which appear in later levels properly, treating them as empty squares. It also has quite a large delay built in to allow time for the bricks to fall away after clicking them. Finally, the algorithm I used for choosing the best move (technically it would be known as a “greedy algorithm”), is almost certainly not optimal.

In addition to that there are various bugs in the code which didn’t seem to affect its operation enough to make them worth fixing but are there, none the less. I’m afraid my coding style is somewhat lacking in the mysterious “pythonic” element, too.

For a couple of hours’ quick hacking the code does what I set out to do and I’m pleased with the end result. It was fun to watch my code take life and play the game by itself. I think adding code so that the bot can look ahead a few moves would be an interesting project, although it would require me to simulate some of the game logic. I think I’d probably also have been a lot more comfortable writing this in a language I’m more familar with such as C++.

Code is available in my public git repository here.