codecentric go challenge 2014: Final Interviews

The codecentric go challenge 2014 is over. Franz-Josef Dickhut managed to defeat Crazy Stone, one of the two strongest go programs worldwide, in four games with three wins to one. You can replay and download the games at go.codecentric.de .

Congratulations to Franz-Josef Dickhut for winning the match and thanks a lot to codecentric for the sponsoring!

As just before the start of the match, we let the contenders speak for themselves.

Let’s start with the winner, Franz-Josef Dickhut (FJD) (translated from German by the author):

1. The games against Crazy Stone were hard fought and, as one could assume from your comments during the competiton, were hard work for you, too. Have you been surprised by the playing strength of Crazy Stone or did you expect such heavy fighting?

FJD: I was not too surprised concerning the playing strength. I expected it to be some piece of work. But the pressure was quite big after the first game went wrong, which made the work that much more exhausting.
I thought it was ultimately a great pity that the final game was actually the least interesting one. From the start Crazy Stone played unexpected moves, which were probably also weak or at least played too early. Along came the technical problem, the white group died and the game was as good as over. Unfortunately, this was no fitting end for such a close and hard fought match.

2. In the end your defeat in the first game remained an aberration. What happened in round 1? What was different compared to games 2, 3 and 4?

FJD: In round 1 everything happened that should not happen: A position that fits the strenght of bots (Moyo [a large but loosely defined sphere of influence that may later become territory]), careless playing and mistakes on my part as a consequence. Because of this Crazy Stone was leading by a lot early on. This powerfully brings to bear the strengths of Monte Carlo programs.
Beginning with the second game, I mainly played more carefully. This was in order not to risk a definitive disadvantage early on again. To relax, I listened to classical music during the games. Maybe this helped, too.

3. One question from go player to go player: At which moments were the games mostly decided according to your judgement? This is hard to guess particularly for somewhat weaker players – for me, too. The reason for this is that Crazy Stone confused me with its many kikashi moves [urgent threatening moves that have to be answered right away by the other player]. I rarely understood if the bot was playing particularly clever or was already in panic mode because it believed it was losing.

FJD: Game 1: Was actually over after 70 moves. But I believe that it may had have become interesting again if I had played move number 170 on the intersection of move number 171.
Game 2: Only with move 183 I was absolutely sure, although I had a good feeling already around move 143.
Game 3: This was the tightest and most difficult game of the match. As the only win with white it may well have been the decisive game. During the game I realized only around moves 233/234 that I had won. Actually, the game was decided with move 210.
Game 4: After the connection with move 81. Thereafter, the white group on the right side was simply and without any compensation for white dead.

4. What are in your judgement the specific strengths and weaknesses of Crazy Stone? Are these – according to your experience – typical for Monte Carlo bots or rather specific for Crazy Stone?

FJD: First, I believe that Crazy Stone is one of the Monte Carlo programs that is hardest to set-up. This is although it still suffers from the typical problems.
Obviously, one of these is the weakness with kos [a specific position with repetitive captures], i.e. an aversion to play ko and the relative incompetence in evaluating ko positions. This is also true for certain corner and nakade patterns [nakade describes groups with a potentially dead shape]. Then there is one weakness that is the big strength, too: If a bot has a good overall position, they are incredibly flexible and at the same time unflinching on their way to secure the win. But if a bot is behind, even only slightly, and recognizes this, then they actively search for a possibility to force getting in the lead again. For a human player on the other side, this looks naive and as if the bot is in panic. But experienced players know that the best one can do in such situations is to keep the game vague. Either one makes the game bit by bit closer again or one increases the variance; and additionally one partly waits for “unforced” errors of the opponent. The bots are lacking in these things in my opinion, at least concerning the fine tuning.

5. Could you stick to your plan to just play good go or did you try during the competition to exploit special weaknesses of Crazy Stone?

FJD: I tried to play the best moves, at least according to my abilities. In situations where there were roughly equivalent alternatives, I of course tried to choose that one which presumably creates more problems for Crazy Stone. But I didn’t play bad moves on purpose only because some known bot problem might possibly or hopefully come to bear.

6. You prepared for your opponent with the commercial version of Crazy Stone 2013. Was this preparation helpful?

FJD: It definitely did help, although at first I sort of resisted against this help. E.g. the joseki [opening moves in a corner] on the 5-3 intersection in game one was not intended this way, but then I felt the urge to experiment.

I also knew and halfway expected some simple typical joseki errors such as move 29 in game 3, omitting a strengthening move on the side after an invasion in the upper right corner, and move 22 in game 4, a one point jump instead of an extension in this 5-3 point joseki. But it is still difficult each time to transform this into a good position.

7. Did you have the impression that Crazy Stone in the match was stronger or equally strong compared to your traning version?

FJD: The match version was stronger, but without principal differences. The reason may have been the hardware, as my computer is nearly four years old. But for those times it was reasonably well equipped with an AMD 6 core processor. But I would have to look it up, if the exact configuration is of interest.

8. In retrospect, would you approach the match differently – concerning the preparation as well as the concrete games?

FJD: Not substantially. But I would probably try from game 1 onwards to play with more concentration and experiment less in order to reduce my risk for big mistakes. I suspected before the match that this would happen to me in at least one game. But the pressure increases immensely when it happens already in the first game. Then one is behind without having an idea how to tackle the opponent under normal conditions, i.e. without a blunder. This was really very unsettling for me and cost me a lot of nerves.

9. Would you be available for a similar competition next year?

FJD: Yes, with pleasure.

Rémi Coulom (RC), developer of Crazy Stone:

1. Unfortunately you did not have the grid-hardware at your disposal as you have had originally planned and thus had to run Crazy Stone from your own desktop computer. You also used the commercial version of Crazy Stone 2013 for all four games. What was the reason for this decision?

RC: The current experimental version is not much stronger than the released commercial version, so I preferred to use a stable well-tested version during the match.

2. For most observers of the games it appeared as if Crazy Stones Monte Carlo engine got into trouble the more unsettled positions remained on the board. It seemed to have difficulties to correctly evaluate certain corner patterns, too. What do you think about these observations?

RC: Crazy Stone always becomes completely confused when the playouts evaluate one life-and-death problem wrong. It is a problem of Monte Carlo programs that is well identified. It is not easy to find a good solution.

3. What are the most important reasons in your view for the losses in games 2, 3 and 4?

RC: Game 3 was very close, and I am not a strong enough player to see any obvious misunderstanding by Crazy Stone. Game 2 and 4 had big life-and-death errors in the playouts.

4. If hardware and software were the same for all four games of the competition, was there any other reason for the comparatively weaker performance of Crazy Stone in game 4, as it was perceived by many spectators of the match? Or was it just bad luck based on the misunderstanding of the big life-and-death situation on the right side of the board early in the game?

RC: Yes, it was bad luck. Also, there was a small technical problem during the game. Crazy Stone disconnected because of a network problem, then I reconnected it, but the first instance was still running in the background, thus reducing available CPU power. But it would still have played badly with the misunderstood life-and-death on the right side, even with 1000 times more CPU power.

5. The time management of Crazy Stone was very basic for the match: It thought for exactly 38 seconds each move. Do you think there is room for improvement in the time management of Monte Carlo bots or doesn’t it really matter?

RC: Yes, of course, this part could be improved. Computer go tournaments usually have a sudden-death time control, and Crazy Stone is more clever there. Optimizing time-management heuristics for such byo-yomi time control is not very interesting and I prefer to focus my energy on more important problems.

6. By the way, is Crazy Stone utilizing the thinking time of the opponent for its own calculations?

RC: Yes.

7. Did you get any new insights into Crazy Stones strengths and weaknesses because of the matches with FJ?

RC: No. The weaknesses of Crazy Stone are very well identified. The problem is I don’t know how to fix them well. Top-level Go programs have stopped making progress for almost two years now.

8. What are your priorities for improving Crazy Stone, what issues will you tackle next?

RC: The main problem is to make random playouts more clever. In the past, Crazy Stone made progress by adding more patterns to the playouts. I believe I still can get more strength by improving the patterns further, but it won’t solve everything. It is not possible to have patterns that work for every possible game. There will always be configurations that show up in games where the playout patterns get the life-and-death wrong. The solution is to adapt the playout policy online. Some algorithms have been proposed, but they don’t work yet well enough.

9. When will the next commercial version of CrazyStone be available?

RC: I can’t tell much about dates, but there will be new versions. A version for the Windows 8 store will be released very soon. For this new version, I worked a lot to improve the playing style of Crazy Stone in the endgame: when it is winning or losing by a lot, it will play less foolish moves that throw points away like it used to. Also, I made “easy” levels that should be more pleasant to play for beginners.

10. And a somewhat tougher timeframe question: How long will Franz-Josef Dickhut or other strongest amateur players still have a chance against bots? How much longer will the professional players last?

RC: I have no idea. As soon as someone finds a good way to solve the playout problem I described in answer #8, then unbeatable programs may come very fast.

11. Would you be available for a similar competition next year?

RC: Yes, with pleasure.