Bots in the Azul Arena

[Mawhrin]

With the advances in AI it seems people are literally just saying “build me an Azul bot” and achieving good results haha.

I have minimal Harmonies experience but I believe the game is more complicated than Azul from a possible-actions standpoint. Azul is relatively simple in that regard.

Some simple prompting will get you a weak Azul bot (it will beat 95% of players). That and wiring it into BGA will take maybe a day or two. Going further, you can expect to spend 6-12 hours per round of training on decent laptop. Training will plateaux around 6-8 round then you need to introduce increasingly advanced / less known techniques and continue training. At this point you need to have working knowledge of AI because ChatGPT etc will just as likely take you down multiple dead ends instead of improving things. In total you are looking at 4 to 6 weeks to create sometime genuinely powerful.

[Mawhrin]

Maybe we need a panel of community moderators that would have enough power to make it not worth cheating? Eg reset arena score

[FrankJones]

A TfmBot was created that is learning to play Terraforming Mars surprisingly well.

Azul is a game with minimal randomness and a small rule set and action space. I am not surprised a bot can play it well.

Other strong bots include Patchwork (zero hidden information and zero randomness once the tiles are laid out and player 1 is chosen) and Race for The Galaxy.

The Strongest Patchwork bot is almost impossible to win against for anyone other thana master-level human.

I'm 300 Elo in RFTG and I get clobbered by the Hard RFTG bot most of the time.

[GTSchemer]

A TfmBot was created that is learning to play Terraforming Mars surprisingly well.

Azul is a game with minimal randomness and a small rule set and action space. I am not surprised a bot can play it well.

Other strong bots include Patchwork (zero hidden information and zero randomness once the tiles are laid out and player 1 is chosen) and Race for The Galaxy.

The Strongest Patchwork bot is almost impossible to win against for anyone other thana master-level human.

I'm 300 Elo in RFTG and I get clobbered by the Hard RFTG bot most of the time.

I was under the impression that bot accounts (and using BGA player actions / data to train) was still against the terms of service. Did something change in this regard?

[OriProf]

Looks like Sam from No Wonders did a video on precisely this about a year ago (with a shout out to Priptonite ). The chap that was profiled as number one now has his Azul ELO at zero. So it would appear that with compelling evidence BGA takes action. Now that I think of it, there was another case before that - this seems an area of some drama. Before this chat I did not appreciate that 'bot' may be a program or it may just be a player that plays using one (i.e. manually). I wonder if the solution is to have a bot vs. bot version - kind of like F1 on one hand between humans and RC racing between bots. Hard to say.

[Mawhrin]

BGA rules seem clear about bots that ‘automate the placement of moves’ and less clear about those that just calculate the best move.

[Priptonite]

Looks like Sam from No Wonders did a video on precisely this about a year ago (with a shout out to Priptonite ). The chap that was profiled as number one now has his Azul ELO at zero. So it would appear that with compelling evidence BGA takes action. Now that I think of it, there was another case before that - this seems an area of some drama. Before this chat I did not appreciate that 'bot' may be a program or it may just be a player that plays using one (i.e. manually). I wonder if the solution is to have a bot vs. bot version - kind of like F1 on one hand between humans and RC racing between bots. Hard to say.

Yeah we were able to get that one shut down, but they’ve only gotten more prominent since then. That was the first one we noticed, but after seeing the timing patterns we identified that a previous high-level “player” was also a bot. The creator of the bot in the No Wonders video did eventually join our discord and discuss their learnings and whatnot. The bot was part of their masters thesis.

[dschingis27]

I think the whole gaming industry is affected by the current AI development. Chess has the problem of AI assistance in online competitions for many years now and they established counter measures at least in high-stakes tournaments that would not be feasable in the BGA world (like players must film themselves with cameras and some other things). Think of online poker where you even play for real money, AI bots are a big problem there. And the list goes on and on.

At the end of the day, I think you will have to fight AI with AI. In the long term, ideally there is some basic automated AI/cheat detection algorithm deployed on BGA (e.g., checking for typical AI behavior like regular move-making 1 second after it's your turn in turn-based games). Cheat detection software is already mandatory in some video games (e.g., Easy-Anti-Cheat in Rocket League https://www.thespike.gg/rocket-league/b ... ket-league). If at least the obvious bots and "easy botting" could be prevented automatically, that would be a big step forwartd. Some sneaky bot players will always be hard to evaluate but if we can make things at least very inconvenient for potential AI bot cheaters, that would be a big help. In the long run, if we try to fight only with human effort without at least some kind of automated pre-filtering, I don't think that works out.

I think BGA will not re-invent the wheel but has to draw from experience from other gaming sites and hopefully there is something that can be used for BGA.

[tbhp]

BGA rules seem clear about bots that ‘automate the placement of moves’ and less clear about those that just calculate the best move.

But that's the same thing, what is the point of forbidding bots if a person using a bot to know the best move is still allowed?

Why do you think they are not clear?

[FrankJones]

BGA's policy makes it very clear it is not allowed to use a bot to calculate the best move.