Board Game Arena

How do you quantify skill in a game? There are mathematical methods to evaluate the decision space, that work for abstract games like Chess or Go, but there is nothing for modern board games.

Board Game Arena's ELO ranking system rewards consistency. If you play a lot of games and win a lot, you will have a high ELO. If you assume that the high numbers of players at BGA evens out ability, you can correlate the high ELOs of top-ranked players with high skill ceilings of the games. In other words, high skill games will give big rewards to the highest skill players. A classic example might be chess - Magnus Carlsen is never going to lose to an average club player. On the other hand, no one above the age of 8 is going to lose a game of tic-tac-toe.
Board Game Arena - ELO ranking of top player in the top 50 most played games.

To test my hypothesis, I looked at the ELO of the top ranked player for the top 50 most played games at BGA. In general the most obviously skillful games mapped to the highest ranks. For example, Hive (ELO 1015) and Azul (ELO 1005) are abstracts with lots of tough choices and consequences. Also the most obvious luck-fests mapped to the lowest ranks, for example LLAMA (ELO 402) or King of Tokyo (ELO 476)

There were a few surprising outliers. Earth (ELO 539) and Viticulture (ELO 576) are both proper euros with play times over an hour, but the top players have relatively low ELOs. Patchwork is a 15 -minute pentomino game, but the top player has an ELO of 975.



It seems that high ELOs correlate to shorter games. eg. Patchwork or Azul, punishing slower games like Ark Nova (ELO 728) or Terra Mystica (ELO 740), but Tzolkin (ELO 961) is long and does well.

Thousands of people are playing these games to win and rank up every day so there must be some truth in this. Some games have gone up in my estimation and some have gone down.

What do you think? Are you surprised at some of these results?

"Viticulture" is a stonemaier game, known for pretty bad balanced game design. Viticulture cards can have a very huge impact or basically none, in addition to the normal wine draw RNG. Games like Tapestry are even worse.

But nice and interesting table!

Calmon70 wrote: ↑21 August 2023, 19:53 "Viticulture" is a stonemaier game, known for pretty bad balanced game design. Viticulture cards can have a very huge impact or basically none, in addition to the normal wine draw RNG. Games like Tapestry are even worse.

But nice and interesting table!

I agree. RNG definitely has some impact on many games. The skill ceiling becomes lower the more RNG the game has.

Not, I think, a surprising set of results. Elo as a measure is going to confound several different factors. You would need a proper analysis of those variables to try and disentangle the different components and begin to use it to get a sense of what you are after. For example, the top Elo score will tend to be higher when the playing community is larger (that would be true even of Snakes and Ladders). So you would probably need to compare say the player at the tenth percentile rather than the top. Another example, Elo has a well known 'inflation' effect caused by players entering with a fixed Elo, losing games, thus contributing their points to the pool, then leaving.

I find your approach very interesting.

Please also consider this : some players play exclusively with friends (or themselves with another account) and always win. That obviously artificially inflates their elo, regardless of the game played.

One way to avoid this would be to look at the top arena scores at the end of each seasons.

Cheers

This was posted on BGG as well, and the comments there are:
1) Elo was designed for two-player games and doesn't work as well for more than that
2) The fact that people can choose their opponents is problematic (as mentioned by the previous poster)
3) Inflation (as mentioned above)

One more thing to consider is how appealing the game is for lesser skilled players. In order to have high ELO, the points need to come from somewhere. If only high-skilled players are active, they will swap the points from one player to the other and back. To feed the system, you need a lot of lower-level players who are willing to repeatedly hit the 100 ELO-bar and thus hand out free points to the masses.

I have a thought, but have done zero research for it.

Is there any correlation between the amount of players/plays and how high ELO can get?
It seems to me (as I said just gut feeling) that games with more players/plays can get higher ELO than games with less players/plays.

Any thoughts on this?

Jeronimon wrote: ↑22 August 2023, 18:09 I have a thought, but have done zero research for it.

Is there any correlation between the amount of players/plays and how high ELO can get?
It seems to me (as I said just gut feeling) that games with more players/plays can get higher ELO than games with less players/plays.

Any thoughts on this?

Just spot checking the data in the first post indicates this is clearly not the case. Compare Wingspan, Lucky Numbers, Can't Stop with Patchwork, Puerto Rico, Hive. Might it be a small factor due to ELO inflation? Sure, but it's a relatively small impact, perhaps even negligible, when compared to a game's amount of randomness.

The overall results here are fairly unsurprising. The best players in a game reach and maintain higher ELO when the games include less randomness. I don't have data to back it, but I suspect friendly games make up a relatively small portion of games played by top ELO players. In most cases they're playing for competitive drive.

Fletcheese wrote: ↑22 August 2023, 21:10 The overall results here are fairly unsurprising. The best players in a game reach and maintain higher ELO when the games include less randomness. I don't have data to back it, but I suspect friendly games make up a relatively small portion of games played by top ELO players. In most cases they're playing for competitive drive.

Completely agree with this.

It's definitely not perfect, but I also use the OP's methodology as a heuristic for estimating the available skill ceiling in a game.

The biggest problem with OP's comments is that they didn't recognize Patchwork as an abstract