ELO on luck based games

[Retired Actuary]

Thanks FrankJones for pulling all those additional games, that was very interesting. I did see a lot of individual outliers when I was looking at those, which is how I went to top 10 (which is also a convenient # to copy since the site has no API).

dschingis27, great earlier post and follow-up. (also, I had no idea the K for Arena is stuck at 40 and never reverts to 20. Why??)

On a related note, I went on a kind of ELO-paper-bender about a year ago, reading anything I could find, and as best I recall (these were all based on simulations) any reasonable K got someone to a run rate neighborhood of their 'true' mean (again, these are simulation) at most somewhere around 250 games. What that would miss however is that the population of players on BGA comes in with a mix of expertise, and some people improve a lot...so some larger # is likely needed to see stability. Don't know what that is though.

On the graph I posted above, the one that interested me most was 7 Wonders, living in a place of its own. I feel like that has something to do with the high player counts for the game...it's a game I would notionally put in a higher group, but either the luck of more players scales differently, or the way ELO is averaged introduces a bias. (or I'm wrong, and that's the right place for it).

Finally, I introduce a theorem that the more options (i.e. expansions) a game has, the higher the opportunity to acquire expertise and reduce luck. I wish I could go back and look at some of these games (e.g. Ark Nova) before their expansions came online.

[tbhp]

Here is an interesting one: Kulami (A zero luck game, no hidden information, pure abstract strategy along the lines of Go, chess, etc.): 324-509.

There are other elements that come into play. Even if a game has zero luck, if people generally don't have a good understanding of how to play the game, then you won't see ELO go very high. It's also possible that a zero luck game doesn't offer a very high variance in skill. Tic Tac Toe, for an extreme example, would have everyone at roughly the same ELO in a competitive setting, because among people who know how to play it, the winrate is always 50% (the person who plays first wins).

[Emerson_Matt]

Here is an interesting one: Kulami (A zero luck game, no hidden information, pure abstract strategy along the lines of Go, chess, etc.): 324-509.

There are other elements that come into play. Even if a game has zero luck, if people generally don't have a good understanding of how to play the game, then you won't see ELO go very high. It's also possible that a zero luck game doesn't offer a very high variance in skill. Tic Tac Toe, for an extreme example, would have everyone at roughly the same ELO in a competitive setting, because among people who know how to play it, the winrate is always 50% (the person who plays first wins).

I don't have high hopes for your tic-tac-toe rating.

[FrankJones]

I'm not sure a lower K-factor will help. To me, 20 is already a low k-factor. Anyway, a lower k-factor would create a tighter bunching among players who perhaps should not be tightly bunched.

A lower K factor would help for sure and it would not create a tighter bunching among players. The mathematics and simulations are pretty clear. Here is a simulation I made a while ago: https://boardgamearena.com/forum/viewto ... 38#p222238
Note that the K factor does not influence the calculated expected win probabilities itself, no matter the K factor, the Elo difference between players has the same meaning. The K factor only influences how strong a single game result changes the Elo score of one player. If single games influence the current Elo score less, then more games over longer time determine the current Elo score more equally. That doesn't mean that the Elo scores between different players cannot spread out over time, it just takes longer for them to spread.

Suppose we decide to use a lower or higher k-factor for games depending on the amount of luck involved. Who would make these determinations, and how?

The dev / publisher or someone at BGA has to decide. It should just be a simple one of the many decisions they have to make when producing a new game on BGA. I suggested a system with only 3 categories to make the decision easy, choose between K=5, K=10, K=20. If we wanted to get an optimal precise K factor for each game, that cannot work, I agree. But having three categories would already be so much better than the current system. Currently, a single game of Terra Mystica counts the same as a single game of Can't Stop which is absolute bonkers.

Sure, sometimes a player whose true skill Elo is 300 might have snuck up to 400 on a lucky streak, and might get matched against a player whose true skill Elo is 500 but has dropped to 400 on an unlucky streak. Isn't it true this can happen in any Elo system?

This would happen much less with lower K factor, however, yes it is inherent in Elo systems so we might want to use a different system.

I remember those charts from that other thread on this topic. I'm not interpreting the data the same way you are. Maybe we are not on the same page though. I'll attempt to clarify when I have more time later today.

[FrankJones]

On the graph I posted above, the one that interested me most was 7 Wonders, living in a place of its own. I feel like that has something to do with the high player counts for the game...it's a game I would notionally put in a higher group, but either the luck of more players scales differently, or the way ELO is averaged introduces a bias. (or I'm wrong, and that's the right place for it).

I have a hypothesis. 7 Wonders is not really a 2-player game. Playing it 2-player requires the neutral opponent, which I and my gaming partner always found annoying. So, we saved 7-wonders for game night when we could plat it 3+ player.

Why is that relevant? 3+ player games can have weird effects on Elo on BGA. Anytime someone quits or runs out of time in a 3+ player game, all other players are considered to "tie" with one another, thereby often resulting in minimal or zero Elo gain. This might drag down overall Elo for the game, by making it more difficult to attain a high Elo.

[Andrei3009]

On the graph I posted above, the one that interested me most was 7 Wonders, living in a place of its own. I feel like that has something to do with the high player counts for the game...it's a game I would notionally put in a higher group, but either the luck of more players scales differently, or the way ELO is averaged introduces a bias. (or I'm wrong, and that's the right place for it).

I have a hypothesis. 7 Wonders is not really a 2-player game. Playing it 2-player requires the neutral opponent, which I and my gaming partner always found annoying. So, we saved 7-wonders for game night when we could plat it 3+ player.

Why is that relevant? 3+ player games can have weird effects on Elo on BGA. Anytime someone quits or runs out of time in a 3+ player game, all other players are considered to "tie" with one another, thereby often resulting in minimal or zero Elo gain. This might drag down overall Elo for the game, by making it more difficult to attain a high Elo.

I think that in games with a large rating difference it would be advantageous for weak players to hate the strong ones and therefore it is difficult to climb rating. I've often seen this in the 3p arena.

Besides, 7 Wonders is just a pretty random game (but not at the casino level like can't stop and flip 7)

[FrankJones]

Yes, that too.

Another example: Hearts. Despite literally millions of games played, nobody can maintain an Elo above 500. Even though hearts of a game of great skill (in the long run), three is massive variance in the short run, especially with the preferred target score being set to 75 instead of the traditional 100.

Hearts being a 4-player non-team game likely contributes to the low Elo ceiling in various ways.

[tbhp]

I don't have high hopes for your tic-tac-toe rating.

Oh no... I was wondering if I was remembering that right...

[Emerson_Matt]

Oh no... I was wondering if I was remembering that right...

It's a tie with best play. I'm sure you'd be fine after playing it for 2 mins.

[Name_Taken]

Here is an interesting one: Kulami (A zero luck game, no hidden information, pure abstract strategy along the lines of Go, chess, etc.): 324-509.

There are other elements that come into play. Even if a game has zero luck, if people generally don't have a good understanding of how to play the game, then you won't see ELO go very high. It's also possible that a zero luck game doesn't offer a very high variance in skill. Tic Tac Toe, for an extreme example, would have everyone at roughly the same ELO in a competitive setting, because among people who know how to play it, the winrate is always 50% (the person who plays first wins).

THIS is why removing luck might make a game feel more fair, but it doesn't make it more interesting. If there is no randomness, then there will only ever be one "best" way to win a game, and whoever goes first and follows the best "script" will always win. IMO that's just as boring as luck.