ELO on luck based games

Discussions about BGA (all languages)
Forum rules
Warning: challenging a moderation in Forum = 10 days ban
More info & details about how to challenge a moderation: viewtopic.php?p=119756
Post Reply
FrankJones
Posts: 2441
Joined: 30 June 2024, 00:24

Re: ELO on luck based games

Post by FrankJones »

I might be saying this I already said earlier in this post. If so, apologies. It has been a long thread.

Elo, on this site, performs several functions. Regardless of what Elo was originally designed for, or what the BGA FAQ says Elo's function is on this site, it is nevertheless true that Elo has several functions here.


Elo here is a leaderboard / record of what has actually occurred. Sometimes, what occurs might differ from what was expected to occur, and results in different Elos than what we might expect based on a player's skill. Maybe that's a problem, maybe it is not. Even at high skill games (let's take "Earth" or "Terraforming Mars", as examples), the best players reach a point at which the combination of small sample size and small player pools cause significant Elo fluctuations. Top players at those two games Fluctuate between mis-600s and low 800s Elo. A player's Elo at any given moment may not reflect that player's actual skill. A player's average Elo over a large sample would be more accurate. Maybe there is a way to use some sort of rolling average Elo when calculating the Elo change after a game?

The nature of a high-luck game is that it is more difficult for the high-skilled player to win. Whichever system is used, that will remain true, right?

I'm not sure a lower K-factor will help. To me, 20 is already a low k-factor. Anyway, a lower k-factor would create a tighter bunching among players who perhaps should not be tightly bunched.

Suppose we decide to use a lower or higher k-factor for games depending on the amount of luck involved. Who would make these determinations, and how?

To some extent, Elo provides some level of predictive value to aid in matchmaking. playing against opponents with similar Elo is more likely to result in being matched with a player of comparable skill than playing against opponents who Elo is plus or minus 200 points from the player. Sure, sometimes a player whose true skill Elo is 300 might have snuck up to 400 on a lucky streak, and might get matched against a player whose true skill Elo is 500 but has dropped to 400 on an unlucky streak. Isn't it true this can happen in any Elo system?

The different Elo spreads in different games, to me, serve a function of illustrating the luck component of the game. Hearts and Spades have low Elo ceilings, indicating a high level of luck. Zero-luck games tend to have high Elo Ceilings, sometimes approaching 1000 or even 1100. To me, that has value. It helps me decide which games I might be more inclined to enjoy playing. I'm not sure it's a problem that some games have lower Elo ceilings.
User avatar
Retired Actuary
Posts: 50
Joined: 19 April 2020, 19:50

Re: ELO on luck based games

Post by Retired Actuary »

FrankJones wrote: 28 March 2026, 18:18 The different Elo spreads in different games, to me, serve a function of illustrating the luck component of the game. Hearts and Spades have low Elo ceilings, indicating a high level of luck. Zero-luck games tend to have high Elo Ceilings, sometimes approaching 1000 or even 1100. To me, that has value. It helps me decide which games I might be more inclined to enjoy playing. I'm not sure it's a problem that some games have lower Elo ceilings.
I've posted this elsewhere here, but I was curious a while back at what the average ELO at a point in time was for the top 10 players of various games. I'd caution anyone seeing this to think of the numbers more broadly as neighborhoods than point estimates, but I think it does capture what you're talking about in terms of how much luck can be diversified away by skilled players.

Image
User avatar
ChiefPointThief
Posts: 745
Joined: 14 August 2020, 22:27

Re: ELO on luck based games

Post by ChiefPointThief »

Retired Actuary wrote: 28 March 2026, 18:38
FrankJones wrote: 28 March 2026, 18:18 The different Elo spreads in different games, to me, serve a function of illustrating the luck component of the game. Hearts and Spades have low Elo ceilings, indicating a high level of luck. Zero-luck games tend to have high Elo Ceilings, sometimes approaching 1000 or even 1100. To me, that has value. It helps me decide which games I might be more inclined to enjoy playing. I'm not sure it's a problem that some games have lower Elo ceilings.
I've posted this elsewhere here, but I was curious a while back at what the average ELO at a point in time was for the top 10 players of various games. I'd caution anyone seeing this to think of the numbers more broadly as neighborhoods than point estimates, but I think it does capture what you're talking about in terms of how much luck can be diversified away by skilled players.

Image
FYI the stats for luck base games can be misleading. For instance just looking at some of the top flip 7 players they played 12p games. Placed high in their last few games granting them hundreds of elo and then stopped. They are still under the 20 & 10 game boosted elo

Depending on when you search can’t stop it can be misleading also. Tons of ppl held onto old accounts and bga used a different pt system back then. But some ppl play a game on their old account every few yrs to get on the leaderboard.

Sometimes cheaters are at the top. For instance I remember looking at no thanks and wondered how someone got in the 500s at that game and their history showed they were clearly cheating. I’ve done this with bang before and almost all the top 10 players were cheaters at the time lol
FrankJones
Posts: 2441
Joined: 30 June 2024, 00:24

Re: ELO on luck based games

Post by FrankJones »

Out of curiosity, how do you know they were cheating? (I'm unfamiliar with the games you mentioned.)
User avatar
ChiefPointThief
Posts: 745
Joined: 14 August 2020, 22:27

Re: ELO on luck based games

Post by ChiefPointThief »

FrankJones wrote: 28 March 2026, 20:41 Out of curiosity, how do you know they were cheating? (I'm unfamiliar with the games you mentioned.)
Pattern of a dummy account letting them win in all games.

There are some games that it is easy to find cheaters. I found it funny. For instance a luck based game and someone gets the 50 straight wins award. Lol
User avatar
tbhp
Posts: 715
Joined: 16 April 2025, 00:35

Re: ELO on luck based games

Post by tbhp »

The idea that Elo is not good because it reflects what occurred rather than what is likely to occur seems like an anti-sportsmanship view. It's as if someone said that Greece's victory in the 2004 Euro was not valid because, well, they were not among the favorites to win it. They won the games to get there, so they earned their victory no matter what anyone may say.

The presence of luck in a board game doesn't change that fact. Luck is part of the game, so the wins you get through luck are just as real as the wins you'd get in a zero luck based game. You just need to accept that skill is not the only thing that determines the outcome of those games, fate plays a role too.

Even in real sports where players are in control of their actions, like football, we like to say that "you always need a little bit of luck to win". The idea that there is some higher entity somewhere that "picks" its champions has always been a part of sports culture, and especially in uncertain ones like football. It's part of the whole mythical aspect of it.
FrankJones
Posts: 2441
Joined: 30 June 2024, 00:24

Re: ELO on luck based games

Post by FrankJones »

Retired Actuary wrote: 28 March 2026, 18:38
FrankJones wrote: 28 March 2026, 18:18 The different Elo spreads in different games, to me, serve a function of illustrating the luck component of the game. Hearts and Spades have low Elo ceilings, indicating a high level of luck. Zero-luck games tend to have high Elo Ceilings, sometimes approaching 1000 or even 1100. To me, that has value. It helps me decide which games I might be more inclined to enjoy playing. I'm not sure it's a problem that some games have lower Elo ceilings.
I've posted this elsewhere here, but I was curious a while back at what the average ELO at a point in time was for the top 10 players of various games. I'd caution anyone seeing this to think of the numbers more broadly as neighborhoods than point estimates, but I think it does capture what you're talking about in terms of how much luck can be diversified away by skilled players.

Image
Other notable games:

Race for the galaxy has numbers 2-10 rated 594-675, btu #1 is 848. !?? How does someone get that bog of a lead in a game with completely random card draws ??

Azul: Top 10 are rated 863-1152 ( !!)

Splendor: 796-928 (amaazingly, the same player is number 1 in Azul and splendor, two very different games)

Patchwork 823-1003

What this shows me is that those 3 games have a low luck factor and high skill factor. (Patchwork has close to zero luck. All information is open and visible. The only luck factor is that sometimes whoever goes first might have an advantage or disadvantage.

Carcassonne: 751-861.

Caylus (I have not played, but the game is listed as luck 0 skill 5, if I recall correctly) : 622-925

Terra Mystica (A complex thematic strategy game for which I expected higher Elos; I thought it was a zero luck game): 671-743

Viticulture 519-675 (This squares with my experience. The game is heavily dependent on random card draws.)

Here is an interesting one: Kulami (A zero luck game, no hidden information, pure abstract strategy along the lines of Go, chess, etc.): 324-509.

A couple possible reasons: I think ties are common, and there is no tiebreaker. ALso, not a large player pool.

Pente: (A personal favorite of mine, I'm a world class player): ZERO luck, 100% skill: Top 10 Elo range from 479 - 705 .

Why? Just not a large enough sample size. I recently played a game against someone who was almost certainly a world class player. But his Elo was "only" 496, or something like that. He's as good or better than I am. Probably better. It was my first and only loss. SO, what happened? Did Elo fail? This is a pure skill zero luck game, and according to Elo, I was "supposed" to win, and I lost a decent amount of Elo for losing.

So, even with zero-luck games, Elo doesn't always work as intended.
Ceaseless
Posts: 1298
Joined: 12 November 2022, 17:06

Re: ELO on luck based games

Post by Ceaseless »

FrankJones wrote: 29 March 2026, 03:03 Here is an interesting one: Kulami (A zero luck game, no hidden information, pure abstract strategy along the lines of Go, chess, etc.): 324-509.

A couple possible reasons: I think ties are common, and there is no tiebreaker. ALso, not a large player pool.
Players not even at 100 rating are in the top 100. Safe guess why the top 10 are as low as they are. Where are the top 10 going to get the rating to move up? Hard enough to find games, and the wins are going to be pretty minimal gains on top of that.
FrankJones wrote: 29 March 2026, 03:03 Pente: (A personal favorite of mine, I'm a world class player): ZERO luck, 100% skill: Top 10 Elo range from 479 - 705 .

Why? Just not a large enough sample size. I recently played a game against someone who was almost certainly a world class player. But his Elo was "only" 496, or something like that. He's as good or better than I am. Probably better. It was my first and only loss. SO, what happened? Did Elo fail? This is a pure skill zero luck game, and according to Elo, I was "supposed" to win, and I lost a decent amount of Elo for losing.

So, even with zero-luck games, Elo doesn't always work as intended.
Looking at the top 100 and seeing the many 100s players there shows that they have little chance to face players to get real rating off of. Even with 100% wins, they're moving at a crawl as a result.
User avatar
dschingis27
Posts: 744
Joined: 27 June 2015, 18:30

Re: ELO on luck based games

Post by dschingis27 »

FrankJones wrote: 28 March 2026, 18:18 I'm not sure a lower K-factor will help. To me, 20 is already a low k-factor. Anyway, a lower k-factor would create a tighter bunching among players who perhaps should not be tightly bunched.
A lower K factor would help for sure and it would not create a tighter bunching among players. The mathematics and simulations are pretty clear. Here is a simulation I made a while ago: https://boardgamearena.com/forum/viewto ... 38#p222238
Note that the K factor does not influence the calculated expected win probabilities itself, no matter the K factor, the Elo difference between players has the same meaning. The K factor only influences how strong a single game result changes the Elo score of one player. If single games influence the current Elo score less, then more games over longer time determine the current Elo score more equally. That doesn't mean that the Elo scores between different players cannot spread out over time, it just takes longer for them to spread.
FrankJones wrote: 28 March 2026, 18:18 Suppose we decide to use a lower or higher k-factor for games depending on the amount of luck involved. Who would make these determinations, and how?
The dev / publisher or someone at BGA has to decide. It should just be a simple one of the many decisions they have to make when producing a new game on BGA. I suggested a system with only 3 categories to make the decision easy, choose between K=5, K=10, K=20. If we wanted to get an optimal precise K factor for each game, that cannot work, I agree. But having three categories would already be so much better than the current system. Currently, a single game of Terra Mystica counts the same as a single game of Can't Stop which is absolute bonkers.
FrankJones wrote: 28 March 2026, 18:18 Sure, sometimes a player whose true skill Elo is 300 might have snuck up to 400 on a lucky streak, and might get matched against a player whose true skill Elo is 500 but has dropped to 400 on an unlucky streak. Isn't it true this can happen in any Elo system?
This would happen much less with lower K factor, however, yes it is inherent in Elo systems so we might want to use a different system.
User avatar
dschingis27
Posts: 744
Joined: 27 June 2015, 18:30

Re: ELO on luck based games

Post by dschingis27 »

Here are some good suggestions on how the system could be different from ages ago. https://boardgamearena.com/bug?id=14455
Post Reply

Return to “Discussions”