Page 8 of 20

Re: ELO on luck based games

Posted: 30 March 2026, 03:26
by Name_Taken
FrankJones wrote: 30 March 2026, 02:52 Wel, with such an intelligent retort, obviously you're right and taking the higher ground.
No I guess taking the high ground would've involved making some huge post citing all the jerky stuff he said and I didn't want to hijack your thread with all that, also I'm not here for all this bickering, I just wanted to enjoy some board games and talking about board games. I'll see myself out.

Re: ELO on luck based games

Posted: 30 March 2026, 03:50
by BarnardsStar
Name_Taken wrote: 30 March 2026, 00:06 Are you following me around the forums for every possible opportunity to call me stupid? You show up with something rude every time I post something no matter what or where it is...
I’m still trying to work out what it is that was said before this comment that was being rude to you.

Obviously the later comment calling you a jerk was rude. *check notes* Wait, this just in, that was you being rude to someone else. Ok, so I don’t have any instances of somebody being rude to you in this thread. What are you talking about?

Re: ELO on luck based games

Posted: 30 March 2026, 13:43
by Suoivax
Name_Taken wrote: 30 March 2026, 03:26
FrankJones wrote: 30 March 2026, 02:52 Wel, with such an intelligent retort, obviously you're right and taking the higher ground.
No I guess taking the high ground would've involved making some huge post citing all the jerky stuff he said and I didn't want to hijack your thread with all that, also I'm not here for all this bickering, I just wanted to enjoy some board games and talking about board games. I'll see myself out.
just talk about board games? but then the first to start bickering?

Re: ELO on luck based games

Posted: 30 March 2026, 19:17
by FrankJones
dschingis27 wrote: 29 March 2026, 09:28
FrankJones wrote: 28 March 2026, 18:18 I'm not sure a lower K-factor will help. To me, 20 is already a low k-factor. Anyway, a lower k-factor would create a tighter bunching among players who perhaps should not be tightly bunched.
A lower K factor would help for sure and it would not create a tighter bunching among players. The mathematics and simulations are pretty clear. Here is a simulation I made a while ago: https://boardgamearena.com/forum/viewto ... 38#p222238
Note that the K factor does not influence the calculated expected win probabilities itself, no matter the K factor, the Elo difference between players has the same meaning. The K factor only influences how strong a single game result changes the Elo score of one player. If single games influence the current Elo score less, then more games over longer time determine the current Elo score more equally. That doesn't mean that the Elo scores between different players cannot spread out over time, it just takes longer for them to spread.

My apologies for the delayed reply. (Earlier in this thread I quoted the above portion and said I had more thoughts that I would post soon).

So, if I understood your simulation correctly, you had the following premises:

1) Two players have Elo ratings 100 Elo apart. (For example, 600 and 700, though the actual numbers do not matter).
2) Those two players then each play 100 games against the same opponents and have the same results.

The conclusions you then drew were:

1) With a higher k-factor, that 100-Elo-Gap disappears more quickly. (That's not so much a conclusion you drew, but rather, an inarguable result demonstrated by your simulation. I'll trust that the math is correct, especially since this result is what I would expect anyway based on how Elo works.)

2) This represents a flaw in Elo and/or the k-factor in some cases.

That second conclusion is where I strongly disagree, because I think your simulation illustrates the exact opposite point.

Let's re-examine the first two premises, especially the second premise.
If 2 players have the exact same results across 100 games against the same opponents, that is literally the definition of "two equally skilled players." Two players who achieve the exact same results against the same players should have the same Elo, and should not have an Elo gap of 100.

Why should anyone care that the "historical" Elo gap of 100 is mostly erased? It should be entirely erased. Because, one of two things is the case:

1) That 100 Elo gap was artificial in the first place. Maybe each player was actually a true 650 rating, but one had a small win streak and the other had a small losing streak, putting them at 600 and 700 instead of both at 650.

or

2) The lower rated 600-Elo player has improved and is now equal in true skill and Elo to the 700-Elo player.

Either way, one purpose of Elo and k-factors is to get players accurately rated, and as quickly possible, right? (Hance the higher temporary k-factors during a player's first 11 and 21 games played). So, the results you showed, in which higher k-factors closed this 100-Elo-gap more quickly - to me, that is not a flaw in k-factor; that is the k-factor working efficiently to correct inaccurate Elos.

Other thoughts:

I took a quick glance at lifetime Elo and Arena scores for "Earth" And "Terraforming Mars" For both games, the spread in Elo (from highest to the floor of 100) is similar to the spread in Arena points (from highest to the lowest shown, keeping in mind that the truly lowest arena scores might not be visible because the actual arena score is hidden for any player who is not yet purple elite.)

So, despite the double k factor of Arena (40) compared to standard games (20), the arena results seem to reasonably mirror the Elo results, despite the fact that Arena scores get reset every 3 months and despite the lower overall player pool.

So, I'm not seeing a problem here either.


One more thought:

I feel as though one point being made in this thread is essentially a matter of circular reasoning. As such, I reject the argument as invalid.
The argument goes like this:

[Some player] says, "I'm an expert level 500-Elo player, but because of the k-factor or Elo system or both, I am not able to maintain an expert-level 500 Elo, thus Elo or k-factor is flawed".

Isn't that an obvious circular reasoning fallacy? One of the premises is being assumed without proof and then used as part of the conclusion.

I'll use myself as an example. "Earth" is a long slow grind to 700 Elo. Few players reach that level. I want to be one of those players. I have stagnated in the high 600s, reaching as high as 674 and currently at 664.

Now, I could say, "This Elo system is unfair. When I win, I gain 3 Elo, and when I lose, I lose a lot of Elo, so much that I cannot get back the Elo I lost even if I win 7 in a row. This is making it impossible for me to become the 700-rated master player I believe myself to be."

But that is making a lot of assumptions.

What my results demonstrate is that I have not played well enough to attain a 700 Elo rating, and if I wish to be a 700-Elo player, I need to play better. I need to avoid losses to 300-Elo players, and I need to play .500 or better against other master-level players. (I had two such chances recently and lost both; though overall, I have satisfied this part of the requirement.)

One more thought. A lower k-factor could make it take longer for players who should have a wide Elo gap to actually attain that gap. Which means, there is a longer delay in getting players accurately rated, and a higher number of games adversely affected by the fact that one or both players have an inaccurate Elo. I'm not really thrilled about playing against a player who is actually truly master level but has an Elo of only 500 because the low k-factor is causing it to take forever for that player to reach 700. So, we play, and I lose, and in doing so, I lose more Elo than I
"should", so now my Elo is artificially lowered, causing other players I subsequently beat to lose more Elo than they "should", etc. I mean, it works both ways.

In either case, over a large enough sample, the results should end up where we expect. And, yes, higher-luck games have higher variance and therefore require more games to meet the criteria of "large enough sample". Some may view this as a problem; I do not.

Furthermore, one could make the case that games with more than "x" amount of luck are just not possible to "master" because the amount of luck means it's just not possible to win often enough to ever reach 700 Elo. Is that a problem? I do not see this as a problem. It is not necessary for every game to have an identical Elo ceiling. I like the fact that different games have different Elo ceilings (as shows by the colorful chart posted by the actuary), because that gives valuable information about the amount of luck involved in the game.
(I would further argue that these results give a better indicator of the skill-to-luck ratio than arbitrary values assigned by the game designer or developer). I've seen games here on BGA for which the developer assigned numerical values for luck and skill that I flatly disagree with.

Re: ELO on luck based games

Posted: 30 March 2026, 20:46
by ChiefPointThief
Strode wrote: 28 March 2026, 16:56
ChiefPointThief wrote: 28 March 2026, 15:51
Strode wrote: 19 March 2026, 21:18 Just don't worry about Elo for luck-based games.

It's less complicated that way. Plus, we'll never all agree as to which games count as luck-based and to what extent.
Don't worry about site crashes, Don't worry about bugs, Don't worry about new games being added etc etc etc. You can apply this to literally every topic.
There's far more reason to worry about site crashes and bugs than about Elo in luck-based games.

Elo was designed for a no luck game (Chess). The more luck a game has, the less meaningful Elo is, and the less you should care about it.
For you maybe. For me, I play hundreds of games and don't have severe issues with bugs. Site crashes are annoying but doesn't hurt my experience to the point in which I want to cancel my subscription. But I've seen numerous users come to the forums and express otherwise. It isn't for me to tell them what they should value in their experience. Just as it isn't for you to tell me what I should care about in my experience here. As a person who plays a lot of games w/ luck factor and am good at those games I would greatly value a better rating system for luck games than having bugs fixed that aren't affecting me. I am not alone! As there are countless threads on this topic.
dschingis27 wrote: 28 March 2026, 16:16
ChiefPointThief wrote: 28 March 2026, 15:51
dschingis27 wrote: 17 March 2026, 15:05 Elo is reflecting win probability. If there is a high luck factor, there is not much difference in win probabilities between players, hence players tend to have similar Elos.
Except the win probabilities are inaccurate for high luck games. Creating false elos.
What are you talking about? The "win probabilities are inauccurate"? Are you one of these "the site is not random" sceptics?

I can agree on the following two things:

(1) The Elos are inaccurate for high luck games because they rely so heavily on a too small number of recent games rather than a large enough sample of games to be meaningful. Hence the win probabilities calculated from those Elos are often inaccurate, that is true, maybe you meant that. But it is clear to me that it is the Elos that are inaccurate first, creating wrongly calculated win probabilities, and not the other way around like you wrote it. It doesn't start with wrong probabilities, it starts with too small sample sizes being used for Elo. The inaccurate Elos would resolve themselves if they would simply reduce the K factor by a lot.

(2) Elo is often not the best measure for skill, particularly in high luck games. In high luck games, small differences in win percentage can reflect highly different skill level, this is not reflected in Elo, that is true. Still, Elo is a model that can describe win probabilities, it's just that we shouldn't use Elo to measure our skill against each other, better things are available indeed.
Did you just call me a conspiracy theorist and then proceed to agree with me? lmao
FrankJones wrote: 28 March 2026, 16:19 Games with more luck are inherently going to have player Elos more tightly bunched, because higher luck means the better player is going to win less often than in pure-skill games.

In the short run, anyway. In the long run, with enough games played, the better players will have higher Elos than the weaker players.
We've been through this. I've been busy/got tired of the debate but I'll continue on from our conversation on this same topic from the last thread. Since then I reached #1 elo in captain flip again. Why? Because I am an elite player at the game. Then I inevitably fell again below several players I am better than. Why? Because the rating system being used is not good for high luck factor games at all. As for the long run I have over 3000 games.

Once you start reaching high 400s in this game you will eventually fall back down. Of the 100,00 players only 0.6% are "strong" players and even vs those strong players you have calculated win probabilities up to the 70s in a game with 4 luck rating(Even the top players don't have win % vs the entire field that high). Meaning if we were to face each other 100x im supposed to beat these other "strong" players over 40+ more times to maintain that rating at a game w/ such high variance. I could do this for 100s of games which would be outstanding and can have 1 bad 10 game streak and fall back into the 300s.

With adjusted win probabilities it would show a better representation of the skill between players. The elo gap would be further and it wouldn't converge players together as fast. Also w/ reduced k factor as an alternative solution (the recommended was 5. 4x lower than that of other games) trying to resolve the same issue it would take way more games to reach these levels. Thus causing issues when trying to filter for players bc there could be solid players ranked low that just didn't play hundreds of games yet. You've mentioned this before Frank in this thread or another but also it is more fun to actual see the ranking move with progress than and more relative to other games on the site rather than only a few points or even decimals at a time.

Re: ELO on luck based games

Posted: 30 March 2026, 21:20
by Ceaseless
An interesting thing to know would be the degree to which transitive rating gaps reduce the accuracy of elo. You might accurately predict how a 100 does vs a 200, a 200 vs a 300... 600 vs 700, etc. But how well do those rates hold up with a 700 vs 100? I do not think games identically scale these.

Re: ELO on luck based games

Posted: 30 March 2026, 22:05
by stillframe
Ceaseless wrote: 30 March 2026, 21:20 An interesting thing to know would be the degree to which transitive rating gaps reduce the accuracy of elo. You might accurately predict how a 100 does vs a 200, a 200 vs a 300... 600 vs 700, etc. But how well do those rates hold up with a 700 vs 100? I do not think games identically scale these.
I am pretty sure the rating math is the same for all games. Of course the 100 rating floor can distort results near that level, so 700 vs 100 is not a good example. But 700 vs 300 should show the same disparity in results no matter what game is being played.

Do you have counterexamples? I am curious.

Re: ELO on luck based games

Posted: 30 March 2026, 22:21
by FrankJones
I think once the gap becomes large enough, the win probability necessary to maintain Elo might be too high to account for luck.

To use the example of 700 versus 100. I haven't reached 700 yet, but I think I've had a few games where I was 650 and my opponent was 100. So, let's see what the standard BGA formula says:

A win for me would gain 0.8 Elo, and a loss would be minus 19.2.
So, I am expected to win 23 times out of 24 if I kept playing different players rated 100.

That's a tough one to analyze. I do think three is enough skill in the game (In this case, "Earth") that I can win 23 out of 24. I don't think I've ever lost to anyone sub-200 but I'm not sure. I do think I've had a loss here and there against a good player (meaning, 200-300).

The real problem occurs when a "new" player (Meaning, new to BGA but possibly very experienced and skilled with the game) ends up getting matched with a high Elo established player.

Some other sites protect against such encounters by only adjusting the established player's Elo by a fraction of the usual amount, that fraction being x/20, to help new players get games without having established players wanting to avoid them for fear of the above situation occurring. (The actual denominator was not necessarily 20, but rather, whatever the standard k-factor was for the site.

I wish we had that feature here, or at least, a way to filter opponents by games played the way can filter by Elo.

Re: ELO on luck based games

Posted: 30 March 2026, 22:42
by Ceaseless
stillframe wrote: 30 March 2026, 22:05
Ceaseless wrote: 30 March 2026, 21:20 An interesting thing to know would be the degree to which transitive rating gaps reduce the accuracy of elo. You might accurately predict how a 100 does vs a 200, a 200 vs a 300... 600 vs 700, etc. But how well do those rates hold up with a 700 vs 100? I do not think games identically scale these.
I am pretty sure the rating math is the same for all games. Of course the 100 rating floor can distort results near that level, so 700 vs 100 is not a good example. But 700 vs 300 should show the same disparity in results no matter what game is being played.

Do you have counterexamples? I am curious.
I don't have data all neatly grouped, just my observation from playing several different games to a decent degree and my observations of forums for even more games. It's how you get players that grind to the top by avoiding anyone closer rated (arena blockers or just repeat cancellers in non-Arena). They don't hold up well vs players at that rating caliber. It's something you see in lower variance games in particular. To use a higher tier example, look into the Hikaru Nakamura qualification issue regarding game count.

In higher variance games the top-rated players would love to dodge the much lower rated players. The reason being you'll get simply unwinnable games against anything with something resembling a brain. The decision tree in each game is very different, sometimes you get games where there aren't any real choices to make. It places a ceiling on your win percentage vs players. Not much of an issue when the win rates are closer to 50/50, but when you need a massive win rate to hold up, it's an issue. It's part of what keeps ratings lower, but I think the scope gets messed with at higher rating gaps more than lower ones.

But since I don't have the data on that, I have no idea how far this goes. I could be wrong about how much this affects. It's something I don't have enough direction information about and think it would be interesting to compare.

Re: ELO on luck based games

Posted: 30 March 2026, 22:50
by FrankJones
ChiefPointThief wrote: 30 March 2026, 20:46
FrankJones wrote: 28 March 2026, 16:19 Games with more luck are inherently going to have player Elos more tightly bunched, because higher luck means the better player is going to win less often than in pure-skill games.

In the short run, anyway. In the long run, with enough games played, the better players will have higher Elos than the weaker players.
We've been through this. I've been busy/got tired of the debate but I'll continue on from our conversation on this same topic from the last thread. Since then I reached #1 elo in captain flip again. Why? Because I am an elite player at the game. Then I inevitably fell again below several players I am better than. Why? Because the rating system being used is not good for high luck factor games at all. As for the long run I have over 3000 games.

I know we discussed this before to some extent. I wasn't trying to repeat / rehash. There were two separate threads with a lot of posts spanning a lot of calendar time, and sometimes it's hard to keep track of what was said, when, and in which thread.

I think further confusion stems from the fact that two separate discussions are occurring. One involving a lower k-factor, and the other involving using an entirely different rating formula, right? And you have been advocating for a different formula for games involving more luck. I am pretty sure I would prefer your idea of a different formula rather than just keeping the same formula but using different k-factors.

Other ideas: Leaderboards could display not only the current Elo, but also historical Elo. Meaning, a second column could be shown for "highest Elo ever achieved". That's one area where the "long term" would be more useful. You've played enough games to reach Elo 1 a few times but also to reach what looks like a 550 Elo, which I would guess might be the highest anyone has ever gotten in this game.

And then maybe a third column: "Rolling average Elo over last [100, 300, 300, whatever] games played".

Having these additional leaderboards would give more valuable information than just a leaderboard of current Elo. And it would encourage people in the top spots to play rather than just sit there and only play once in a while when they drop off the leaderboard due to inactivity.

When you reach Elo 1, do you try to avoid games against anyone below 400 Elo? Outside the arena, that's my approach. Once I hit an Elo level that can be filtered, I avoid players below that filter. (Unfortunately, there is no filter between 300 and 500.)

And that doesn't help in the Arena. (Thus, I generally avoid Arena. Of the games I play, I only play one of them in the Arena).

Another option, one I use for some games that have too many factors that I feel put my Elo at risk: I just stick to friendly mode games. I'm not sure whether that's your thing, but it might help. I mean, yes, you're right, people shouldn't tell you, "Oh, just don't worry about your Elo at this game, problem solved!", because as you noted, different people care about different things. That said, I do hope you do everything you can within the BGA framework to protect your high Elo once you attain it, such as, any of the thigs I mentioned above.

Personally, if I had reached 550 in this game, I would have never played a rated game again against anyone below 500.