ELO on luck based games

Discussions about BGA (all languages)
Forum rules
Warning: challenging a moderation in Forum = 10 days ban
More info & details about how to challenge a moderation: viewtopic.php?p=119756
FrankJones
Posts: 2441
Joined: 30 June 2024, 00:24

Re: ELO on luck based games

Post by FrankJones »

I understand.

Any argument I make is "Faulty" or "misleading", and any argument you make it iron-clad and indisputable. Is that how this works?

So, the Patriots going 18-1 compared to a 14-6 Giants team, but winning the trophy because they won a 1-game playoff - that somehow proves YOUR point and not mine? I thought your argument is that the champion should be determined by whoever played best over a larger sample of games.

Maybe the reason most players don't visit the forums is most player do no care enough about any BGA issues, including this Elo "issue", enough to bother visiting the forums to post or complain about it.

You have done a very good job of explaining why you care about this issue and how it affects you. You have not done a good job of explaining why everyone else should care, or why a system-wide change should be made.

This thread, in my estimation, blurs the line between fact and opinion. People (myself included) are throwing around words like "inarguable", "obviously", etc.

I'll refer to a different game, one I don't think I've mentioned yet. Aquatica. I believe there is an extreme balance issue in this game. It adversely affects my desire to play this game. It is a fact that in my games, I have demonstrated a tremendous advantage for player 1. I can point to factual, observable evidence and statistics. I could even list the players I have discussed this with, all of whom agree with me.

Nevertheless, few people care about it to post in the forums or demand a change, even though a change would be very easy to make. And, my wanting this to be addressed and fixed, and my belief that the game is broken without this fix? That's my subjective opinion. I need to accept that other people do not care enough to make a major fuss about it.

That's the way it goes sometimes. We see something we perceive as a problem, but we care more than other people do, and there is not enough momentum to implement change.

So, we do the best we can under the circumstances. People aren't going to care about our issues as much as we do ourselves.
FrankJones
Posts: 2441
Joined: 30 June 2024, 00:24

Re: ELO on luck based games

Post by FrankJones »

Also, let's not forget - BGA is a company. Companies want to make money. Maybe BGA's decision-makers believe the best way to do this is to appeal to the broadest audience, not the small number of players at the very high or very low end of things. Perhaps they believe that fixing every real or perceived problem for top players would turn off the vast majority of casual players.

Personally, I like the fact that high-luck games have different-looking Elo curves and ranges than low-skill games. Terms such as "strong", "expert", "master", are arbitrary. To me, mastering aa game means being able to win at a very high rate. If a game is mostly luck, it is going to be difficult to win more than 65% of the time, and thus, in my opinion, difficult or impossible to "master", again keeping in mind this is a subjective term.

There are many games I enjoy but do not play in the Arena for various reasons. I'm okay with that. I accept it. I do not expect BGA to cater to my every need or desire. So, I play those games outside the arena, but rated with Elo.

There are some games I enjoy playing, but for one reason or another, do not enjoy playing rated for Elo. So, I play friendly mode.

The main game I play in the arena has some swingy fluctuations, and I'm okay with that. Maybe a lot of players enjoy it. Maybe it's fun and exciting toward the end of the arena season.

The "high" (again, this is subjective) k-factor of 20 for non-arena games is not a problem for me, even in games involving luck. It means players can improve and learn, and have fun seeing relatively quick results from their studying and improvement. Maybe that is more desirable and fu than being bogged down by hundreds of results from previous games before they knew how to play the game well.

That's one nice thing about BGA - we have lots of features and game options. Arena, non arena but rated, friendly mode. Skill games, luck games, and a wide array somewhere in the middle.

I can also easily look at someone's profile and see the historical graph and get a sense of that player's rolling average and also Elo peak. Anyone can do that same thing. Each person can then decide what to place more value on: A player's Arena ranking, a player's current Elo, a player's peak Elo, or a player's rolling average.

Anyone who has ever achieved Elo #1 gets that permanent trophy/achievement. Maybe you want fewer players to have that. But maybe that's not what BGA wants, or not what is most profitable for BGA.

I do think more leaderboards would be a good idea (I've suggested specific ideas several times). Another one could be, "Number of days at world Elo #1."
User avatar
dschingis27
Posts: 744
Joined: 27 June 2015, 18:30

Re: ELO on luck based games

Post by dschingis27 »

FrankJones wrote: 07 April 2026, 11:19
Frank, have a break. Your posts on page 10 and 11 of this thread have been pretty wild to be honest. You delve soo much into side topics and sometimes misrepresent points. It has become very tiring and annoying to discuss with you. I give you 3 examples what I mean.

(1) The "circular argument"
FrankJones wrote: 06 April 2026, 23:03
ChiefPointThief wrote: 06 April 2026, 20:02 I think you are still missing his point. The gap is cut in half after only 15 games in aes and 30 games regular not 100. Which is literally nothing for luck based games. To give an example I recently had a bad game stretch in captain flip losing 7 of 10. Going from 408 to 333 which is 17th to 144th all time. The difference in chess is that you aren't going to lose x amount of games based off of luck. I've seen games in which my opponent made suboptimal moves all game and still won :lol: this isn't possible in chess.
And I will continue to point out the obvious circular argument being made:

You are saying Elo in these games is severely flawed. Yet, you are utterly convinced that a player with a 408 Elo should actually have that Elo, an assumption which you then use to prove your argument, when in fact, one of your primary premises is contradicted by your conclusion.
This seems to be your main gripe in the discussion about changing the K factor. But you misrepresented or misunderstood the point here. The core of the complaint is not that the original high Elo must be correct and kept for all times. "you are utterly convinced that a player with a 408 Elo should actually have that Elo" That is not what is written above in ChiefPointThiefs post, it's something you made up yourself. Instead, the main point is that people notice if Elo moves so rapidly then it can never be correct at all in high-luck games. People being people, they notice the speed of Elo changes more strongly when their Elo goes down rather than up. If you have a closer look at the complaint posts, it's mostly not that people assume their highest ever Elo is their correct Elo, it's rather people say, "WTF is that?" To give you an intuitive analogy: Imagine you have a weighing scale (a kitchen scale) and some mass laying on it, and the scale displays some weight value. Whenever you add a thing that weighs roughly 100g (or 80g to 120g), the display changes by 300g, whenever you remove a thing of roughly 100g from the mass, the display changes by 300g, too. Clearly we observe that the weighing scale is broken. We don't even need to know what the true value of the original mass is or even what the exact true masses of the smaller things are. Just the fact that the weighing scale display changes so quickly is enough to conclude the weighing scale is broken. Now exchange "weight" with "Elo" and "weighing scale" with "Elo rating system" and you have the argument being made. When a measurement value changes much quicker than the underlying construct to be measured is changing, then the measurement value must be off. And that would be mitigated by changing the K factor.

FrankJones, by concentrating so much on the idea that people want their high Elos preserved, you are ignoring the strong version of the argument. The weak version is, "High Elos should always stay high", which is stupid, yes, I agree. But we should ignore this rather than making it the center point of the discusssion.

(2) The "why 2 players's Elos should be preserved" argument
FrankJones wrote: 05 April 2026, 16:20 You are simultaneously saying "Elo here at BGA is inaccurate" and "It's very important to preserve the Elo of any 2 players at one snapshot in time."
That "2 players preserve their Elo" thing was only an absolute side aspect of the simulation I made. In the simulation, I fixed the true Elos of both players at 300 and I wanted to measure accuracy. Clearly yes, as a trivial consequence of those fixed true Elos, it is better (= more accurate) when Elos are preserved. I didn't say the (possibly inaccurate by BGA) starting Elos need to be preserved. I thought it should be obvious that I care about the true Elos, not the starting Elos, but you make it sound that starting values fixed at true Elos is a problem in my reasoning because in my reasoning, Elos on BGA are inaccurate. The point is I could have easily set inaccurate starting Elos in my simulation and it wouldn't really change the outcome of my simulation. The fact that I need to explain this is annoying. The fact that the starting values of both players was the same was not very important. It was only the same because I wanted to create equal conditions, the 2 players are equal in win probability, true Elo, and starting Elo, yet their calculated Elo values deviate strongly many times. That is literally the definition of mismeasurement, two exact same conditions lead to widely different measurement values, and the degree of mismeasurement is stronger with higher K factors. You have not engaged with that point, you haven't really engaged with my simulation.

Instead you make statements like, "We need to keep in mind that "fluctuate too much" is in itself a vague, subjective phrase. What constitutes "too much"?" Man, "fluctuate too much" is not vague in the context of our discussion. I relatively precisely showed in the simulation what level of fluctuation amounts to what level of inaccuracy. It was clearly shown that when true Elo does not change in a small series of games (which is approximately a very realistic and usual scenario), then calculated Elo is very inaccurate a lot of times. Now if you wanted to say that people have different tastes in preferred accuracy levels, that's a very bold statement. Why would people want less accuracy, I don't get it? The job of Elo is to measure skill, that is it's definition. If you personally want Elo for other purposes, that is fine but then you should spell out what these are and you shouldn't assume that other poeple do not care if Elo doesn't represent its label. People hate wrong labels.

(3) The "100 equal game results should lead to equal Elo" argument
dschingis27 wrote: 05 April 2026, 07:21 100 games are way to few to make a definitive statement about skill or Elo for that matter. Every competitive Backgammon or Can't Stop player can tell you that stretches of 100 games are usually not representative, both mathematics and simulations show that there is a high variance in win rate in only 100 games. If player A had 200 wins out of 300 games before and player B had 100 wins out of 300 games before, and then they have equal performance in 100 games, every sane person should assume that player A is much better than player B.
You bring this up again and again saying that 100 equal results should lead to equal Elo. I repeatedly said that this is not the main point anyway and if you look at the above quote, it was just an illustration of the actual real point I made in the sentences before. My point was that larger historical series of games are more accurate than smaller recent series. The fact you can refute my illustration doesn't refute my point, you haven't refuted math and simulations. (That's what I mean when I say you are so much into side topics.) You haven't engaged with my point. Instead you create hypotheticals and details in the illustrative example so that it doesn't work anymore. Yes, of course, if player A had weaker opponents at first or player B learned a lot, that explains the scenario. That is obviously not how I wanted the example to be understood. I could also add "what ifs" to make it obviously right, like what if player B had much more luck than player A in the 100 recent games? Shouldn't we consider this possibilty?, to use a wording you used yourself. We can go endlessly through different hypotheticals. But I need good-faith readers to not fill in details in my examples that don't have a relation to the point I was making. You found one situation where my example doesn't work but that doesn't add much to the general discussion. I know you wanted to refute my indeed dramatic statement that "every sane person should think A is better than B" and you did that. But that deflects so much from the point about Elo and K factors that we were discussing.

To give you another example: I could say, "There is a lot of animal life in the pond." and you could respond by saying, "But not when the whole pond is completely frozen." That means you made a correct statement - but no one really cares because the main statement wasn't about that, it was about another greater thing, the observable lifeliness of the pond, it wasn't directed at the frozen pond. You don't refute biology saying there is a lot of life in the pond by saying "but not always", in the same way you don't refute a general statistical point about Elo by saying, "but not always".

Extra:
You write so many stories from low-luck games that also don't add anything. The complaints about Elo are not targeted at these games like Earth.
Last edited by dschingis27 on 11 April 2026, 10:16, edited 1 time in total.
User avatar
dschingis27
Posts: 744
Joined: 27 June 2015, 18:30

Re: ELO on luck based games

Post by dschingis27 »

I have one major question to you. You said so many times that you are "fine" when Elo fluctuates much. For instance, you said:

In general, I don't consider it an issue that Elo here at BGA jumps around.
Or that: The "high" (again, this is subjective) k-factor of 20 for non-arena games is not a problem for me, even in games involving luck.

You said several times in several ways that you are fine. But you have not really explained well why? You mostly re-iterated your opinion statement. But why do you prefer that Elo fluctuates so much? (If you just wouldn't care, you shouldn't mind a change in K factor anyway, but you seem to really prefer the higher fluctuation in high-luck games.)

I am raising this because you say yourself you would look at rolling averages of Elo or historical graphs to get a "more realistic" (your words) picture. You mentioned few times that you also think that larger samples are needed for high-randomness games. I can't bring that together with your whole stick that Elo on BGA is "fine". So what concretely do you get from the less realistc picture that is provided by current Elo? I think most people just don't want to do this extra effort to get to their realistic picture. What is the purpose of having a less accurate Elo value?
FrankJones wrote: 09 April 2026, 16:56 It means players can improve and learn, and have fun seeing relatively quick results from their studying and improvement.
If your main point really is that skill improvements in high-randomness games are observable quickly, that is just absurd. Skill levels in high-luck games can't be seen quick, no matter what. The element of luck is just soo much stronger in those games that you could see skill improvements quickly measured accurately in a rating system. And this is not a subjective statement.
Last edited by dschingis27 on 11 April 2026, 10:11, edited 1 time in total.
User avatar
dschingis27
Posts: 744
Joined: 27 June 2015, 18:30

Re: ELO on luck based games

Post by dschingis27 »

I want to give a tl;dr of my main point again before I probably leave this thread.

Why should we care about K factors in high-luck games?
(1) The K factor modulates how strong a single game changes a player's Elo. (Higher K leads to greater change, lower K to smaller change.)
(2) Because of this, if the K factor is lower, longer series of games are important for your current Elo.
(3) In high-luck games, longer series of games are needed to measure skill accurately.
(4) Hence we should have a lower K factor in high-luck games if we want to measure skill accurately by Elo.

I don't think that any of the above statements are complicated, nor are they controversial.

In low-luck games like chess, a single game result is a big piece of information for your skill assessment, because mostly your skill determines the game result. Hence small series of games are enough for decent skill assessment in low-luck games.

In high-luck games like Lucky Numbers, a single game result is a very small piece of information for your skill assessment, because mostly your luck determines the game result. Hence only long series of games in aggregation can lead to a decent skill assessment in high-luck games.

When you agree with the above and you want an accurate skill measurement by Elo, you should be heavily in favor of having different K factors for different games on BGA.
FrankJones
Posts: 2441
Joined: 30 June 2024, 00:24

Re: ELO on luck based games

Post by FrankJones »

dschingis27 wrote: 11 April 2026, 09:33
Your posts on page 10 and 11 of this thread have been pretty wild to be honest. You delve soo much into side topics and sometimes misrepresent points.
I do not agree.
dschingis27 wrote: 11 April 2026, 09:33 It has become very tiring and annoying to discuss with you.

Your continued replies to me are voluntary. Nobody is forcing you to do so. Why are you continuing to engage in an activity you find tiring and annoying?
FrankJones
Posts: 2441
Joined: 30 June 2024, 00:24

Re: ELO on luck based games

Post by FrankJones »

I have stated very clearly that I would like some changes to be made. I have state clearly what those changes are. I have stated clearly why I feel that way. I do not see any need to repeat myself. You recently apologized for your aggressive tone in one of your posts. Now you seem to be posting in an identical manner to the post(s) in which you perceived yourself to be aggressive.

The probability of BGA making sweeping changes to the Elo system is likely close to 0.0%. So, this lengthy discussion serves little practical purpose; it is just an exercise in debate.

Whereas, implementing some of the suggestions I made would merely be a matter of displaying information that already exists, and may have a chance of being implemented.
Last edited by FrankJones on 11 April 2026, 19:54, edited 1 time in total.
Ceaseless
Posts: 1298
Joined: 12 November 2022, 17:06

Re: ELO on luck based games

Post by Ceaseless »

How would you determine the k-factor for different games? I see the rough idea of wanting a lower k factor for higher luck games, but how do you determine luck and work out what an appropriate k factor is?
Last edited by Ceaseless on 11 April 2026, 20:44, edited 1 time in total.
FrankJones
Posts: 2441
Joined: 30 June 2024, 00:24

Re: ELO on luck based games

Post by FrankJones »

Ceaseless wrote: 11 April 2026, 19:23 How would you determine the k-factor for different games? I see the rough idea of wanting a lower k factor for lower luck games, but how do you determine luck and work out what an appropriate k factor is?
Just for clarity - they want a lower k-factor for higher luck games.

I'm not sure how that would work either - trial and error, input from the game designer and high-volume high-skill players would probably be the best way.

Has anyone posted this as a suggestion in the BGA general bug/suggestion forum? I'm not convinced it would get a lot of support, but I could be wrong.
FrankJones
Posts: 2441
Joined: 30 June 2024, 00:24

Re: ELO on luck based games

Post by FrankJones »

dschingis27 wrote: 11 April 2026, 09:33
FrankJones wrote: 07 April 2026, 11:19
Frank, have a break. Your posts on page 10 and 11 of this thread have been pretty wild to be honest. You delve soo much into side topics and sometimes misrepresent points. It has become very tiring and annoying to discuss with you. I give you 3 examples what I mean.

[Truncated]

You write so many stories from low-luck games that also don't add anything. The complaints about Elo are not targeted at these games like Earth.
I'm not sure I understand what your end-game is here. Why do you feel compelled to continue the discussion with me? Why does it matter whether I agree with you? There are many things I would change about this site if I were in charge, but I don't expect everyone to agree with me.

I feel as though you're telling me my opinion is "wrong", which is an odd approach to take. Yes, I understand your opinion is that the Elo system is flawed and needs to be changed. I understand that you and I do not agree on that. I'm okay with your having a different opinion than mine.

I'm not the person you need to persuade. So, why bother? The people to persuade are the BGA site admins. And the best way to convince them would be to show them suggestion reports in a large number of bug/suggestion forums for specific games. Suggestion reports that have a lot of upvotes.

This may vary by developer, but for one very popular game here, I know the developer pays more attention to suggestions that receive a lot of upvotes.

Just to give another example: There is a game here that I like. I won't say which game, because I don't want to be criticized again for bringing up a game you don't think is relevant to the conversation. But, this particular game has what I believe (my opinion, supported by a lot of evidence ) to be a major flaw that would be very easy to fix. So, what did I do? I went directly to the game designer and asked him to consider my idea, because his opinion is the only one that really matters. He is the one who can make the change.

As for your closing sentence: "Earth" has a luck component, so, it is relevant. If we are going to change the k-factor for games that have luck, Earth would need a different k-factor. Any game in which it is possible to lose to a lower rated player purely because of bad luck is a relevant game to this discussion. And, Earth definitely meets that criteria. So does Terraforming Mars. And Wingspan.

In general, yes, I think it would be nice if I had a master-level status for games I believe I have mastered, but I'm okay with that not being possible within the current framework. I understand that BGA has goals, and those goals might not be compatible with changes and modifications I personally feel would be good ideas.
Post Reply

Return to “Discussions”