BGA has a serious bully problem

[MarkSteere]

I haven't seen anything to indicate that anyone in this thread "has no idea what they're talking about".

If you understand Elo yourself well enough to assert that others in this thread have no clue, you should be able to explain why in your own words rather than copying and pasting from chatGPT.

For the record, I have seen chatGPT wrong on matters of strict fact.

Anyone who thinks the Elo system works as well for luck based games as it does for the pure skill based games it was designed for has no idea what they're talking about.

Whatever factual matters ChatGPT may be wrong about, this isn't one of them.

After the statements you've made about Elo, are you now asserting with a straight face that you have a clue?

In my own words, there's clearly a flaw in the Elo system as relates to luck based games. Elo makes an assumption about how skill translates into outcomes, and that assumption breaks for luck based games. Luck flattens and caps the probability distribution.

Anything else?

[tbhp]

ChatGPT...

Elo does not perform as well for luck-based games because it was designed around a skill-dominant outcome model.

What ChatGPT might be trying to say is that elo doesn't perform as well if your goal is to use it as a tool to infer the winning probabilities of future games.

That would be correct. It's obvious that if a game involves more luck, then predictions will be harder to make, no matter what system you use. Any ranking system will be less meaningful because it will be relatively more tied to the luck of recent results, as opposed to skill.

What I don't follow is the idea that elo is not correct at representing the reality of what has happened. If it says someone has a 90% chance of winning against a certain opponent, this is an accurate representation of how the past results went for both players. It doesn't mean we can know if this streak will continue. Maybe the higher rated player will unexpectedly choke their next game. Maybe the game is luck based and they just happened to be lucky until now. Elo can't know those things, it only sees results, it doesn't see the games themselves. It can't tell the difference between someone who reaches a certain elo thanks to luck, and someone who reaches it thanks to skill.

None of that means elo is bad for luck based games, it just means that before being a prediction tool, elo reflects first and foremost a player's results, which is its main function anyway. When you see that Magnus Carlsen reached the historic elo of 2882 in chess, you don't care to know if he was "likely" to win the game that brought him to that elo. All you care about is the reality that he won that game, and all the ones that came before. A player's ranking conveys their achievements. Whether they were "lucky" to earn those is a secondary discussion.

[BarnardsStar]

Here’s what our new overlord told me. It actually gave about a page of text but here are choices excerpts including its bottom-line summary:

The system uses probability to predict match outcomes, which inherently accommodates luck … In luck-influenced games, while individual matches may be swayed by chance, the ELO system relies on cumulative results across many games. Over time, player skill becomes more pronounced, and the effects of luck dilute.

Conclusion: The ELO rating system remains effective in games where luck is a factor due to its adaptive mechanics in evaluating skill levels over various matches, its probabilistic approach to outcomes, and its ability to accumulate results over time. This allows it to account for the inherent unpredictability of luck, making it a robust tool for ranking players fairly.

Tom Lehrer once said, “Life is like a sewer: what you get out of it depends on what you put into it.” I’d say the same thing about LLMs.

[MarkSteere]

Here’s what our new overlord told me. It actually gave about a page of text but here are choices excerpts including its bottom-line summary:

The system uses probability to predict match outcomes, which inherently accommodates luck … In luck-influenced games, while individual matches may be swayed by chance, the ELO system relies on cumulative results across many games. Over time, player skill becomes more pronounced, and the effects of luck dilute.

Conclusion: The ELO rating system remains effective in games where luck is a factor due to its adaptive mechanics in evaluating skill levels over various matches, its probabilistic approach to outcomes, and its ability to accumulate results over time. This allows it to account for the inherent unpredictability of luck, making it a robust tool for ranking players fairly.

Tom Lehrer once said, “Life is like a sewer: what you get out of it depends on what you put into it.” I’d say the same thing about LLMs.

"Effective" isn't the first word that comes to mind to describe Elo for luck based games. I was playing a lot of Backgammon a while back. After a long winning streak my Elo would go way up hundreds of points, and similarly for the other direction. I'd say it "kinda works ok much of the time" as a very crude measure for luck based games. Like your chance of ever beating anyone with a way higher rating is slim. It would require a long series of very opportune rolls.

...or one extremely lucky roll at just the right moment.

[tbhp]

Suppose I have an elo 400 points higher than someone, and it is a luck-based game where I can't win more than 75% of the time against a competent opponent. In that case, if I care about my elo, I should be a jerk and refuse to play someone who is rated substantially lower than me.

This is a false premise, because if the game is truly that luck based, you wouldn't be able to get to elo 400 in the first place. The only way you may is if you had an exceptionally lucky streak. In which case your elo wouldn't be representative of your skill.

In luck-based games, it is very hard to keep an elo higher than 300. The reason is a flaw in elo, and not that "nobody is genuinely rated that high".

It is exactly the reason. While the elo 300 players may be the best in the world at that game, it would be true that they don't have a mastery of the game comparable to the mastery of someone who is elo 700 in another game, for the simple reason that it's a game where it's impossible to master things to such a degree. Some games just don't offer you the same level of control over their outcome.

What happens in luck-based games is that your elo will bounce around from 200 to 400, based on whether you are on a winning streak or losing streak. Did I really become 100 points of elo worse just because I went on a luck-based losing streak?

Yeah, because as in all sports, your ranking is simply the sum of all the games you have played. If you recently lost a bunch of games, it's only fair that this would make you go down in the rankings. Your mistake is to think that your ranking should be reflective of your "skill", rather than what actually happened.

[FrankJones]

I haven't seen anything to indicate that anyone in this thread "has no idea what they're talking about".

If you understand Elo yourself well enough to assert that others in this thread have no clue, you should be able to explain why in your own words rather than copying and pasting from chatGPT.

For the record, I have seen chatGPT wrong on matters of strict fact.

Anyone who thinks the Elo system works as well for luck based games as it does for the pure skill based games it was designed for has no idea what they're talking about.

Whatever factual matters ChatGPT may be wrong about, this isn't one of them.

After the statements you've made about Elo, are you now asserting with a straight face that you have a clue?

In my own words, there's clearly a flaw in the Elo system as relates to luck based games. Elo makes an assumption about how skill translates into outcomes, and that assumption breaks for luck based games. Luck flattens and caps the probability distribution.

Anything else?

Yes, I "have a clue" about how Elo works. I'm not entirely sure you do though (Maybe you do, maybe you don't, but it's unclear based on your posts), but I have no desire to lower myself to your level of insults and unpleasantness.

Maybe we should start by defining terms. What exactly is a "luck-based game"?

The Elo system here on BGA works reasonably well. Is it perfect? No. Nor does it need to be. Generally speaking, the better players have higher Elo.

Do you have any evidence to the contrary, or do you just wish to continue spouting insults and copy and paste from chatGPT and proclaim yourself an authority on the matter?

Either way, I think we can all be polite and civil to one another in this thread.

[FrankJones]

I don't understand MarkSteele's statements. Maybe someone less hostile can translate for me and the others in this forum expressing similar thoughts to mine?

If a player has a 350 Elo and then loses a few games, that player's Elo might go down to 300. That means the player's Elo, at one of those two points, was off by 25 or more points from representing that player's true skill level.

Thus, it makes little sense to proclaim that An Elo difference should predict the winning percentage when 2 players play against each other. Because, we don't know their actual true skill level.

The argument being made is entirely circular, because the conclusion itself negates the premises.

We can make reasonable approximations, and general statements, but attempting to do anything further would be error-prone. These facts do not mean Elo is flawed or broken, or that it does not work in luck-based games (whatever that means; nobody has defined that term yet).

Can anyone show me evidence of a game in which a player consistently maintains a 550 Elo but cannot consistently beat 150-Elo opponents?

[tbhp]

"Effective" isn't the first word that comes to mind to describe Elo for luck based games. I was playing a lot of Backgammon a while back. After a long winning streak my Elo would go way up hundreds of points, and similarly for the other direction.

What is strange is that you seem to be assigning the blame on the elo system, rather than on the game itself.

If a game is very luck based, then nothing about its results will be very certain, so no ranking system will ever be able to give players a stable position.

It's like shooting the messenger. You don't like the way a game's results are distributed, but instead you choose to criticize the system that is simply relating to you those results.

[FrankJones]

"Effective" isn't the first word that comes to mind to describe Elo for luck based games. I was playing a lot of Backgammon a while back. After a long winning streak my Elo would go way up hundreds of points, and similarly for the other direction.

What is strange is that you seem to be assigning the blame on the Elo system, rather than on the game itself.

If a game is very luck based, then nothing about its results will be very certain, so no ranking system will ever be able to give players a stable position.

Correct. Nor would it be able to predict results for a single game or small sample.

In the long run, higher skill players (even in "luck-based games", as long as some skill is also involved) will outperform lower skill players, and this will show in their respective Elos.

Even in games of athleticism and skill, we cannot predict what will happen in the short term.
The Oklahoma City Thunder have been outperformed in recent weeks by the LA Clippers. Does anyone want to wager on which team will have a better record when the season ends?

[MarkSteere]

Maybe someone less hostile..

I never said Elo "doesn’t work" in luck based games. What I said was that it doesn't work as well in luck based games as it does in the skill based games it was designed for. And it doesn’t. Not nearly.

It depends what you mean by "consistently." In a luck based game the much lower ranked player might win. In a skill based game they won't.

I'm not trying to be hostile. I enjoy spirited debates, as apparently do you. I thought I was just matching your level of fervor. There's no malice on my part.