ELO on luck based games

[FrankJones]

Whenever you have this kind of questions, WIKI is your friend :

[See previous page for the Wiki quote]

Dude. I already know all that. I'm not sure why you think I am ignorant of what Elo was originally designed for many decades ago. There are many things in life that were designed with one intent, a long time, and are later used in different ways.

SO, yes, Elo was designed a long time ago for a specific purpose, primarily for chess. That does not necessarily mean that is the one and only intended use or meaning of Elo here. Elo here at BGA serves several purposes and functions. Nor can we know exactly what BGA's intentions or desires are. Even if the BGA page has a copied and pasted definition of what Elo is (or was), that might not match exactly what they want from Elo on this site. There are other ways in which BGA's stated policies differ in various ways from how things actually work here.

I have explained this numerous times in this thread. Weren't you the one who previously yelled at me in other threads for not reading what you said earlier in a long thread?

[FrankJones]

the question is : Does this statistic have the predictive value for further outcomes (as the chess-ELO seems to have) ? There are situations where the answer is no --- in these cases ELO doesn't exist.

My question was (and is) : are there any studies about other games than chess (and GO) which show that the statistic has the claimed predictive value ??

1) In games that do not have "zer0-luck" Elo might not be perfect in its predictive value. I don't see why this is a problem. Would it appease people to just call it something else? "BGAlo", or something?

2) Do we need studies? Why would we assume is flawed when used in different games of zero-luck and no hidden information? The premise of this lengthy thread is that Elo is flawed for games that have any luck component, implicitly asserting that Elo works for zero-luck games.

[Andrei3009]

I don't think changing the Elo calculation will make any difference — in fact, it will make things worse, because now the same Elo rating will be earned differently depending on the game.

Instead, I prefer to give Elo a different weight in different games. The more randomness a game has, the more I care about a player's average Elo over their recent 10-50 games.

[FrankJones]

I don't think changing the Elo calculation will make any difference — in fact, it will make things worse, because now the same Elo rating will be earned differently depending on the game.

Instead, I prefer to give Elo a different weight in different games. The more randomness a game has, the more I care about a player's average Elo over their recent 10-50 games.

Agreed, and I've been saying pretty much the same thing.

Be prepared to be shouted down by others in this thread who are going to tell you how wrong you are, and posting internet links to what Elo is and must always be.

[Andrei3009]

One more thing: despite the randomness concerns, Elo still shows overall skill quite well in any game (if game has enough popularity).

I typically evaluate skill by taking the average rating of the top 10-20 players, and that number has almost always aligned with my own impressions after a deep dive into the game.

[Fuchur]

If ELO (in the sense of a linear measure of skill for a specific game, giving predictions for the average outcome) exists, its predictive value is perfect, luck-heavy game or not.

If it exists, one might struggle to find appropriate formulas which give good approximations of ELO --- especially in luck heavy games combined with evolution of skill (if there is a lot of random influence you need a lot of game results to cancel them out; if you want to follow closely the evolution of skill of one player you shouldn't take into account "old" games). As far as i understand this thread several people believe that the formulas used by BGA are suboptimal. There were even numerical simulations which showed (under the assumption that ELO exists) that the k-factor used right now on BGA results in wide deviations between estimated ELO and the "true" ELO.

Why would we assume that ELO exist -- being able to reduce the skill of any player (of a given game) to a simple number which indicates not only who is better but gives precises prediction of the win probabilities seems a extremely strong condition (at least to me).

You could have said that you don't know of any studies -- much shorter (and i wouldn't have been forced to answer to your post).

the question is : Does this statistic have the predictive value for further outcomes (as the chess-ELO seems to have) ? There are situations where the answer is no --- in these cases ELO doesn't exist.

My question was (and is) : are there any studies about other games than chess (and GO) which show that the statistic has the claimed predictive value ??

1) In games that do not have "zer0-luck" Elo might not be perfect in its predictive value. I don't see why this is a problem. Would it appease people to just call it something else? "BGAlo", or something?

2) Do we need studies? Why would we assume is flawed when used in different games of zero-luck and no hidden information? The premise of this lengthy thread is that Elo is flawed for games that have any luck component, implicitly asserting that Elo works for zero-luck games.

[FrankJones]

So, the issue is BGA is misusing the term "Elo", and they should just call it something else. "Rating" or something similar.

[Ceaseless]

its predictive value is perfect

What is this supposed to mean?

[FrankJones]

its predictive value is perfect

What is this supposed to mean?

The "point" Fuchur is making is that if Elo on BGA does not perfectly predict with 100% accuracy the outcome of games between the players, then Elo does not in fact Exist.

It's one of the more bizarre arguments I've ever heard.

[FrankJones]

Suppose there exists a game that has so much luck, even the best players will lose 35% of the time.

So, given that premise, any ratings formula that could ever have an expected win probability of higher than 65% would .... suddenly mean Elo fails, does not exist, etc.?

How does that conclusion possibly follow logically from the premises?

Maybe the conclusion to draw when there exists such a high-luck game is actually: "This game has so much luck, it is nearly impossible for players to reach [some specific desired target Elo], because mastering a game with this much luck cannot be done."

At that point we can either:

1) Accept this limitation of the game design itself, and accept that for this game, ratings will fluctuate, and the Elo range and Elo ceiling might be lower than in high-skill games,

or

2) Complain that it's a broken system in need of repair, and demand a different formula for every game, based on the amount of luck involved.

Option 1 seems reasonable to me. None of those things bother me.

Option 2 seems potentially confusing, and to me, unnecessary.

The question of whether you choose option 1 or 2 is an individual personal subjective choice. Personally, I like the information gained from using the same Elo formula for every game, and I appreciate the avoidance of the chaos and confusion that would occur if each game had its own rating formula.

As used here on BGA, Elo serves several functions:

1) It's a record of what has occurred, with a bias toward the more recent games. So What? BGA explicitly states it wants players' ratings to change quickly to more quickly reflect players' skill level. Sometimes a player suddenly gets good at a game. That player is rewarded with a relatively quick rating increase. Good. I do not see a problem.

2) It helps players recognize high-variance, high-luck, and high-skill games by looking at the relative Elo ceiling and Elo spread. This is a good thing, in my opinion. I would not want every game to have an identical Elo spread and/or Elo ceiling.

3) It serves matchmaking purposes by use of Elo filters. Is it perfect? No. Sometimes you have a player who is better than the low-ish Elo indicates, and sometimes a player is worse than a high-ish Elo indicates. So what? It still works reasonably well. Even in high-luck games, I have observed a high correlation between a player's Elo and skill.

4) It allows for fun leaderboards with a challenge to remain atop the boards.

I see no issue with any of this.