ELO on luck based games

[BarnardsStar]

Well, how important are skiing and shooting for Biathlon? This is determined by the rules of Biathlon, the way the players approach it, etc. The game serves as the multi-dimensional means of measuring the combination of the various skills involved. From there people pick their methods. I don't know what it's set to for Biathlon or how accurate it is.

But it’s also determined by the very course and weather. A flat course may favor different competitors than a hilly one. Somebody might be a better shot on cloudy days and somebody else a better shot on sunny days.

Just as the (basic) rules for Terraforming Mars are the same but the different collections of cards enabled by various expansions… those are like different biathlon courses.

I remain with the idea that it is a useful approximation, as long as we bear in mind it’s just that. But it is interesting to at least consider these things.

[Ceaseless]

Well, how important are skiing and shooting for Biathlon? This is determined by the rules of Biathlon, the way the players approach it, etc. The game serves as the multi-dimensional means of measuring the combination of the various skills involved. From there people pick their methods. I don't know what it's set to for Biathlon or how accurate it is.

But it’s also determined by the very course and weather. A flat course may favor different competitors than a hilly one. Somebody might be a better shot on cloudy days and somebody else a better shot on sunny days.

Just as the (basic) rules for Terraforming Mars are the same but the different collections of cards enabled by various expansions… those are like different biathlon courses.

I remain with the idea that it is a useful approximation, as long as we bear in mind it’s just that. But it is interesting to at least consider these things.

It is. Not to mention how a player is feeling, if they're sick, tired, injured etc. There's no perfectly measuring the impact of all of that, and that chaos is just something players have to confront to the best of their ability. For all of those things it's not like the results of completely random, there are things we can reasonably discern, or at least as well as we do other things.

[ChiefPointThief]

I don't think changing the Elo calculation will make any difference — in fact, it will make things worse, because now the same Elo rating will be earned differently depending on the game.

Im confused. How would it make it worse? You do know elo already isn't equal amongst all games right? Team games and coop use an entirely different format. I know the devs even have the ability to put whatever desired formula they want for coops as one dev even asked the player base for input on creating the formula for a coop. Making an adequate rating system for luck base games wouldn't change how I view players in non luck games like terra mystica.

And actually there is already evidence of how inaccurate calculations affect rankings. There is also evidence that if you fix those inaccurate calculations it can result in better rankings. Case in point, bga changed the calculation for team games. For the record it is still VERY flawed but they made a tweak for the better. The result, prior to the change at any given time in spades their was only about 15 players ranked above 300. After the tweak now there sits 54 players and a lot in the upper 200s. The upgraded ranking system has given us a more accurate view of the skill level amongst players. Which would be the same if what I suggested for luck base games was implemented. Are you currently looking at players elos from non luck games different now that team games are being ranked differently? Ofc not and some of you probably didn't even know that fact.

Whenever you have this kind of questions, WIKI is your friend :

[See previous page for the Wiki quote]

Dude. I already know all that. I'm not sure why you think I am ignorant of what Elo was originally designed for many decades ago. There are many things in life that were designed with one intent, a long time, and are later used in different ways.

SO, yes, Elo was designed a long time ago for a specific purpose, primarily for chess. That does not necessarily mean that is the one and only intended use or meaning of Elo here. Elo here at BGA serves several purposes and functions. Nor can we know exactly what BGA's intentions or desires are. Even if the BGA page has a copied and pasted definition of what Elo is (or was), that might not match exactly what they want from Elo on this site. There are other ways in which BGA's stated policies differ in various ways from how things actually work here.

I have explained this numerous times in this thread. Weren't you the one who previously yelled at me in other threads for not reading what you said earlier in a long thread?

If you understand the idea of evolution then why are you constantly advocating to keep an inadequate ranking system in place simply because we can "get by" using it. Can we not complain about the poor ranking system and just come here and play games until we are 100yrs old? Sure. But im confused by the opposition to evolution and why that would bother some of you. I don't see consumers of apple being angry at the newest iphone because the iphone 1 worked just fine. Bga put in this ranking system years ago and even they knew it was flawed to some capacity and have already made tweaks to it. But it could still benefit from more tweaks.

Suppose there exists a game that has so much luck, even the best players will lose 35% of the time.

So, given that premise, any ratings formula that could ever have an expected win probability of higher than 65% would .... suddenly mean Elo fails, does not exist, etc.?

How does that conclusion possibly follow logically from the premises?

Maybe the conclusion to draw when there exists such a high-luck game is actually: "This game has so much luck, it is nearly impossible for players to reach [some specific desired target Elo], because mastering a game with this much luck cannot be done."

A few things you are failing to understand is that the level of "mastering" a game is different depending on the game. If you manage to understand a near coin flip game to the point that you can win close to 70% instead of 50% then you've somewhat "mastered" that game. And thus your rating shouldn't be able to fall below that of a player who only historically wins at a 50% rate based on a small lucky streak by player b or a small unlucky streak by player a after player a and b have shown over thousands of games what their skills are at the game.

Another thing you are failing to acknowledge is how fast that major predicted win probability kicks in. As I've already given examples that you overlooked only a 64elo difference already bumps a coin flip game win probability to 59% (+18%) and as someone else demonstrated at just 120elo difference it is 2 to 1. So when I reach 500 in captain flip my projected win probability over other top 15 players is close to what the win% is for top players across the entire field. These are inaccurate calculations which result in inaccurate rankings.

Unrelated to the quote above:
The elo model assumes that if A beats B 64% of the time and B beats C 64% of the time (difference 100), then A must beat C 76% of the time.
Similarly, if A beats B, and B beats C 95% of the time (difference 500), then A must beat C 99.7% of the time.
For a game where this is not true (for instance because there's always a 1 in 20 chance that you'll draw the "you win" card), elo cannot be an accurate model. We can discuss whether it's a good enough approximation, or whether the assumption is actually true for a game like chess...

Some of your post in this thread indicates that you completely understand how elo can be flawed which is interesting considering you've argued against me on this same topic in the past.

Continuing on using captain flip as an example this issue exist of losing a certain % of game just based off luck. It is a game in which you reach your hand in a bag full of tiles, pull one out at random, and place it on your board or flip it. If your first tile is one that gives you no points, or the lookout that is more useful after several turns because it must be placed on the top, etc. there's a certain % chance these things will happen and can result in a bad game that is out of the players control.

[tbhp]

A few things you are failing to understand is that the level of "mastering" a game is different depending on the game. If you manage to understand a near coin flip game to the point that you can win close to 70% instead of 50% then you've somewhat "mastered" that game.

That's not how this site defines the term "master". Here it is directly equated with win probability and Elo level, so if you are at 500 Elo in a game where the top players don't go much over 500, then you are one of the best players of that game, but you are not considered a "master", because that term is reserved for players whose win probabilities correspond to Elos over 700. Which for some games just isn't possible.

This also corresponds with how language understands those terms. Someone who manages to win a seemingly random game 70% of the time would be seen as a "magician" or a "seer", they wouldn't really be seen as a master. That term is used more for when people have a high level of control over a discipline.

And thus your rating shouldn't be able to fall below that of a player who only historically wins at a 50% rate based on a small lucky streak by player b or a small unlucky streak by player a after player a and b have shown over thousands of games what their skills are at the game.

I don't think "historically" should matter too much. People can get better or even have recent luck streaks that effectively put their recent results on par with those of the best players. Here again it comes down to whether you think Elo should measure a stable notion of skill (which is in itself a nebulous notion) or rather should be based on a "results first" perspective that acknowledges what a player recently achieved (regardless of how much luck was involved in that).

[Jellby]

Some of your post in this thread indicates that you completely understand how elo can be flawed which is interesting considering you've argued against me on this same topic in the past.

Of course, elo can be flawed, because *any* (or rather *every*) method of ranking players (or whatever elo is doing) is flawed, in the sense that it cannot do everything people may expect it to do. That doesn't mean I have to agree on what the flaws are (where someone sees a flaw others see a strength), or how to "fix" them (which often, if not always, means creating other other flaws).

[tbhp]

Some of your post in this thread indicates that you completely understand how elo can be flawed which is interesting considering you've argued against me on this same topic in the past.

Of course, elo can be flawed, because *any* (or rather *every*) method of ranking players (or whatever elo is doing) is flawed, in the sense that it cannot do everything people may expect it to do. That doesn't mean I have to agree on what the flaws are (where someone sees a flaw others see a strength), or how to "fix" them (which often, if not always, means creating other other flaws).

That's the problem with any system that expects to measure probabilities of real life phenomenons. While the math itself is accurate, the application to the real world always involves a lot of guesswork.

If we live in a deterministic universe, then probabilities don't actually exist, except in a theoretical sense. Everything either has a 0% chance of happening or a 100% chance of happening (either it will or it won't). Probabilities are just a way for us to try to predict things as best as we can, based on our limited knowledge of the world.

[Cappa]

And the award for greatest waste of space goes to … drumroll please …

Ok bud

Was it really necessary to quote 13 paragraphs to add zero meaningful content?

Ok bud.

Yes.

[FrankJones]

[ note - this is a quoted portion that was a reply to Andrei3009]

You do know elo already isn't equal amongst all games right? Team games and coop use an entirely different format. I know the devs even have the ability to put whatever desired formula they want for coops as one dev even asked the player base for input on creating the formula for a coop. Making an adequate rating system for luck base games wouldn't change how I view players in non luck games like terra mystica.

I thought Elo was the same for all 2-player games, and then uses a modified formular for 3-player games, but the same modification for all 3-player games. For a team game that is 2 versus 2, I thought the 2-player standard formula was used after taking the average Elo for the two teammates and applying the standard formula.

Co-op games can make things murky, but that's a separate matter, right?

I do wish there was more transparency and that all players could easily view the exact Elo formula for any game.

And actually there is already evidence of how inaccurate calculations affect rankings. There is also evidence that if you fix those inaccurate calculations it can result in better rankings. Case in point, bga changed the calculation for team games. For the record it is still VERY flawed but they made a tweak for the better. The result, prior to the change at any given time in spades their was only about 15 players ranked above 300. After the tweak now there sits 54 players and a lot in the upper 200s. The upgraded ranking system has given us a more accurate view of the skill level amongst players. Which would be the same if what I suggested for luck base games was implemented. Are you currently looking at players elos from non luck games different now that team games are being ranked differently? Ofc not and some of you probably didn't even know that fact.

My apologies for my very belated reply. This portion of your post is very interesting to me. I was not aware of this recent change by BGA for Elo in team games. I do play spades, and was ranked 7th overall at one point, when nobody was over 400. I recall having great difficulty improving my Elo despite being one of the best players on the site. Thank you for bringing this to my attention. What exactly was the change made to the Elo formula for Spades, and when was the change implemented?

If you understand the idea of evolution then why are you constantly advocating to keep an inadequate ranking system in place simply because we can "get by" using it. Can we not complain about the poor ranking system and just come here and play games until we are 100yrs old? Sure. But im confused by the opposition to evolution and why that would bother some of you. I don't see consumers of apple being angry at the newest iphone because the iphone 1 worked just fine. Bga put in this ranking system years ago and even they knew it was flawed to some capacity and have already made tweaks to it. But it could still benefit from more tweaks.

I'm not sure the iPhone analogy works here. But, I am not opposed to improving the system, as long as it is clear how and why changes are being made.


______________

Suppose there exists a game that has so much luck, even the best players will lose 35% of the time.

So, given that premise, any ratings formula that could ever have an expected win probability of higher than 65% would .... suddenly mean Elo fails, does not exist, etc.?

How does that conclusion possibly follow logically from the premises?

Maybe the conclusion to draw when there exists such a high-luck game is actually: "This game has so much luck, it is nearly impossible for players to reach [some specific desired target Elo], because mastering a game with this much luck cannot be done."

A few things you are failing to understand is that the level of "mastering" a game is different depending on the game. If you manage to understand a near coin flip game to the point that you can win close to 70% instead of 50% then you've somewhat "mastered" that game. And thus your rating shouldn't be able to fall below that of a player who only historically wins at a 50% rate based on a small lucky streak by player b or a small unlucky streak by player a after player a and b have shown over thousands of games what their skills are at the game.

I understand this. I realize that "mastering" a game means different things for different games. I just don't have a major gripe with this fact. For OFCP, mastering the game here means an Elo of 300. For Kulami, it means an Elo of 400. Earth? 700. Some abstract games? 850+. Etc.

I understand this and I am okay with it. What exactly are you suggesting? Do you want every game to have a modified Elo formula such that we have exactly x% of the player pool at 700 (master) Elo? Such that every game has the exact same Elo ceiling for the world Elo#1 player? Why is that necessary? When I want to see who is the best at a game, I do not look only at current Elo. I look at the historical graph to see peak Elo and average Elo over time.

Another thing you are failing to acknowledge is how fast that major predicted win probability kicks in. As I've already given examples that you overlooked only a 64elo difference already bumps a coin flip game win probability to 59% (+18%) and as someone else demonstrated at just 120elo difference it is 2 to 1. So when I reach 500 in captain flip my projected win probability over other top 15 players is close to what the win% is for top players across the entire field. These are inaccurate calculations which result in inaccurate rankings.

I don't see it that way. I just view that as confirmation of the fact that higher-luck games are more difficult for players to separate themselves from the pack near the top of the rankings. As would be expected for a high-luck game.

Case in point: OFCP. I recently created an open-invite game available to anyone over 100 Elo. I should not have done that. I was the 2nd or 3rd ranked player with an Elo around 300, maybe 310. My game was joined by a 133 Elo. This player proceeded to demolish me in a game in which I was so unlucky, the match did not even go the full number of hands because of the silly bankrupt scoring feature. From what I could see, the player played well above a 133 level. (In fact, the player had been 250-ish and was on a losing streak). Such is the nature of OFCP. I was annoyed, yes, and I lost 16 hard-earned Elo or something like that. But I have since made it back to 300.

I don't see the big deal. And I could have simply locked sub-200 Elo players out of my game when I created the table.

[FulkNerra]

Why is everyone yelling Elo's last name?

[nik592]

I suspect mostly because it is displayed that way everywhere in BGA. On profile pages, game pages, in the FAQ. Most people don't know what Elo actually is (I didn't, and still don't know a lot about it, as I don't really care), so go by what is used here. Which is typically ELO.