ELO system is a failure

[gdBlaze]

To me, the problem is not the ELO system itself but how the individual can use it to accomplish the stated goal of BGA. BGA says to use ELO to find 'like' players as you. But this is very difficult. One reason is that the restrictions that you can put on a table are too arbitrary. For instance, if you are a player with 1850 ELO you probably don't mind playing players with 1750 ELO but if you allow good players then you also have to accept 1601 players too. This problem is even more pronounced if you're a 1650 player. Again you probably don't mind playing 1550 players but if you allow regular players then you have to accept 1200 players too.

Personally, I don't go around kicking players out of my game when this happens, it's too rude for me to do. Then when I lose 60 ELO I get very annoyed. I think many think the system is broken because of this.

I think that the fix is to build some better restrictions than just good, expert, master. To accomplish BGA's own goals, the restrictions should be plus or minus the ELO of the player that started the table. For instance, if you ELO is 1650, why not have a restriction of plus or minus 150 (so, 1500 to 1800). That would be finding 'like' players.

[asd123321]

It is clear that the ratings are not taking much consideration for the number of games you have played so it is mainly based on your last 20 games or so. The amount your ratings change should be much less as you play more. However if you lose 40 points to a lower ranking and if you beat one you went up 10 points, in 4 games you are back to the starting amount. if it was changed so you lose 4 points to a lower ranking player and if you beat one you went up 1 point, in 4 games you are still back to the starting amount.

[kaneaaa]

It is clear that the ratings are not taking much consideration for the number of games you have played so it is mainly based on your last 20 games or so. The amount your ratings change should be much less as you play more. However if you lose 40 points to a lower ranking and if you beat one you went up 10 points, in 4 games you are back to the starting amount. if it was changed so you lose 4 points to a lower ranking player and if you beat one you went up 1 point, in 4 games you are still back to the starting amount.

Yes i agree. I think in a healthy game community an expert should be able to play with noobs and risk a defeat at the hands of luck without being severely punished in ELO. It just segregates the community too much because an expert will only play with other experts and cant enjoy the game with new players. IT JUST DOESN'T MAKES sense and really cheapens the idea of expert and master to mean performance in the last few games...of course expecially in games with a high luck factor this is an esentially MEANINGLESS criteria. CHANGE IT PLEASE.

I have noticed things in particular games which i have played extensively.


SABOTEUR
In games with larger numbers the ELO fails miserably. In saboteur, where table numbers are often 7 or 8, luck is high. According to elo, there is only 1 expert in in sabotuer in the whole community, essentially this has gone to a good player who got a very lucky streak and stopped playing once he got to expert. This is silly. There are a number of people who know saboteur really well and are equivalent in expertise to say an expert in hanabi but the elo system fails to note it as it is dogmatically the same for each game, requiring 2000 points. This is silly


6 NIMMT
normal game 5 players. the label of MASTER is deeply flawed. Why? Because it is not possible to remain a master consistently as luck is heavy and performance is only shown in long-term stats, which of course the ELO system ignores. Most of the "MASTERS" in 6 nimmt stay master for about 2 or 3 games before they fall, even though they are consistently better than expert level in overall performance. Again this is not taken into account in ELO. Also t has resulted in the very sad result of various players who want to use a lucky streak to get into top 10 etc and stop playing. THE LONG TERM MASTERS OF 6 NIMMT ARE ALL INACTIVE. Clearly they just wanted the label of such, and then stopped playing. This kind of behaviour is implicitly encouraged by the elo system, as it gives its titles to short term wins. THIS IS WRONG AND NEEDS URGENTLY TO BE RE-THOUGHT FROM THE GROUND UP.


There has to be in my mind a massive shift in values of the elo system. It is too cheap, short term and encourages inactive players to attain a title and only keep it by being inactive. How sad is that? ... STOP CONGRATULATING AND BESTOWING THESE CHEAP TOP WHATEVER RANKINGS ON OPPORTUNISTIC FAKE GAMERS WHO ONLY CARE ABOUT APPEARANCES. STOP LIFTING UP THOSE WHO JUST PLAY THE SYSTEM BECAUSE OF ITS RIDICULOUS FLAWS I AND OTHERS HAVE POINTED OUT ABOVE. There has to be a change, anyone with common sense can see that. Depose this 'playing the system' culture of your flawed elo. Start caring about solid good indicators of excellence in a player, please for the love of God...


I would strongly encourage the developers on this site to get some player feedback from the consistent players on this site as a beginning to redraw a system which is more sound and is fairer towards honest gamers who play without playing the system and just are solid excellent players, etc.

[Slimbo]

The only fix i can see that would be an improvement is if it took the point spread into consideration.

[cosmosis]

I'm in the same 6 nimmt situation as you. I've been to master several times and fallen, but I don't see that as a flaw in the system. ELO is great at what it does, which is to calculate the probability that you'll beat another person. ELO is not designed to say that one person is better than another.

My ELO might fluctuate, but I can pretty consistently stay in the Expert range at 6 nimmt now. In more random games, you shouldn't look at your ELO at any specific point in time, but your average ELO over your most recent n games. If that's going up, you're doing well. If you can consistently stay in expert and above for most of your games I'd say you're an expert player. Same with master. I don't think just getting to 2000 ELO once or even a few times means you're a master player.

Overall, I'm a fan of ELO. I don't feel that many other ranking systems are better than it.

THE BAD:

Titles

I do feel that a case can be made against how BGA treats titles. The ELO spread of different games varies a lot (generally the more luck-based a game, the smaller the spread). This means that with BGA's definitions of Expert, Master, etc. there are some games that have no masters, or even no experts (like Senet), while others have many.

Perhaps the fix here would be to decouple the titles from the ELO system. A Master could be top 1% of players in a specific game, expert top 10%, etc. However, I kind of like the current definitions just because they're static, so if I'm an expert one day I'm guaranteed to still be expert the next. Perhaps, then, the titles could be set to different ELO levels for each different game? But that's pretty messy. BGA's way of handling this might not the best, but I think it's okay.

I don't think this is a problem with ELO as a ranking system.

Stagnation

THE LONG TERM MASTERS OF 6 NIMMT ARE ALL INACTIVE.

ELO does have an issue with stagnation. The higher you get, the less you want to risk your position. But BGA makes sure that players can't just sit at the top forever. If a player remains inactive for a few months they disappear from the ratings. This is a slow process, but it means the top players must eventually play to stay at the top.

Being at the top does not mean people no longer play games, or that they can no longer play with new players. Just play with ELO off. I see this all the time in games like Race for the Galaxy. If you want to have some fun with new players go teach a game to people in the Game Tutors group. You can get large groups of players to play there with ELO off.

Conclusion

I'm not really sure what you want to replace ELO with. It seems like your gripe is more with earning a title and then having it disappear again. Maybe you want it so that once you've hit master once, you always have that title? But there are some issues with that.

Perhaps you want it so that ratings change less the more games you play. But then leader boards become harder to move up/down. I'd rather have the potential to gain a lot of ELO quickly than have to progress up or down by a few ELO points each game. It's more exciting, and makes those wins so much nicer.

The more luck-based a game, the more you have to look at average ELO's over many games, and that's the way it should be.

I would strongly encourage the developers on this site to get some player feedback from the consistent players on this site as a beginning to redraw a system which is more sound and is fairer towards honest gamers who play without playing the system and just are solid excellent players, etc.

Consider me a solid supporter of the ELO rating system. Titles could maybe use some changes.

[kaneaaa]

Sure but what about the issue that ELO vastly overrates the effect of you last games. That is not consistent with the titles of 'good player' or whatever because it doesn't make sense that the last 3 plays should determine 90% of the ELO number, especially in games with a significant luck factor. It just doesn't make sense, it's completely inconsistent with the meaning of the word expert etc. Also, it's completely counter-intuitive as a way of grading performance. In relation to what you said about limiting how fast people move up and down, the result of the system currently is that constant large swings in number become an arbitrary indicator of performance. It is also counter-intuitive to common sense that a person's performance swings widely and quickly, and I disagree with excitement. Maybe initially you may find that 'exciting' but when you see that over time these swings are cheap and fast, they are quite dull actually and are very misleading. Also, you (BGA) can't have it both ways. You either say that the elo basically shows your recent performance over a very small sample of games (which I repeat in games with high luck factor is essentially meaningless), OR on the other hand you have a rating which shows if people are 'expert' etc. which is a TOTALLY different criteria from looking at a recent small sample size. YOU CAN'T HAVE IT BOTH WAYS, it's just patronizing to the community I feel to say an expert is an expert because he had a lucky recent streak, or vice versa a person is not expert because he had a recent bad spell which was not of his own doing, etc. Either just have the ELO number and be honest about what it represents, or have the titles BUT have them based on strong and more long-term criteria. The words expert and master are very STRONG words in the English language, and in no kind of context merited over small sample sizes, it's just complete logical contradiction, I'm afraid, to be blunt.


"I do feel that a case can be made against how BGA treats titles. The ELO spread of different games varies a lot (generally the more luck-based a game, the smaller the spread). This means that with BGA's definitions of Expert, Master, etc. there are some games that have no masters, or even no experts (like Senet), while others have many.

Perhaps the fix here would be to decouple the titles from the ELO system. A Master could be top 1% of players in a specific game, expert top 10%, etc. However, I kind of like the current definitions just because they're static, so if I'm an expert one day I'm guaranteed to still be expert the next. Perhaps, then, the titles could be set to different ELO levels for each different game? But that's pretty messy. BGA's way of handling this might not the best, but I think it's okay."

Yes I like that idea. I feel there is a weakness in defining expert as say 1800 for all games, because expertise in something like SABOTEUR (mainly because a decent game needs about 7 people) is shown far less in small sample sizes compared to say a 2 player game of Caylus. I do feel that the higher the player number, the weaker the ELO system is at gauging performance, and shows its flaws more. Also, if someone gains 'expertise' in a game, he doesn't lose it if it is expertise in the meaningful sense of the word. For example, I had 4 or 5 games of saboteur where I was doing very well averaging about 2nd place amongst 5-7 players, and went up a 100 ELO, and then had a really poor game where people were making lots of mistakes which had knock on effects on other players and I also happened to have a very bad draw which gave me little room for strategy and finished last amongst 7 players, and dropped that 100 immediately. Common sense tells me that that is complete rubbish as a system, I'm afraid. You tell a person that he is no longer 'good' etc. because he had a bad game beyond his control in a game which has a high element of luck factor and in which the mistakes of some people in the game can have negative effects on others in the same game beyond his control. This is just not right, UNLESS you take ELO to be an indicator of recent performance in which case the 100 drop is still a completely over-exaggeration and is, again, very patronizing as an indicator of my performance.

Cosmosis, your decoupling idea is interesting and may work if implemented properly.

In any case I think I have made it clear as to the very significant problems that are happening here in the ratingd/titles system on BGA, and how it is only favouring those who play the ELO system rather than the games.

[Lolipops]

hi, i ve played a lot( 15k + ) games in chess, wich use elo . for me elo is a good and bad indicator:

good cause it shows a mix of your skill, how much you blunder, and your "skill of these days"

and bad here because for me you should have a different elo for different numbers of players:
some games are different and need different skills, even quite no skill dipendending on the numbers of players :
6 nimmt in 2 5 10 players lead to the same elo Oo
hannabi in 2 or 5 player
caylus 2 or more
stone age, race, colt express .. etc etc

cause when you play with or versus someone, with totally wrong elo, based on a different number of player, it can just ruin a game

[intrepid]

hi all and sorry for my typing errors.elo system is severly hurting the great game of Hearts. #1TO MANY PLAYERS DONT TRY TO GET LEADER WHICH IS THE OBJECT OF GAME. #2 IF THEYED LEARNED TO GET LEADER BETTER GAMES AND WOULD MAKE THEM A BETTER PLAYER. THKS FOR LISTENING.

[tap-zbyna]

kaneaa has a point...

Once in libertalia I was second from six and I lost points on that game! (everyone who has played Libertalia knows, that finish second in 6 players game is pretty good)

Since that I play 1v1 only, since it is the fairest way to work with ELO.

There should be maximum possible loss of ELO - lets say 40 points. If you have 1800+ ELO, one loss with a "weaker" player (is someone with 1650 points really weak?) would cost you so many points, that you would have to win like 20 games to get back on your points. It really does not make sense.

[squarerootofthree]

I have absolutely no problem with the ELO system. It's used in so many other settings. For everyone so concerned dropping a certain # of points when you lose a game, don't worry about it! Play enough and ELO will be pretty accurate over the long run.

But this thread spawned a great idea from Cosmosis.

I do feel that a case can be made against how BGA treats titles. The ELO spread of different games varies a lot (generally the more luck-based a game, the smaller the spread). This means that with BGA's definitions of Expert, Master, etc. there are some games that have no masters, or even no experts (like Senet), while others have many.

Perhaps the fix here would be to decouple the titles from the ELO system. A Master could be top 1% of players in a specific game, expert top 10%, etc. However, I kind of like the current definitions just because they're static, so if I'm an expert one day I'm guaranteed to still be expert the next. Perhaps, then, the titles could be set to different ELO levels for each different game? But that's pretty messy. BGA's way of handling this might not the best, but I think it's okay.

That makes a lot of sense to me. Especially since in high-luck games it's essentially impossible to reach into the 2000s. Instead, the percentile system would normalize that.