ELO system is a failure

[senatorhung]

I have absolutely no problem with the ELO system. It's used in so many other settings. For everyone so concerned dropping a certain # of points when you lose a game, don't worry about it! Play enough and ELO will be pretty accurate over the long run.

But this thread spawned a great idea from Cosmosis.

I do feel that a case can be made against how BGA treats titles. The ELO spread of different games varies a lot (generally the more luck-based a game, the smaller the spread). This means that with BGA's definitions of Expert, Master, etc. there are some games that have no masters, or even no experts (like Senet), while others have many.

Perhaps the fix here would be to decouple the titles from the ELO system. A Master could be top 1% of players in a specific game, expert top 10%, etc. However, I kind of like the current definitions just because they're static, so if I'm an expert one day I'm guaranteed to still be expert the next. Perhaps, then, the titles could be set to different ELO levels for each different game? But that's pretty messy. BGA's way of handling this might not the best, but I think it's okay.

That makes a lot of sense to me. Especially since in high-luck games it's essentially impossible to reach into the 2000s. Instead, the percentile system would normalize that.

the only issue i have with the ELO system is that it is not really designed for multiplayer games. if ELO as currently calculated were restricted to 2.player matchups, i think that would be fine. i did come across http://www.gautamnarula.com/rating/ which implements a multi-player ELO system for settlers, but i'm not enough of a math geek to be able to figure out whether or not it would work better for random luck games than what is currently in place on BGA.

the problem with the suggestion to decouple the 'master' ratings from ELO hard values, is that the ratings are still based on the ELO which can fluctuate extremely. so yes, you could become a master at 6 nimmt but you will likely only hold that ranking for a few hours until your next match. some way of incorporating win.rate and final position (1st worth more than 2d worth more than 3rd etc.) could smooth out some of those fluctuations, but then you're getting into more arbitrary weighting decisions that might just end up muddying things further.

i am happy to try to boost my ELO in certain games that i find to be less luck-based (Takenoko, Battle Sheep, Palaces of Carrara), but in other more luck-based games (Race for the Galaxy, 6 nimmt, Color Pop), ELO is not a real consideration for me. that said, 4.player Takenoko or Battle Sheep is much more luck-oriented than 2.player matchups of those games.

[Pisaac]

It should be where once you get past 1600, you get to level 2, good player, and you can't go down again below level 2, etc etc...

[kaneaaa]

Also think about this, a bad rating system sends the wrong message to the community on BGA... something to think about. It affects the gameplay, and the attitudes of gamers....

[Meowcate]

the only issue i have with the ELO system is that it is not really designed for multiplayer games.

I have to agree with that for another reason. I recently was in top20 Colt Express (woohoo), and 2 days later back around ~1500. That happened ? My ELO score was too big apparently. When I play on 1v1 games like Season I'm ok with this. When I play with 4-5 players around my level, ok too. But when I play against some lower scores, there is a particular problem : even if the current game can let one of the others guys win, players often focus against me because as the biggest ELO here, I'm the threat. And I understand that, I take a look at bigger ELO in multi too, but not to forget the goal of the game.

Sure, it's only based on my (bad and recent) experience, but I think showing ELO can be a problem on 3+ players games, when it's no more about winning, but don't let the biggest guy win.

[Shaq Jenkins]

the only issue i have with the ELO system is that it is not really designed for multiplayer games.

I have to agree with that for another reason. I recently was in top20 Colt Express (woohoo), and 2 days later back around ~1500. That happened ? My ELO score was too big apparently. When I play on 1v1 games like Season I'm ok with this. When I play with 4-5 players around my level, ok too. But when I play against some lower scores, there is a particular problem : even if the current game can let one of the others guys win, players often focus against me because as the biggest ELO here, I'm the threat. And I understand that, I take a look at bigger ELO in multi too, but not to forget the goal of the game.

Sure, it's only based on my (bad and recent) experience, but I think showing ELO can be a problem on 3+ players games, when it's no more about winning, but don't let the biggest guy win.

The same thing happened to me in Colt Express. I was the 3rd Best Player, and I severely fell down in ELO from Expert to barely a Good Player in just one game because the other 3 players ganged up on me, which I expected. It was actually a 5 player game; one player went for the strongbox, got the gunslinger achievement, and ended up winning because the other players were more focused on keeping me from winning than trying to win the game themselves. That's one of the reasons why there aren't any 2000+ ranked players in multi-player games like Colt Express and Saboteur.

Though in Colt Express, I have much more fun playing when I'm in the Good Player or Average Level range because I'm less of a target and I'm not unfairly getting shot 5 times per round.

I agree that ELO works perfectly for 2 player games like Seasons but not 3+ player games.

[kaneaaa]

These are all really good points! I wonder if some admin from bga can also give their opinion along with the players?

[kaneaaa]

Yes I have to say the elo system is awful. The fact that in 6 nimmt, 6 5-player games of getting 1st and 2nd ranking can be completely negated elo-wise by one 5th-place result just shows how bad the system is. No word from the developpers acknowledging the problem. I can't spend money subscribing to a site which has a system which is so badly implemented like this. You're not being fair on the players firstly, and secondly by having such a meaningless system, you are encouraging this culture of people just getting lucky to get in the top rankings and then stopping playing the game, because they very well know that the more they play the more likely they are going to drop 100 points in about 1 or 2 games no problem. Big surprise that the 'top players' in ranking on a game are just inactive, or shamefully finding a couple of beginner players to play against to bump up their stats from time to time. It shows what your 'Halls of Fame' is largely just a list of people who are mostly concerned with playing the system to look good, and that that is exactly the thing that BGA is encouraging. I'm actually really quite frustrated that the developers haven't pointed out these obvious problems. If it continues, I'm done with this site and will find somewhere else that treats its players fairly rather than just releasing more and more games with this bad fundamental system.

The system is just an insult to intelligence. Fix it or I'm out. And please communicate with your player base about it, because if you don't you show you just don't care...........