Why ELO doesn't suit to BGA

Suggestions & ideas / Suggestions & idées
User avatar
ollyfish2002
Posts: 302
Joined: 29 January 2015, 09:32

Re: Why ELO doesn't suit to BGA

Post by ollyfish2002 »

I don't care about my ELO.
But it is annoying when I cannot play because of my ELO in some games (in either way, too high or too low).
I think there should be an Experience index that counts the number of finished games (not the games being quitted and lost, the ones that give a real result), the number of wins.
This way, games can be restricted on this basis.
I play Hanabi. There is more than luck in the deck. I have played with players trying to reach perfect scores and asking to abandon the game if it is not perfect, or bombing if somebody do not agree with them.
Is this my fault to lose ELO ? I don't think so.
I still think you give too much points to ELO ranking than Playing. Encourage playing more than cheating (See what happens in Can't Stop).
my2p
demiurgsage
Posts: 229
Joined: 07 August 2015, 10:08

Re: Why ELO doesn't suit to BGA

Post by demiurgsage »

ollyfish2002 wrote:But it is annoying when I cannot play because of my ELO in some games (in either way, too high or too low).
I think there should be an Experience index that counts the number of finished games (not the games being quitted and lost, the ones that give a real result), the number of wins.
The fact is that some players aren't able to learn. I played several times "Race for the Galaxy" with opponents who has more than 5000 games played. Their moves were completely pointless. The rating isn't bad thing, but it should be fair.
User avatar
cosmosis
Posts: 107
Joined: 06 April 2014, 08:10

Re: Why ELO doesn't suit to BGA

Post by cosmosis »

Am I the only one who likes the ELO system?

I do agree that it's not a perfect system, but I don't think any rating system is perfect. ELO is a good, mathematically fair, well-known-and-tested system though. I think switching to a new one would be a lot of unnecessary work, and then people would just complain about the new system. From the forums, I have seen some people complain about the ELO system but it's mostly been a few, very vocal complainers. For the vast majority, the system works; and I think BGA has other areas it should concentrate its efforts on.

That being said I agree that tournaments are currently a weird system. I would love to see something like sourisdudesert's season-rating suggestion being implemented for tournaments. In fact, I'd love to see more integration for tournaments within the site. So far, they've been kind of a weird offshoot. Awarding yearly ratings based on tournament performance would be awesome. I also think ELO-off should be the standard setting for tournament creation – maybe even the only setting?

I think switching the leaderboards to a season-based rating system risks a lot of negative feedback from everyone who has spent a long time working their way up those leaderboards... I know I'd be upset (and for the record, I don't play with convenient opponents, etc. beyond restricting games to certain ELOs).
User avatar
cosmosis
Posts: 107
Joined: 06 April 2014, 08:10

Re: Why ELO doesn't suit to BGA

Post by cosmosis »

Here are my responses to some of the thoughts in the thread:
The calculation of ratings' changes after the game end are based on a player gain victory vs every player. [5th player loses a lot of ELO, etc...]
Seems fair to me... In a 5-player game it's much easier to avoid 5th place than avoiding 2nd in a 2-player. So getting 5th should give a bigger penalty.
ELO prevents people from playing, especially in luck-based games.
Playing ELO-off seems a good solution. This works great in 2-player games, especially if you just want to learn from the master and don't care too much about ELO. I can see that it might be hard to start a game with more players and ELO off.
ELO prevents high-ranking players from joining tournaments. They are afraid of playing with people with far less rating. ELO doesn't have the maximum of rating's change in one game.
The solution I've seen is that players start ELO-off tournaments, which I've been happy with. I think ELO-off should be the standard case for tournaments.
ELO promotes opponent selection. Players try to play with convenient opponents to increase their rating.
Literally all rating systems seem to have this problem.
ELO promotes table restrictions. You can often see players with rating about 1610 that restrict table for 1600+.
You could start your own game without restrictions then? Some games I wouldn't play without restrictions like that. In some games, I want to count on other players denying opponents and playing with more strategy, and restrictions are necessary for that.
ELO doesn't show how good player is. In some games 1850 is very high level. It depends on number of active players and history of playing the game.
ELO does vary by game. Games with more luck have a lower ELO spread. I don't think number of active players has much effect, actually (for example, check out ratings for Dark Agent which has few players - around 4000 plays total - but the highest ELO is 2400).

If you're worried about titles being impossible in some games (for example, there are no Master players in Senet), that could be a concern. But varying titles earned based on which game is played and its ELO-spread just seems needlessly confusing to me.

1. The rating of a player with big number of games should change very slow. If you played 200 games every new game give you only a little expirience. The rating should reflect your skill.
I would not like this. Oftentimes my first games are learning games and I lose ELO. I like the ability to climb the ranks quickly after learning a game (even if it means I can also fall quickly). The ability to rise/fall quickly with ELO is exciting to me.
2. The changes of the ratings in result of a game should take into account the randomnicity of the game. A victory in a skill-based game should increase your rating more than a victory in a luck-based game.
This already seems taken into account to some extent since in skill-based games you can achieve higher ELO's than in luck-based games.
3. The rating of a player who play too little should decrease over time.
I have played on sites that use this type of system and I hate it. HATE IT.
It makes me feel bad for not playing on a website for a while. It makes me less likely to return to that website.
If its implemented for specific games, it would make me feel bad for not playing a specific game. I have played almost all of the 90ish games on BGA and imagining all of those ELO ratings dropping if I didn't keep playing all the games. Ecchh....
4. (Not necessarily, but can make rating system more stable.) New players should start with no rating. They'll receive a rating after 10 games or after playing vs 20 players total in games with rating. Until they get the rating, their results won't affect the ratings of other players.
This would probably make people less likely to play with new players.
User avatar
Oso
Posts: 701
Joined: 29 September 2011, 04:04

Re: Why ELO doesn't suit to BGA

Post by Oso »

Also ELO should be like in chess system, when you got X ELO the K number should decrease.

30 or less games = K 60

-1599 = K 40
1600-1799 = K 30
1800-1999 = K 25
+2000 = K 20
demiurgsage
Posts: 229
Joined: 07 August 2015, 10:08

Re: Why ELO doesn't suit to BGA

Post by demiurgsage »

I see some people think that tournaments should be played without ELO/rating. I strongly disagree with it. What about players who play only/mostly tournaments? They will never get a rank. Some of them play on Master level but they are just Average or Good players.

2 cosmosis
I understand your position that new system can be worse than current one, but I think it can be better.
I comment some of your statements.
1. ELO isn't mathematically fair for luck-based games. ELO assume that one of the players wins due to a skill advantage. But it isn't a case for luck-based games. Even the worst player in the world can win due to luck.
2. I don't suggest to work on change of rating system instead of improving stability of site or fixing some annoying bugs.
3. Every good idea can become a nightmare due to an awful realization. It concerns your worry about decrease of rating's changes and decrease of rating over time. There will be minimal value, so changes of rating will not become zero. As Oso wrote this decrease can be connected with skill level of players. The decrease of rating overtime is needed only for best players not to allow them play one game in a month. It can be very little or zero for other players.
4. Why in skill-based games top players have higher ELO? Cause there is no chance for an Average player to win against a Master in a skill-based game. So you can increase your ELO to whatever number you want playing against Average/Good players.
Liallan
Posts: 1221
Joined: 26 May 2014, 07:01

Re: Why ELO doesn't suit to BGA

Post by Liallan »

cosmosis wrote:Am I the only one who likes the ELO system?

I do agree that it's not a perfect system, but I don't think any rating system is perfect. ELO is a good, mathematically fair, well-known-and-tested system though. I think switching to a new one would be a lot of unnecessary work, and then people would just complain about the new system. From the forums, I have seen some people complain about the ELO system but it's mostly been a few, very vocal complainers. For the vast majority, the system works; and I think BGA has other areas it should concentrate its efforts on.

That being said I agree that tournaments are currently a weird system. I would love to see something like sourisdudesert's season-rating suggestion being implemented for tournaments. In fact, I'd love to see more integration for tournaments within the site. So far, they've been kind of a weird offshoot. Awarding yearly ratings based on tournament performance would be awesome. I also think ELO-off should be the standard setting for tournament creation – maybe even the only setting?

I think switching the leaderboards to a season-based rating system risks a lot of negative feedback from everyone who has spent a long time working their way up those leaderboards... I know I'd be upset (and for the record, I don't play with convenient opponents, etc. beyond restricting games to certain ELOs).
I couldn't agree more with this. Well, except the part about the tournaments being weird cause I don't do tournaments and have no idea what's going on there. But I do agree about the part that people who spent a long time working up in a leaderboard could be very upset over a change like that.

Now, back to just the basic ELO concept... I don't know that I "like" it, but I don't dislike it. i.e. I'm not that concerned about it one way or the other. I have just enough ego to like watching when mine goes up, but I'm aware it's not perfect and I'm not losing any sleep over my ELO's (that I can't see anyway).

The important point in here is that it would be a lot of unnecessary work, and people would complain about it anyway. On Yucata, they use Trueskill, and people are constantly complaining about it and pointing out all its flaws. So using that system won't solve the complaining. Here they have ELO and everyone is constantly complaining about it and pointing out all its flaws. It isn't going to matter which system you use - people WILL complain about it and you aren't going to stop that. Ever. Period.

I really think there's very few things to deal with in terms of ELO. Like having ELO on with the bots wasn't appropriate (but apparently an error). Might be better not to use it for Hanabi. But mostly I think it works fine, I don't have any issue with it. I also think it works fine for high luck games. Someone pointed out that you can't get as high in those. That seems to be the case with the Trueskill system as well. And some might think, well yeah, but if you can't get as high you can't easily get the higher rank. Well, if it's high luck, why should you be able to? I mean, seriously, a Master at Can't Stop??

What we do with ELO I think is more important, not the ELO itself. I don't mind having the ranking titles, probably cause I don't take them all that seriously. I think it's great using them for restricting tables. (I don't know who it was who thought it was a problem that someone at 1610 restricts a table to >1600. So what?) I think the restriction thing could use some work, which has been discussed in other threads so not sure I want to address that again here. (In a nutshell, I think the 200 pt spread is way too much and it could use a bit of customization.) And we do seriously need something below Average. But this is how ELO is being used, not about the ELO itself.
User avatar
sourisdudesert
Administrateur
Posts: 4630
Joined: 23 January 2010, 22:02

Re: Why ELO doesn't suit to BGA

Post by sourisdudesert »

Liallan wrote: It isn't going to matter which system you use - people WILL complain about it and you aren't going to stop that. Ever. Period.
True. That's why we are going to keep ELO as an indicator.

What we want to improve is elsewhere : best players should play against other best players to compete to be the "BGA champion". With ELO this is almost impossible to do it, so we are looking for another system for leaderboards.
Willie333b
Posts: 116
Joined: 06 June 2016, 13:57
Location: NTC, TWN, ROC

Re: Why ELO doesn't suit to BGA

Post by Willie333b »

An example of my opinion:
3 players, player ABC get the 1st place; CAB 2nd; and BCA 3rd. ABC won 2 players: (ABC vs. CAB + ABC vs. BCA) divided by 2.
That end in my opinion.
demiurgsage
Posts: 229
Joined: 07 August 2015, 10:08

Re: Why ELO doesn't suit to BGA

Post by demiurgsage »

For me complaining of people isn't a point. People often make mistakes. I think the rating should be fair and competitive. If you do such system on base of ELO or another rating system it will be great.

Many people don't understand what's wrong with using ELO in luck-based games.
I'll try to explain it using math.

R1' = R1 + K * (S1 - E1)
This is the formula for the new rating after the game, where R1, R1' - old and new rating of a player, K - K factor, S1 - result of the game (1 - victory, 0.5 - draw, 0 - defeat), E1 - expected result of the game calculated on difference of players' ratings.
This formula can work for luck-based games. The thing you need to do is to count E1 correctly.
For skill-based games E1 calculated directly:
E1 = 1 / (1 + 10 ^ ( (R2 - R1) / 400 ) )
But for luck-based games E1 should take into account the random factor L. L shows how random the game is (0 - no random, 1 - full random). Every player gets additional chance to win equal L/2. The skill part of the game is decreased accordingly.
So the corrected value of E1 will be E1*
E1* = L/2 + (1 - L) * E1
The main problem is to find correct L for the game.
For "Can't Stop" it can be ~0.8 - 0.9, For "Race for the Galaxy" ~0.4, For Stone Age it's less than 0.1

E1* - E1 = L/2 - L * E1 = L * ( 0.5 - E1 )
It's easy to see that E1* is less than E1 when E1 is greater than 0.5 what happens in the case R1 > R2.
So every time you play with a player lower ranked than you he steals your ELO on average. And every time you play with a player higher ranked than you, you steal his ELO on average.
Locked

Return to “Suggestions”