Page 1 of 3

New ELO formula

Posted: 21 May 2012, 21:58
by sourisdudesert
Hello,

This new topic to gather discussions started in several topic of the forum about the new ELO formula: Your criticisms are:
  • Players having high ELO with previous formula will keep having high ELO because it's way more difficult to gain&loose points with new formula.
  • Players who loose an important number of points during the last period of time will have difficulties to restore their rankings.
  • The new formula does not give enough points - which is not really encouraging.
Facts are:
  • With the old formula, when 2 players with the same ELO plays together, they gain/lost +30/-30 points.
  • With the new formula, when 2 players with the same ELO plays together, they gain/lost +15/-15 points if they played less than 30 games, and +7.5/-7.5 afterwards.
Benefits from the new formula are:
  • More accurate rating system. By using the ELO formula, the rating difference between 2 players determine their probability to win.
  • The new players rating converge to their "real" value more faster, and are more "stable" afterwards.
How do we address the "N-players" games with N>2:
  • We consider that each player win a 2-players game against each opponent after him in the game result, and loose a 2-players game against each opponent before him in the game result.
  • This way, a N-players game is equivalent to (N.(N-1)/2) 2-players games.
  • Of course, the ELO difference is divided by (N-1) because there is no reason than a game with a lot of players makes the ELO ranking progressing faster than a 2-players game.
What we will do to address the issues:
  • Find with you what is the best "elasticity" for ELO (like before ? less than before ?).
  • Recalculate ELO rankings, taking into account all games from the beginning of BGA (yes, this is feasible).
  • This way, everyone will get an ELO ranking as if we always used this new formula in the past.


Don't hesitate to post your feedback there.

Cheers,

Re: New ELO formula

Posted: 21 May 2012, 22:14
by Roger
Bienvenido el nuevo sistema de puntuación.
Pero no pueden implantar este sistema dejando los records anteriores, es dejar permanentemente a los high levels en el mismo puesto. Así los nuevos jugadores no tendrán motivación por la puntación, quitando un gran aliciente de la pagina.
Deben buscar la manera de re-calcular los records actuales y ajustarlos lo mas cercano a los valores reales basados en el nuevo sistema.
No deben ni resetear, ni poner un numero tope o cualquier metodo rigido. Se debe respetar el desempeño de los jugadores y basarse su re-ajuste en un calculo igual para todos.
Deben comparar las formulas de calculo nueva y la vieja, y buscar un equivalente de re-conversion.

Roger

Re: New ELO formula

Posted: 22 May 2012, 00:33
by yaron
Thank you for listening to the feedback!
Here are a few comments:
More accurate rating system. By using the ELO formula, the rating difference between 2 players determine their probability to win.
It's true that the rating difference in ELO determines the probability of winning.
However, this is true of all ELO based schemes, including the one you used before.
So, the old system was just as accurate as the new scheme.
The only thing that changes is the table that converts rating difference to probability of winning.
Here are the tables for both systems:

Code: Select all

Win %       New system       Old system 
90%            366             752
80%            240             564
70%            149             376
60%            72              188
50%            0               0
As you can see, both systems have a relation between your probability to win and your difference from the opponent, and both are "accurate" implementations of Elo's idea.
Which is better? Empirical data shows that a linear table (the one you used before, on the right) is better then Elo's original table (the left) at predicting the results of professional chess games. However, chess is a different game from most games we play here (deeper strategy, no luck), so it's possible that for us this is not correct. In any case, both are probably good enough.
Players having high ELO with previous formula will keep having high ELO because it's way more difficult to gain&loose points with new formula
I think some of the critics are mixing up two different effects.

Having low K (low elasticity) makes it slower to gain and lose points. But it doesn't make it impossible to reach high scores - you just have to play more. I will discuss this later.

However, the change in the table (above) does make it impossible to reach (or maintain) a high score, no matter how much you play.
For example, suppose I can beat new (rating 1500) players 90% of the time. Under the old system, my rating would converge to 1500+752 = 2252. With the new system, I'll have 1500+366 = 1866, even though I'm playing at the same level.

You will see this when you recalculate the scores. No matter what K you choose, the best ratings will be a lot lower than they were with the old system.
However, if you do recalculate the scores, than this is not a problem.
How do we address the "N-players" games with N>2:
We consider that each player win a 2-players game against each opponent after him in the game result, and loose a 2-players game against each opponent before him in the game result.
This way, a N-players game is equivalent to (N.(N-1)/2) 2-players games.
Of course, the ELO difference is divided by (N-1) because there is no reason than a game with a lot of players makes the ELO ranking progressing faster than a 2-players game.
This is great!
Find with you what is the best "elasticity" for ELO (like before ? less than before ?).
I'm in favor of the old level of elasticity.
Low elasticity makes stable scores, but stable scores are not correct - because they don't reflect recent changes in ability.
Additionally, it is frustrating to have to play very many games to see any change.
Because all scores will be closer together (as I explained above), a K of 30 (gain/lose up to 15 points each game, like in most implementations of Elo) will be equivalent to the K=60 we had before. I think this is great - I'd rather not have a lower K.

Thanks again!

Re: New ELO formula

Posted: 22 May 2012, 05:50
by zfm
Recently I played a Puerto Rico game in a 4p game, our ratings (before the game), respectively to the final position, are 2077, 1777, 1859, and 1583.

After the game finished, I saw that our rating changes, respectively, +2, +4, -3, -5. I have to say two things:
1. It is totally unpleasant to play a good 4p game, with players with quite high ELO (their ratings average was around 1740) and got only 2 points! Yes, I was not playing just for the ELO, but or a total changes of 14 in 4p game with quite near was totally disappointing.
2. My rating and the 2nd rank difference was 300, while the difference between the 3rd and 4th was 270. Yes I know that 300 is bigger than 270, but I can't believe with only 30 difference, I got 2 pts lower than the 2nd rank, while the 3rd rank got 2 pts higher than the 4th rank. A difference of 30 pts costed 4pts (on a game with total of 14pts, 4 pts is around 28%)? That was totally strange for me. Some explanation needed here.
3. I agree to recalculation of all the data, yet I believe that this game (and all the game after the new implementation system) needs to be recalculated also, because I'm sure that our ratings were not 2077, 1777, 1859, and 1583, but rather around 1800, 1600, 1700 and 1550.
4. As said by Yaron, I also prefer more K, K = 30 (or 15) is too low. I wonder how a chess player got 2500 by this calculation.

Re: New ELO formula

Posted: 22 May 2012, 08:04
by fdillon
I agree with zfm. There is not enough at stake in multiplayer Puerto Rico in general.
Puerto Rico is ideally played with 4 players but there is little point in playing anything other than a 2-player if you are playing for points.
The highest ranked players play the large majority of their games as 2-player because there is not enough incentive to play with more players.
As well as providing better, more idealised gameplay, 3+ player PR games have a much larger time commitment than 2player games.
For these two reasons I think that 3+player PR games should award more ELO points to the victor than 2player games.

Re: New ELO formula

Posted: 22 May 2012, 09:33
by Masathyr
The elo ranking means nothing per itself.

If the elasticity is 30 points per match, then you can be master around 2000.
If the elasticity is 3 point per match, then you are a master around 1640 :)

So for me the real solution is "Recalculate ELO rankings, taking into account all games from the beginning of BGA (yes, this is feasible)."

Re: New ELO formula

Posted: 22 May 2012, 10:36
by sourisdudesert
Okay, thanks for feedbacks.

I think you're right when saying the elasticity should be high to make the whole thing "fun". However I really think this elasticity is was too high before (ex: some player lost 300 ELO points during a single week-end...).

What if we do the following:
  • K=60 for the first 30 games (= same elasticity as before).
  • K=30 afterwards (= +15/-15 points per match for a 2p-game)
It would make the ranking twice more "stable" than it is today after 30 games. For the first 30 games, the gain/lost will be approximately like before.

Another additional proposal: we can introduce an elasticity bonus for game with more than 2 players, in order to take into account the fact that these games are more time-consuming and to encourage this type of game. Full example:
  • For 2p-games: K=60 for the first 30 games, K=30 afterwards.
  • For 3p-games and more: K=75 for the first 30 games, K=45 afterwards
It's also possible to have a more detailed ranking system (ex: different rules per game), but it would imply that we have this discussion for each new game :)

Don't hesitate to post your feedback.

Cheers,

Re: New ELO formula

Posted: 22 May 2012, 17:41
by FulkNerra
I think I lost 300 points in a weekend (or was it a single day?) so I agree that ratings swung too severely. However, whatever system you adopt I'd like it to be applied retroactively to the games we played before the change. The ratings are a big motivator for play here and consistency is a good thing.

Re: New ELO formula

Posted: 22 May 2012, 19:24
by yaron
sourisdudesert wrote:
  • K=60 for the first 30 games (= same elasticity as before).
  • K=30 afterwards (= +15/-15 points per match for a 2p-game)
I think these are good numbers.
sourisdudesert wrote:Another additional proposal: we can introduce an elasticity bonus for game with more than 2 players, in order to take into account the fact that these games are more time-consuming and to encourage this type of game. Full example:
  • For 2p-games: K=60 for the first 30 games, K=30 afterwards.
  • For 3p-games and more: K=75 for the first 30 games, K=45 afterwards
Excellent idea!

Are you still planning to introduce the "no more than 400 points difference" rule? I think it can be easily abused by strong players that only play very weak players, getting "free" rating.

Thanks,
Yaron

Re: New ELO formula

Posted: 22 May 2012, 20:13
by sourisdudesert
Thanks for your feedbacks

@eatpraylove: the new formula will be applied retroactively.

@yaron: With this 400 limit we ensure that when you win a game you gain at least 3 points. It would be nice to keep at least a system that give a player at least 1 point. If we set up a 600 limit, a victory brings at least 0.9 ELO point: could be a good compromise...