Thoughts about Elo

[Woodruff]

Link: https://boardgamearena.com/bug?id=13021

Hi everybody

I would like to share here some ideas about the current Elo system (which applies both in Arena and non-Arena games) and the player categories associated with it. I think that system is already very good but I wish it was even better!

Why Elo is good? Well, basically, it measures a theorical strenght of a player comparatively to each other. Based on the past results of this player on a given game, it tells you about the probability you can beat him comparing his Elo with yours.

More specifically, if you have both the same Elo, those odds are theorically 50%, the same as flipping a coin. I think most of you are aware of that.

But what if the Elo are not the same? In that case, there is a formula that links the difference of Elo D with the probability to win or lose p(D).
The exact formula for the Chess system is:


As far as I know, BGA currently uses that same formula.

For example, if you have Elo 240 and you opponent 160, the difference is +80 that gives you 61% chances you win against that player (and 39% to lose, if you neglect the possibility of a tie).
What if now you have Elo 440 and your opponent 360, well the difference is the same +80, so the odds are the same: 61% vs 39%.

You may wonder where I want to lead you with this. So let's talk about the players categories as defined in BGA. Here's the recap:


The first categories are special: below 100 it is not possible to loose any Elo and when you reach 100 you can't go below 100 anymore. These are minor differences with a true Elo system but I like them because it positivate a beginner progression and limits frustration after that.
Now, I will pinpoint the range (width) of the main categories starting with average:

Average player: 100 (Elo 100 to 199)
Good player: 100 (Elo 200 to 299)
Strong player: 200 (Elo 300 to 499)
Expert player: 200 (Elo 500 to 699)

See the problem coming? The range is not the same for all categories but I showed before that the difference of Elo shows the probability to win/lose, hence the relative strenght between players. A 250-Elo player has the same probability to win against an 350-Elo (approx. 36%) than a 350-Elo player against a 450-Elo. Yet in the second case both players belong in the same category (Strong players) whereas in the first they don't (good player vs strong). This seems a bit inconsistant to me.

As a casual player of Race for the Galaxy, I often experience this fact (and you may have seen it too). My level is around Elo 300 and I know I have little chances to win against a 450-ish player (30% chances to win or less). I does not seek to win at all cost but I wish I could set the matchmaking up to Elo 400 to have a fairer set-up).

Now I would like to have your opinions about this subject. Maybe I am wrong in my approach and this classification is fine.
But with the new design of achievements and Xp, it saw the occasion to truly discuss this subject and maybe get the BGA admins involved.

If you are interested, you can read my proposal I made some time ago to divide the strong player class into 300-399 and 400-499 and maybe vote for it if you wish: https://boardgamearena.com/bug?id=13021

Note that there is the same problem for Expert players but I am personally OK with this because there are much fewer players in this category. But maybe some skillful players would disagree because they feel aggrieved too

But I repeat, I already consider the ranking system very good and this are only two-cents to improve it just a bit more. After all, for games with weak audience, nobody cares for that level of detail (and the Elo rating system mechanically lose accuracy anyway because there is less diversity in player encounters). Some people can object that the Elo does not reflect the true world ranking for lots of reasons which have been debated elsewhere in the forum, but I consider main problems of that kind have been solved with the introduction of Arena. Anyway I personally don't care much about player ranking and just take the Elo for what it is: an indication of the general level of the players I play against.

Thanks a lot for reading, and looking forward to hear from you!
Take care of yourself and have fun

Woodruff


Fun fact : Elo must not be written in full uppercase "ELO" because it is not an acronym but the name of the guy who invented this ranking system! His name was Arpad Elo.

Sources:

• https://en.wikipedia.org/wiki/Elo_rating_system
• viewtopic.php?f=10&t=1422
• viewtopic.php?f=10&t=10974&hilit=player+categories+elo

[Sandoval]

I don't mean to intrude in your beautifully written post, just wanted to give my two cents with experience on this site: Was trying out Arena for the first time, was matched against a Silver player at Bronze 1. Lost the game, went to Bronze 0. Next time I queued up for Arena, I was matched against the current number 3 from the leaderboard. WTF? May I only play against other Bronze players, not the worldwide elite players, pretty please?

[Jest Phulin]

@Sandoval -- buried somewhere deep in the settings is an option to play Arena games only against people of your league. I don't remember exactly where it is, though.

@Woodruff -- I appreciate the thought that went into this, but I think there may be a better solution available, one that has been suggested by other people before. Instead of limiting tables based on arbitrary titles and rankings, be able to limit tables based on an Elo range. So (for example) if I have an Elo of 185 I can set the table to be from 160 to 210, +/- 25 from my current position. This will allow a reasonable chance of winning but not open the table to people that will have low chances of defeating me or a low chance of winning. I don't have to include all Average and Good players, just the ones that will be reasonably close to my skill level.

[Woodruff]

@Jest Phulin -- Indeed, your solution would be the best matchmaking system, though maybe less readable for new players? (not really ultimately a problem, just a thought). I would say:

• New fixed size classes (100-by-100):
  • More straightforward and user friendly (everybody see that good player vs strong player implies)
  • Maybe easier for the BGA team to implement
  • New classes could be linked to new achievements and Xp
  • A better matchmaking can be set than the current system (thought not ultimate as you pointed out: Elo 399 vs 401 still not in the same class...)
• Option to select a relative range of Elo in table settings:
  • Ultimate solution in terms of matchmaking
  • Can be opt-in or oup-out too if the player chooses to ignore that setting completely
  • A further knowledge of how Elo works would be required by the table admin players for that option to be set properly

Both solutions could be implemented thought, as they remain complementary: new classes for Xp and relative range for matchmaking.
If you have the link for the bug report you can post it here. I will then upvote it as well.


@Sandoval -- I advise you to start a new thread, since the leagues in Arena system are not based on Elo (except the last one, which relies on EAS, a parallel Elo computed using only Arena results).

[Woodruff]

Little up
Any thoughts about this?

[Taanz]

The Elo points is a good way to rate players in general. However, it is not a good system as it stands in the arena. Using Elo some players at the top stop playing and just rest on their points. For a competition, this is unfair IMHO. Imagine if Federer stopped playing with top ranking and stayed there forever. For a season that runs three months, there should be a sliding window time period where your points from older games get dropped forcing you to be active and keep playing. Thus, a champion is a more consistent player who played more games.

[sprockitz]

The Elo points is a good way to rate players in general. However, it is not a good system as it stands in the arena. Using Elo some players at the top stop playing and just rest on their points. For a competition, this is unfair IMHO. Imagine if Federer stopped playing with top ranking and stayed there forever. For a season that runs three months, there should be a sliding window time period where your points from older games get dropped forcing you to be active and keep playing. Thus, a champion is a more consistent player who played more games.

I'm against any sort of decay, especially in a short season, why are games played early on worth less than games played later? What I'd suggest the need for is #2 of my suggestions here: https://boardgamearena.com/bug?id=14455. Likely the issue is with short games where you can play a lot, get a lucky streak and then stop. K factor should be lower with such games so that you have to play more and can't get a big rating swing by being lucky in a handful of games (and also so you can't stockpile your wins in turn based mode)

Conversely if games are long the K factor should be larger, such that realistic ratings can be achieved in more similar amounts of time to the short games...right now top tier players can get to a stable rating in some games with less than 2 hours of playing while other games take 100+ hours.

[sprockitz]

Little up
Any thoughts about this?

I think this is a legitimate ask. I think it should apply across the board...reaching each additional 100 threshold is a big achievement and having something to commemorate it would be nice.

Adding levels to Master would be cool...like International Master, Grand Master

100-199 Average
200-299 Good
300-399 Strong
400-499 Distinguished
500-599 Expert
600-699 Master
700-799 International Master
800+ Grand Master

you can still have trophies for things above 800, but don't need a new name for them....only a few games where people can legitimately get to 900 anyways.

[Woodruff]

Hi sprockitz,

Good scale and nice naming!
Maybe be cautious with "International" Master since that depends of the game audience in the first place. Having a Elo > 700 is not necessaerely a guarantee that you are skilled at an international level (depending of the game, it could be far before that, further for some others). But I would not mind. The key point would be to have all classes be 100 Elo wide.

And I agree with you, it is a easy opportunity to unlock brand new challenging achievements (of course, Xp awarded with it should be decided accordingly, related to the difficulty it would be to reach that given level)

Hope the admins would consider do something about it!

Keep upvoting here! https://boardgamearena.com/bug?id=13021

[compte obelète]

well, if they create new levels into strong and expert, they should be named "Very Strong" for 400-500 elo and "pre-masters" or "apprentice-master" in 600-700 elo.