Main goal and state of the ELO system

[FunKyMonx]

I've been thinking about the following questions lately: what should be the main goal of the ELO system and how can this goal best be achieved?
My best shot at this is question is: "The ELO should be able to predict the outcome/results of a game with one or more players of all skill levels".
In a game with 2 players this means prediciting the winner and in a game with more this means predicting the winner and ranking. A good ELO can even make calculations about the expected score difference. When the predictions are right, the ELO hardly changes, when they aren't, the ELO rating changes accordingly.

I'm very curious about your thoughts about what the ELO should or shouldn't do and what the opinions are of the current ELO system. Do the current ratings reflect the skills of all the player levels and is it able to differentiate well and soon enough (even for newbies)? Or do the odds sometimes feel like flipping a coin (50-50)?

[Grey Lord]

My opinion is that ELO main goal is predicting probability that one player win over other. From that point of view ELO works well.

Unfortunately many players ask from ELO impossible.
For example: Expected score difference - For that you would need to have different system not only for each game but also for each game variant and each possible number of players.
Or the same distribution of ELO ranking for all games - games with big luck influence cannot have high ELO players because noone is able, in long time, win 9 out of 10 games agains average oponents.

ELO is itteration proces - you need 10 or 20 games played to have ELO that match your skill. Newbie can see the skills of other players from ELO but his own ELO is underestimated (starts on 0) but he will grow very fast to his real ELO.
Before beginers started with ELO 1500 and you can loose points under that value. It was better for estimating skill of newbies but probably to depresive for bad players who fell soon under 1400 or lower.

[FunKyMonx]

My experiences with the ELO system are that it works well for good players, but not for the other half. This is mainly because of deleting the left half of the normal distribution. Because of this most of the players seem to be within the 100-200 range. The following image shows wat I mean.

This is the current ELO distribution with a peak in the first category. The normal distribution does a better job in spreading in skill levels.

Half_normal_pdf.svg.png (36.4 KiB) Viewed 1361 times

The normal ELO distribution in chess with an average of 1000 and a minimum of 0 does a better job in spreading/differentiating in skill levels. The distribution would look something like this:

This how the ELO could be @BGA

phpbCyapv.png (12.22 KiB) Viewed 1361 times

I agree that new players should start with 0 and get a ranking according to their skill. So they see their rankings rise to their current skill level. This is good for motivation. The current system is not good for motivation because I bounce between 100 and 150 in a lot of games. There is too much variety in skill level in this rating range. I'd rather grow from the 5th percentile of players to the 10th than bouncing around in the current average category.

[JPreto]

For a ELO system to work it needs 2 basic things:

1) variety of matches (matches with different players, of all levels)

2) many matches (the more, the better)

But here in BGA, I see people deliberatly trying to flaw ELO by:

a) playing with the same players (either friends or duplicate accounts).

b) 300+ ELO playing with aprentice players (the odds of losing are so low that is just a "boring" +2 or +3 ELO gain per match).

c) playing only 20-30 matches (where ELO is boosted) and then never play again and keeping the rank (rating decay is good but it takes time).

ARENA mode should be good for this, randomly atributing players to games, the issue is that some players are already quitting when the match in ARENA is not of their pleasing (https://boardgamearena.com/forum/viewto ... =3&t=19116).

[FunKyMonx]

I can imagine rating decay is a real thing when you'd study the player data. The longer you haven't played a game the worse you get at it. I wonder what the relation is between player inactivity duration and chances of winning. They must be able to calculate it and build it into the rating system.

But to getter grasp on the real skill level distribution of players I've added the suggestion to add percentiles to each ELO rating. This would clarify a great deal about the mysterious ELO number. It would help If you'd vote. Here is the link: https://boardgamearena.com/bug?id=33042

[dschingis27]

While I generally agree that having percentiles can be a useful information, we will need to answer which users we formally include in the player base of a game to calculate the percentiles. Everyone who played the game once, or at least 5 times, one game in the last month, or everyone who one at least once? Or sth else? I would personally argue for all players who won at least once since it is a marker that you understood the game.

For some games, the percentile can be very volatile (as Elo itself tends to be for luck-heavy games). So if you play Backgammon or Yahtzee, you could go from Top10% down to Top40% with very few games played.

Btw, something similar was already suggested here: https://boardgamearena.com/bug?id=14455

The 2 main problems with the current Elo system are addressed in the above suggestion: First, it doesn't reflect skill accurately over different player counts. Second, the volatility (or let's call it speed) of Elo change is not adjusted for game type. For games like Can't Stop or Backgammon, your Elo can go up and down in a wide range within only one hour of play. The Elo value in these games does not so much reflect your true skill but more so your luck in the last 10 games played. The easy solution would be to adjust the K factor in the Elo formula that determines the speed of Elo change.

------

A much deeper and more general problem with Elo is the inherent assumption of multiplicativity. What do I mean by this? For instance if Player A is 3:1 favorite against player B, and Player B is 2:1 favorite against Player C, then it follows in the Elo model that Player A is 6:1 favorite against Player C. For many games, this is absolutely not true! In fact, the "sharpness" or functional form that relates skill to winrate is very different for different games. For some games (e.g. Chess), minor skill gains can make a big difference in winrate against weaker players but not necessarily so much against equally skilled players, while for other games (Backgammon), big skill differences are not really reflected in winrate at all.

[T72on1]

Not really from a mathematical point of view, but my feeling is that ELO actually quite accurately tells how skilled a player is. The example of lucky games like yahtzee and Backgammon has been given here, but isn't that the point of luck based games like those, that there is less skill involved? So even if your ELO fluctuates much harder in such a game, it works as intended. I know that in the games that I play, which hardly involve any luck (Dungeon Twister and Battle of Lits come to mind), you can definitely tell from the ELO rating of your opponent how much of a chance you stand to win. I know that both are 2 player games, so don't apply to one of the issues mentioned here, which is ELO in higher player count games. But even in those I feel that, if I really take it to get to know the game and improve myself, I'll have no trouble getting at least above 200 ELO, most of the times 300+ too.

That isn't to say that the current ELO formula cannot be improved upon. But I'll leave that to the more mathematically inspired people around here .