Why ELO doesn't suit to BGA

[demiurgsage]

ELO is rating system for 2 player skill-based games with low draw probability. And even for this kind of games it has issues. You can read this in wikipedia https://en.wikipedia.org/wiki/Elo_ratin ... cal_issues
The issues with implementation of ELO on BGA are connected with the fact that the games on BGA are multiplayer (3+ players) and luck-based.

Why ELO doesn't work in multiplayer games?
The calculation of ratings' changes after the game end are based on a player gain victory vs every player who ends the game lower than he. So ending last in 5-player games you gain 4 defeats. In fact it isn't fair. You played only 1 game. It's only one defeat. If you bet on the game $10, you'll loose only $10. To win a multiplayer game is more difficult than 2-player game, so the reward of the winner should increase with number of players. The fair system can be like a tournament when in games with a small number of players (2-4) the winner takes all, in games with more players some points to 2nd (3rd) places are awarded.

Why ELO doesn't work in luck-based games?
ELO predicts a player with 200 points higher rank will than other win 75% of games. Now we can take a game when a good player has at least 45% chance to win against the best possible player. This game is really exists. It's "Can't Stop". So for this game ELO doesn't reflect player level absolutely. It only reflects his luck and ability to choose opponents. A fair system should take into account the level of luck of the game.

Other issues of ELO
ELO prevents people from playing, especially in luck-based games. There are many masters on BGA that didn't play "ELO on" games for monthes.
ELO prevents high-ranking players from joining tournaments. They are afraid of playing with people with far less rating. ELO doesn't have the maximum of rating's change in one game.
ELO promotes opponent selection. Players try to play with convenient opponents to increase their rating.
ELO promotes table restrictions. You can often see players with rating about 1610 that restrict table for 1600+.
ELO doesn't show how good player is. In some games 1850 is very high level. It depends on number of active players and history of playing the game.

So my suggestion is to change the rating system on BGA to more suitable one.

[sourisdudesert]

hi,

Thanks for your message.

I agree with most of your conclusions. I just want to moderate some of these :

So ending last in 5-player games you gain 4 defeats. In fact it isn't fair.

This is not 100% true. The formula is the following :

• The system considers first you lost 4 duel against each of the opponents.
• The ELO lost is divided by 4 (number of opponents), so this is more or less equivalent to a single defeat.

=> This system was used quite a long time on BGA. The issue was that you used to win the same number of ELO point for a victory against 4 opponents than against a single one (the same for defeat). This is why we added the following :

• Finally, we increase the ELO variation by a factor of 1.5 on 3 players game, 2 on 4 players game, 2.5 on 5 players game, ... and so on ... until the ideal number of players for this game is reached.

Because of this last rules, playing with 3+ players is a little bit more rewarding/risky than 2 players games, but not too much, and it incite players to play the "ideal number of players" for a given game (when it exists).

[sourisdudesert]

About the other points :

As said I agree with most of your conclusion, but I think the problem is not the ELO, but the way we are using ELO here on BGA : we are using ELO as the main variable for rankings, and THIS is maybe what we should change.

We are thinking a new way to build leaderboards for each game, based on recent results and not on ELO. ELO would be keep as it is, in order to be able to know if a player is strong or not, but it wouldn't be used anymore for rankings/leaderboards.

This would solve most of the issues :

• it won't prevent Masters for playing / joining tournaments anymore.
• opponent selection would become useless.

In addition to this, we could make Good Player / Master / Expert depends on each game, so that it is relevant for each game.

[demiurgsage]

Thank you for a quick response.
I forgot to mention the other big problem with current system in multiplayer games. It promote people to defend 2nd, 3rd, ..., n-1 places, rather than trying to win the game. In many games it isn't good as they assume cooperation of players against the leasder.
Are you going to do something with it?

[sourisdudesert]

This is always better to defeat the first player, but in case - as second player - you feel this is impossible I see nothing's wrong to defend its second place, even in real life.

However, the new "ranking" system we have in mind only rewards the #1 player at a game. So it may solve this issue too.

[ollyfish2002]

Hello
BGA is a place to play games.
So I think you should rank by number of ended games. I am a great fan of Hanabi (full of red thumbs and abandonned games). I play less, only with players I know.
Personally, I don't care about my ELO, but some masters play ELO off to keep their rewards!
So give better rewards to players, not winners.
my 2p

[sourisdudesert]

We have in mind a system based on "Season points" like this :

_ when you beat an average player : +1 pt
_ when you beat a Good Player : + 10 pts
_ when you beat an Expert : + 100 pts
_ when you beat a Master : + 1000 pts
_ seasons points are reset to zero when each period (ex : year or season) begins.
_ at the end of the period, we attribute trophies based on who get the most points.

Note : only the first match against a player during a given period gives you season points, otherwise you'd just have to play 100+ games against the same player to be champion.

Note : ELO would remains as a simple "indicator", and to determine who is Master/Expert, ...

[demiurgsage]

According to the system you have in mind, what is the point for the Master to play with an Expert? Will "Season points" compensate chances to loose ELO and Master status? It looks like your system will encourage top players not to play each other. They will try to find a target that doesn't thretean them. Another good tactic will be trying to find players which have higher status than their real skill level.
Good that top players with this system will not be able not to play at all. But i am afraid table restrictions become more popular.

[senatorhung]

We have in mind a system based on "Season points" like this :

_ when you beat an average player : +1 pt
_ when you beat a Good Player : + 10 pts
_ when you beat an Expert : + 100 pts
_ when you beat a Master : + 1000 pts
_ seasons points are reset to zero when each period (ex : year or season) begins.
_ at the end of the period, we attribute trophies based on who get the most points.

Note : only the first match against a player during a given period gives you season points, otherwise you'd just have to play 100+ games against the same player to be champion.

Note : ELO would remains as a simple "indicator", and to determine who is Master/Expert, ...

i like some of these ideas for moving ranking away from the ELO score, and also for adjusting the ranges for Expert / Master based on each individual game. i like the idea of a 3-month season with trophies for spring, summer, fall, and winter.

that said, i think only having the first match between 2 players count is problematic, especially for games with sparser player populations. one master could beat another and then avoid playing that player again until the end of the season. or if the system only rewards positive points, they could just trade off a win each and none of their subsequent games will count for the seasonal trophy (whereas now the ELO scores will fluctuate as they see-saw).

i would be more inclined to assign season points based on a win-loss record over the season, modified by the skill level. not sure how that could be done, but someone who plays more matches should have some kind of leg up over someone who plays fewer.

[sourisdudesert]

Thanks a lot for your feedback.

what is the point for the Master to play with an Expert?

Indeed, he system encourages you to play with opponent with your level. Note that if you don't manage to beat any Master, you'd better beat some Expert before the end of the Season

It looks like your system will encourage top players not to play each other

This is the contrary : a Master who only play against weak players will never get enough points to become champion.

if the system only rewards positive points, they could just trade off a win each

Only fist match count so there is no possible trade off : only one of the two players will get the season points of the first win.

someone who plays more matches should have some kind of leg up over someone who plays fewer

The system only rewards positive points so there is no risk to lose Season points. Someone who plays more matches will have more chance to get points and become champion