🌿ELO System Updates

Board Game Arena Official announcements
User avatar
franko
Posts: 11
Joined: 20 August 2013, 21:31

Re: 🌿ELO System Updates

Post by franko »

Stroom wrote: 18 December 2023, 23:31 Your original claim is pointless. Other players are playing with the same constraints as well. So overall, it will even out and the rankings are still comparable. Elo is not a score that is supposed to increase indefinitely. It is a very rough approximation of your skill. Just play the games and let it rise as your own skill level increases... not by gaming the system by avoiding players who are always beating you or playing only weaker opponents who you are more likely to beat.
The issue with your statement is that the changes as proposed and implemented in this change actually break the incentives to play against really anyone.

Again, for me the game I can reference is Kingdom Builder (as I'm one of the top active players in that community). If I look at page 1 of ELO, I'd be winning in a "streak" if I beat player #2 (hb2014), #4 (Nothim), #6 (Demosthenes88 -- currently I have 3 consecutive wins), #7 (majorossk), #9 (Urho -- currently I have 6 consecutive wins)... I could probably go deeper down in the ratings (as I'm currently 19th as I had a couple of losses when climbing back up in Arena rating and then this ELO change happened and I don't see why I'd play for ELO given this new system).

What this shows to me is that I'd actually be better off playing off against lower ranked players because (1) they'd be easier to beat, and (2) I would potentially get more ELO given the consecutive win penalties (especially against Urho, unlucky for them to have lost the last 6 games against me, but luck is part of the game). The fact that I should actively avoid two players with >600 ELO because a win would give me less benefit than beating a ~300 ELO player shows how broken this change is.

Looking at lower ranked players, I certainly have some people that I have an overall high % of wins against, but due to the fact that there is some luck in the game I happen to play I obviously am not expected to just always win. Pulling up at least one name I can easily recall: https://boardgamearena.com/gamestats?pl ... me_id=1225 -- I'm at 35 wins out of 51 (~68%) games, but in our lifetime of games I've LOST 67 ELO. I'm even less incentivized to play against weaker players because that loss will be higher given the change as implemented (since I'd gain less ELO during win streaks, and then the inevitable losses will take away the same amount of ELO).

So now I'm left wondering who I'm meant to play against. I should avoid stronger players as once I hit a win streak the EV is negative (there's no reason to expect a win streak to keep going with an opponent that also has a 70-80% win rate, and I'd earn less per win), and I should avoid weaker players because even there the EV is pretty low.

This really is a fix for nothing as far as I can tell. The only way you can abuse the pre-existing system to get infinite ELO is to play against someone with 100 or less ELO, as they can't lose ELO (and you'd keep gaining... at some small number <1 per game). Okay, that should be easy to address in a way that doesn't break all these other cases. Make the new rule "you can only gain ELO against someone with a rating of <=100 once per day (or week?)". Done.
User avatar
gric chick
Posts: 6
Joined: 13 August 2021, 23:01

Re: 🌿ELO System Updates

Post by gric chick »

Power levelling
This is a terrible solution to the problem and penalizes the wrong players. There are many people who regularly play at the same time of the day… after work for example, and as a result are often paired with the same other players in Arena, people they don’t even know in real life. You are penalizing a large percentage of honest players in an attempt to punish a few cheaters who will simply find another way… for example create even more accounts and alternate games etc. This is the worst of the changes and should be reversed immediately.

ELO Decay
ELO should reflect proficiency regardless of how often you play. For example, there must be an increase in activity on BGA during the winter months… should these players really lose ELO when summer arrives and they play less or not at all for a few months? The argument that their ELO should degrade because they will be ‘rusty’ makes no sense to me… if that is the case they will lose a few games until they regain their edge, and those loses will balance their ELO accordingly.

Red Thumbs
This change is also unfortunate and worsens an already sad situation that exists in the longer 1 hour plus games, where people quit far too often with laughable consequences. Now people will have to play with even more quitters which lessens enjoyment of the game and goes against the spirit of boardgaming. If this must be done, can an option to raise the karma threshold to 90-95% be implemented?
User avatar
h_illes
Posts: 381
Joined: 14 March 2022, 20:08

Re: 🌿ELO System Updates

Post by h_illes »

Stroom wrote: 18 December 2023, 23:31 Your original claim is pointless. Other players are playing with the same constraints as well. So overall, it will even out and the rankings are still comparable. Elo is not a score that is supposed to increase indefinitely. It is a very rough approximation of your skill. Just play the games and let it rise as your own skill level increases... not by gaming the system by avoiding players who are always beating you or playing only weaker opponents who you are more likely to beat.
Let me try to explain. Assume two players play against each other regularly. (They might play against others as well, we'll get to that in a moment, but let's just focus on their games against each other for now.) Their skill level remains the same, with no improvement or degradation. Then their ELO should remain roughly the same, right? Only fluctuating a bit due to the randomness of recent results. Well, with the current ELO power levelling fix, both their ELO will go down in the long run (as games are negative-sum instead of zero-sum). And if they play against other players as well, their ELO will go down even more, since this is true for every pairing. Actually, with the power levelling fix, the total ELO of all player bases in all games will go down simply from playing. That's just wrong.
User avatar
Silene
Posts: 980
Joined: 23 October 2013, 17:50

Re: 🌿ELO System Updates

Post by Silene »

The only counter effect is from bumping into 100. Those 2 opposite forces will eventually even out. But imo way too late. It's good to have expert and master ranks in some games. Forcing everyone down with the new penalties will probably result in an elo range that's almost meaningless, where the random fluctuations are bigger than skill differences.
Check out the Allround-League: https://boardgamearena.com/group?id=7870115 --> a league where you have matches of random games vs. other players in your group. Season 9 registration oct. '25.
User avatar
Siveure
Posts: 25
Joined: 24 January 2021, 16:01

Re: 🌿ELO System Updates

Post by Siveure »

I have doubt about a lot of these changes. They seem targeted at players behaving badly because they overly care about rating. I believe the only thing that could stop people who care about their rating from caring about their rating is to abolish rating, at which point you could lose the players who care about rating from the site entirely. Given that I doubt this is a desired result or will ever be implemented, there will likely be a perpetual problem of players who care overmuch about their rating behaving badly.

There is also a larger issue that many individual games and communities, and even individual players, will receive systems applied generally in very different ways. For instance: the thresholds of what each 100 rating means is very different in different games. There are some games where ~500 is the pinnacle (can't stop), some where ~700 is the pinnacle (agricola), and some where 900+ is the pinnacle (azul), and what this means for the rankings underneath is quite different on a game to game basis. What an arena season's settings look like, or how much it changes seasonally, varies greatly from game to game. Even the length of a game has a massive impact on how it deals with general systems.

With the above in mind, i would like to address the individual changes:

Team play:
I have no experience with team play games on this website (which does matter), but this change sounds good. Although if I understand correctly, should this change not make the players on a team have different elo wins/losses against the opponents, they just do not win/lose against each other?

Cooperative play:
I also have no experience with cooperative play games on this website. However, my understanding of what is described fails to understand how hiding ranks to this limited a degree will stop players from being elitist and competitive about ranks. You might not be able to tell what a player's exact rank is anymore, but you will still know roughly what their rank is, with a range of variation on that. While it may no longer be possible to confidently claim you are superior to another player based on ranking, it will still be possible to unconfidently claim this, especially falsely, so this does not seem like a clear improvement.

Power leveling:
This is a clear case of the cure being worse than the disease. Firstly, I'd like to point out that multi-accounting is a problem above and beyond directly transferring rating from one account to another. I can think of three uses for multiple accounts to get a very high rating that don't rely on transferring rating from one account to another one and they have all happened on this site. (I will not explain what they are as I don't want to let people know how to cheat). This new repeated win restriction system applies everywhere, to everything. It is also deflationary - the sum total of ratings across a game can now be smaller after the game than before, as the win-streak penalty only blocks gains and doesn't prevent losses in a loss-streak. Even were this fixed, it would not be a good change.

It is not infrequent in small communities, or multiplayer games (or both), to queue the same opponent repeatedly, and any winstreak (even of 2) in this is flagged as being power levelling. For players at the top of the ratings, it is very likely that you will be winning against a lot of the general playerbase many times in a row.
On a quick count of my hit stats I have played 101 players 2 or more times in Agricola since the start of this arena season in 155 games (and another 109 players once). Not all of these games were arena or even ranked (as there is currently a new expansion in alpha requiring friendly games), but it's still a reasonable sample. I have a 100% hit rate vs 31 players with 2 games matched, 8 players with 3 games matched, 6 players with 4 games matched and 1 player with 5 games matched. (I also have 0% hit rate vs 3 players with 2 games matched in that time period, one of which is a massive upset as the player is currently ~370 rating lower than me). So that's 46/101 players that i played multiple times in the past two months where i definitely currently am on a multiple game winstreak against them, ignoring most of the players i actually play the most because i did not maintain 100% hit rates vs anyone past 5 games (and I'm also probably on a winstreak vs some of them, or they are versus me). There are currently 639 players listed across all ranked categories for the arena season - assuming all my opponents played arena i've played a third of the entire arena population, and about a sixth of it multiple times. I also have used the game filters specifically to avoid playing players in bronze (although it's not particularly effective in turn-based games) so it's in effect 382 potential opponents - meaning that I've probably played more than half of my potential opponents, more than a quarter of them multiple times, and likely have an incentive to not play against more than a ninth of my potential opponents. If I queued three games vs completely random opponents i am expected to queue vs someone I've beaten multiple times before. This is not getting into the issue that queueing games will not give you random opponents, due to timezones and other factors.
There's also an irony that the next two points are players not wanting to play with specific other players (and now there is a new rating-related reason to do this beyond abstract questions of your belief in your winrate vs specific players not matching the system's belief in your winrate vs those players - it can now reach a point where you can literally only lose rating by playing another player, so to preserve your rating you should try to not play them) and players wanting to not play to maintain their rank (when the new system is deflationary, and thus you are expected to lose rating if you play a game vs fairly ranked opponents, so you should not play to maintain your rank).

ELO Decay:
I don't entirely understand what this is for. There are two things that already do this: hiding inactive players, and arena seasons. Given that anyone who hasn't played rated games in a while isn't going to show up on that game's leaderboard and is thus already incentivized to keep playing (and if they're not it's probably because they've lost interest in the game), and that the decay is probably not frequent enough to meaningfully influence players (one game in three months is not much more than zero), this change only really affects players who start playing again after the decay happens. It should be noted that the rating system is not merely aspirational, it's also a fairly effective tool with which you can ensure a good quality, roughly evenly matched game. A game between two 700 rated players or two 300 rated players will probably be better than a game between a 700 rated player and a 300 rated player. Making large artificial changes to people's ratings degrades the value of rating as a way to judge how good a game will be. If it must be done, increase the uncertainty of a player's rating if they don't play for a long time.

The issue of decay not being frequent enough is awkward to solve - while you can increase the frequency enough that those players at the top of the rankings would have to play a noticeable amount, it will still be less than they probably played beforehand (because the most highly rated players probably tend to at some point be some of the most active players), and it will start decaying players who are playing a normal amount. Variability in games matters a lot here - if you decay once a week this affects turn based games that last a day a lot differently to games that last a month.

Red thumbs:
The red thumb system absolutely could be abused to keep players out of games for rating based reasons - I have done it myself to filter out weak player wildcards who I feel are guaranteed to kingmake randomly in multiplayer games (before I started just filtering out all weak players with the level filter). It is also used as it is the most effective form of ensuring good behaviour from players - you filter out the badly behaving players so they don't get in your games. I am not clear on what is going to happen, but this sounds like it could easily lead to a cure being worse than the disease.

I would also note that if you want to avoid a specific couple of players in arena you can do so by looking at whether they're trying to queue for a game or not (or in a game, or offline). This is far too much effort to do for many people, but if you're trying to avoid only the top couple of players it is very simple.

Overall, the changes seem to mostly not have considered the changing incentives of the new systems that are being implemented, and assume that players will choose to behave in a way that would not have required us to make changes in the first place while in some cases introducing incentives to behave badly. In addition, these changes add deflationary pressure on the rating making playing games less exciting for players who are interested in rating.
User avatar
ChiefPointThief
Posts: 745
Joined: 14 August 2020, 22:27

Re: 🌿ELO System Updates

Post by ChiefPointThief »

Stroom wrote: 18 December 2023, 21:38
6element wrote: 18 December 2023, 20:30 After six games against a particular opponent I've got 0. Next game I had to expel them. I played with them a lot (and liked it), but less than 1% of my total games, my win rate against them is 64% (so I'm a bit stronger, but not drastically). Now BGA essentially forces me to stop playing against them. This is not good at all.

So, I have to have a list of people I play against a lot, and expel them from my tables? Not fun :(

BGA guys, this feature is well below shippable quality, seriously. Stop gathering feedback, revert this.
You can still play against them... Elo is a meaningless metric. You don't have to play only to gain it :D
You were one of the first people to comment against the changes to coops when it was announced. You clearly care about elo as it pertains to you (hanabi) but you think others shouldn't care about the changes that affected them. Why would anyone want to participate in a game that at best all they can do is break even? If bga took a 5 out of every hanabi game and only allowed you to score a max of 29pts (meaning only allowing you to break even and not actually win anything) would you still participate?
Stroom
Posts: 428
Joined: 14 July 2016, 19:10

Re: 🌿ELO System Updates

Post by Stroom »

ChiefPointThief wrote: 19 December 2023, 03:58
Stroom wrote: 18 December 2023, 21:38
6element wrote: 18 December 2023, 20:30 After six games against a particular opponent I've got 0. Next game I had to expel them. I played with them a lot (and liked it), but less than 1% of my total games, my win rate against them is 64% (so I'm a bit stronger, but not drastically). Now BGA essentially forces me to stop playing against them. This is not good at all.

So, I have to have a list of people I play against a lot, and expel them from my tables? Not fun :(

BGA guys, this feature is well below shippable quality, seriously. Stop gathering feedback, revert this.
You can still play against them... Elo is a meaningless metric. You don't have to play only to gain it :D
You were one of the first people to comment against the changes to coops when it was announced. You clearly care about elo as it pertains to you (hanabi) but you think others shouldn't care about the changes that affected them. Why would anyone want to participate in a game that at best all they can do is break even? If bga took a 5 out of every hanabi game and only allowed you to score a max of 29pts (meaning only allowing you to break even and not actually win anything) would you still participate?
These are two completely different things. I care about elo being displayed because then I can better assume the skill level of a player. But the actual number of who is good or bad, changes over time. But the calculations changes I do not care about. If everyone is playing by the same rules, over time everyone's elo will still be comparable. Maybe today I would think that 900 elo is an indication of bearable game partner but tomorrow it might be 800. Doesn't matter i nthe grand scheme of things. If power play and elo decay affect everyone equally, it won't really matter.

The actual implementation isn't really explained well but people get very defensive without any mathematical analysis, whether it actually becomes worse or not... Not by individual games but rather by the overall system. If you only play games with a handful of players, then your elo calculation does not become comparable to the whole elo system. You are meant to play with a variety of players.

If you have 2 groups completely separated from each other and they play the game differently, one group's top player and another group's top player might end up with the same elo... but you can not compare these elos with one another as one group could just be playing on a different level. But for individual elo calculations, they might have the same score. Now if those 2 top players would play, you might see that actually there is a skill difference and the player in one group easily defeats the other group's top player. Similar things happen when you are used to playing in real life and suddenly play online - much more variety in games and more developed strategies.
Stroom
Posts: 428
Joined: 14 July 2016, 19:10

Re: 🌿ELO System Updates

Post by Stroom »

franko wrote: 19 December 2023, 01:17
Stroom wrote: 18 December 2023, 23:31 Your original claim is pointless. Other players are playing with the same constraints as well. So overall, it will even out and the rankings are still comparable. Elo is not a score that is supposed to increase indefinitely. It is a very rough approximation of your skill. Just play the games and let it rise as your own skill level increases... not by gaming the system by avoiding players who are always beating you or playing only weaker opponents who you are more likely to beat.
The issue with your statement is that the changes as proposed and implemented in this change actually break the incentives to play against really anyone.

Again, for me the game I can reference is Kingdom Builder (as I'm one of the top active players in that community). If I look at page 1 of ELO, I'd be winning in a "streak" if I beat player #2 (hb2014), #4 (Nothim), #6 (Demosthenes88 -- currently I have 3 consecutive wins), #7 (majorossk), #9 (Urho -- currently I have 6 consecutive wins)... I could probably go deeper down in the ratings (as I'm currently 19th as I had a couple of losses when climbing back up in Arena rating and then this ELO change happened and I don't see why I'd play for ELO given this new system).

What this shows to me is that I'd actually be better off playing off against lower ranked players because (1) they'd be easier to beat, and (2) I would potentially get more ELO given the consecutive win penalties (especially against Urho, unlucky for them to have lost the last 6 games against me, but luck is part of the game). The fact that I should actively avoid two players with >600 ELO because a win would give me less benefit than beating a ~300 ELO player shows how broken this change is.

Looking at lower ranked players, I certainly have some people that I have an overall high % of wins against, but due to the fact that there is some luck in the game I happen to play I obviously am not expected to just always win. Pulling up at least one name I can easily recall: https://boardgamearena.com/gamestats?pl ... me_id=1225 -- I'm at 35 wins out of 51 (~68%) games, but in our lifetime of games I've LOST 67 ELO. I'm even less incentivized to play against weaker players because that loss will be higher given the change as implemented (since I'd gain less ELO during win streaks, and then the inevitable losses will take away the same amount of ELO).

So now I'm left wondering who I'm meant to play against. I should avoid stronger players as once I hit a win streak the EV is negative (there's no reason to expect a win streak to keep going with an opponent that also has a 70-80% win rate, and I'd earn less per win), and I should avoid weaker players because even there the EV is pretty low.

This really is a fix for nothing as far as I can tell. The only way you can abuse the pre-existing system to get infinite ELO is to play against someone with 100 or less ELO, as they can't lose ELO (and you'd keep gaining... at some small number <1 per game). Okay, that should be easy to address in a way that doesn't break all these other cases. Make the new rule "you can only gain ELO against someone with a rating of <=100 once per day (or week?)". Done.
Your incentive to play is to... have fun. That's all there is. Whether your elo rises or decreases is meaningless. You are supposed to keep playing in order for elo to actually display your skill level compared to other active players. Elo is not supposed to become stagnant when you don't play. Elo is constantly inflating and deflating due to changes in the system. A score of 600 is different from a score of 600 5 years ago. Even if player skills were the same. So it makes sense that it decays if you don't play for a long time or don't play with different players.

Many players here complain that elo changes make scores lower... as if elo is supposed to increase indefinitely. But it would be the same system for everyone. So it is evened out.

The rest of your comment just explains how elo works... Same as now. And random games are random anyway. Over time, the elo diff still will even out. Show actual mathematical proof that it would be worse rather than bandwagoning with everyone :D
Last edited by Stroom on 19 December 2023, 06:50, edited 1 time in total.
User avatar
Lumin_S
Posts: 165
Joined: 09 October 2018, 00:51

Re: 🌿ELO System Updates

Post by Lumin_S »

The "power levelling" ELO adjustments are absolutely ridiculous, especially in multiplayer games. I'm getting 3rd place out of 4 in games and yet am still penalized for being a "repeat winner" against 4th place? That feels bad, even though ELO is just a number, and seems completely unnecessary.

Mathematically, I can't believe this was implemented. The ELO deflation is so harsh and impacts each game's community ratings like wildfire. And I don't know how lowering the ELO of strong players so intensely will actually benefit the lower rated players - strong players didn't lose skill, but now get matched up against a wider range skill levels when, guess what, everyone benefits from playing people of similar skill level. The usefulness of ELO to match people (its original purpose, right?) is steeply declining.

For my ratings' sake, it is best to not play anything that I've done reasonably well in. Pretty wild stuff.
Stroom
Posts: 428
Joined: 14 July 2016, 19:10

Re: 🌿ELO System Updates

Post by Stroom »

h_illes wrote: 19 December 2023, 01:52
Stroom wrote: 18 December 2023, 23:31 Your original claim is pointless. Other players are playing with the same constraints as well. So overall, it will even out and the rankings are still comparable. Elo is not a score that is supposed to increase indefinitely. It is a very rough approximation of your skill. Just play the games and let it rise as your own skill level increases... not by gaming the system by avoiding players who are always beating you or playing only weaker opponents who you are more likely to beat.
Let me try to explain. Assume two players play against each other regularly. (They might play against others as well, we'll get to that in a moment, but let's just focus on their games against each other for now.) Their skill level remains the same, with no improvement or degradation. Then their ELO should remain roughly the same, right? Only fluctuating a bit due to the randomness of recent results. Well, with the current ELO power levelling fix, both their ELO will go down in the long run (as games are negative-sum instead of zero-sum). And if they play against other players as well, their ELO will go down even more, since this is true for every pairing. Actually, with the power levelling fix, the total ELO of all player bases in all games will go down simply from playing. That's just wrong.
Elo is meaningful only if you play with a wide variety of players. A group of players only playing amongst themselves and their elo going up or down, is not comparable to the whole population.
Post Reply

Return to “Official announcements [en]”