🌿ELO System Updates

[h_illes]

Also, since there seems to be a ~30 day cutoff when counting winning streaks,

How do we know it?

I thought Elo stealing is only applied to the games played after December 12.

Personal observation by me and others. I quote one relevant comment below. The question is, if you had a win streak against a certain player before the changes, then win against them after the changes were applied, how many of the previous games count towards the win streak. There appears to be a ~30 day cutoff, so wins before that are not included in the win streak. (As a side note, BGA should have provided the players with exact details of the change, instead of players having to find out important aspects like this cutoff or the penalty % values through personal analysis.)

To be slightly more specific, from my conversations with others, it does seem there is some time horizon, I'm just not sure where it is. I *think* it may be looking at every game in the last 30 days or month, etc. Perhaps it goes back a bit farther, but I've seen some analysis from others that it doesn't go back for years.

[Stroom]

Do you have any proof of elo leakage or is it just speculation?

There is a penalty for every win-streak game. Before the changes, there was no penalty. That's the leakage. The actual penalty value is displayed per game, if that's what you need as proof.

But there is also the introduction of new points - if a new player joins a few games, loses and then stops playing.

So where is the balance? To me it seems that more points are added into the system than are taken out. But I don't have any proof either. Leak or no leak, it doesn't really matter in the grand scheme of things because you can still compare players based on who has more or who has less points. The background system is just more volatile. But the way you compare yourself to other players is still kind of similar, just by different numbers.

[RicardoRix]

By observation there does seem to be a net gain of ELO. If you look at the #1 players of a game with lots of players. Terra Mystica, Stone Age. Then it's slowly going up. This might not be permanent effect, but certainly it starts at 500/600ish, slowly creeping up towards 700 and 800.
So if you stop playing then essentially you are already losing ELO relatively speaking to the others that are playing. This would certainly be true if you think that there is an ELO net gain of new players arriving.

With the current new ELO decay, then if you stop playing then almost no matter what your ELO, you will end up at 200ELO. How is that useful?
ELO is meant to be a gauge of skill, not how much you play. I would now much rather see the Max ELO achieved figure next to people's profile.

The whole reason Arena exists is to address the issue with the ELO leader board. Now you have a platform to reward regular players, and ensure a regular competitive environment.
This idea should in no way leak into the ELO calculation. It's simply diluting what is otherwise already a fairly rough calculation. Now it just makes it a useless indicator.

[Siveure]

As far as I'm concerned, any negative sum or zero sum or positive sum nature of the elo ecosystems is basically irrelevant to the powerlevelling change being bad. My critique of it remains even if the K factor changes so that an individual game remains zero sum regardless of the cumulative wins between players.

My issue with the power levelling change relies on three facts:
In games with a small player pool or large player counts per game (or both), repeatedly playing the same person is common even if you are taking completely random opponents.
Very strong players have a very high winrate against the general population.
There are reduced elo rewards for beating the same player, so that matches against them effectively overestimate the elo differential.

Thus strong players in games with a small player pool or large player count, if they care about their elo, are incentivized to avoid playing particular players that they have recently played.

As far as I am aware there are three potential solutions for these players. If somebody can think of another, please let me know.

1) Don't care about your elo, play anyway and accept that the games you are getting are bad for your elo.
2) Avoid the players you are on winstreaks against (potentially by red thumbing them).
3) Just don't play any games at all.

I think a lot of strong players care about their elo (this is probably a lot of why they are strong), so the first solution is not viable for these players (or probably more accurately, they'll do this while complaining a LOT in this forum).
The second solution goes against a principle the admins have expressed a dislike to by making this change - you should not be curating your games to only play specific opponents.
The third solution seems bad for the site - having a change that encourages players not to play does not seem in BGA's interest.

A side note I thought of while writing this: I believe there must be a ranking difference threshold where, assuming no reset on winstreaks over time, you cannot gain any elo in the long term against the same player no matter how many games you play - either you run afoul of the power-levelling change, or your winrate is not high enough to sustain your elo because of the difference. I don't know exactly how to calculate this threshhold (beyond the fact that the simple approach - you need 6 wins vs 1 win to maintain elo without powerlevelling change - is an overestimate of the difference because of the incremental penalties applied before the sixth win), but at a guess it's about 300 elo difference.

[Tulivuori]

Sorry, I'm a bit out of the loop. So, if I'm at top of the leaderboard, I just play one match every quarter, so I won't be hit by any penalties, while anyone trying to catch will be hit by power leveling? Sounds excellent.

[h_illes]

But there is also the introduction of new points - if a new player joins a few games, loses and then stops playing.

So where is the balance? To me it seems that more points are added into the system than are taken out. But I don't have any proof either. Leak or no leak, it doesn't really matter in the grand scheme of things because you can still compare players based on who has more or who has less points. The background system is just more volatile. But the way you compare yourself to other players is still kind of similar, just by different numbers.

The balance depends highly on the specifics of the game, and also the size of the player base. Before the changes, this is how it worked:
(a) rating is added to the system due to the fact that players cannot lose rating below 100;
(b) rating is taken out of the system when players stop playing.

New players do NOT introduce rating directly to the system as they start at 0. They do introduce rating through effect (a), but so do other players who hover around 100 rating.

(a) and (b) did keep some balance in the long run. Of course, there can be changes in the balance; for example, when a game goes from beta to published, there is typically an influx of new players, many of whom will hover around 100 rating after a few games, making (a) more prominent than (b) for some time, and thus raising the rating of everyone slightly. However, (a) applies at the bottom, and (b) applies at every level, so at the top of the rankings, this has very little effect. But once the size of the player base settles and does not change drastically anymore, (a) and (b) will keep balance.

Now if leaking (e.g. win streak penalty) is introduced, it will push all players down to a level where the rating added at the bottom balances out the leakage in addition to (b). Note that by pushing all players down closer to 100, (a) will also be more prominent. Eventually there will be a new balance, that's true. But in this new balance, factors other than skill will also be included; for example, out of two players of equal skill, the one that plays more often will have a lower rating. That's an issue for multiple reasons, but an important issue is that it promotes playing less. Apart from that, the system as a whole will be weaker at measuring skill, because the same players will be crammed into a smaller scale (closer to the bottom). Also, convergence to the new balance might take a while, depending on how often the various players play, resulting in imbalances that might take months (maybe even years) until they eventually converge to the new dynamic balance.

Also, note that the stated intent of the power levelling fix was not to make things more 'volatile' or 'dynamic' at the top, but to stop cheaters from power levelling. We are actually discussing the side effects, none of which were intended (or at least not declared so).

[Stroom]

The balance depends highly on the specifics of the game, and also the size of the player base. Before the changes, this is how it worked:
(a) rating is added to the system due to the fact that players cannot lose rating below 100;
(b) rating is taken out of the system when players stop playing.

New players do NOT introduce rating directly to the system as they start at 0. They do introduce rating through effect (a), but so do other players who hover around 100 rating.

(a) and (b) did keep some balance in the long run. Of course, there can be changes in the balance; for example, when a game goes from beta to published, there is typically an influx of new players, many of whom will hover around 100 rating after a few games, making (a) more prominent than (b) for some time, and thus raising the rating of everyone slightly. However, (a) applies at the bottom, and (b) applies at every level, so at the top of the rankings, this has very little effect. But once the size of the player base settles and does not change drastically anymore, (a) and (b) will keep balance.

Now if leaking (e.g. win streak penalty) is introduced, it will push all players down to a level where the rating added at the bottom balances out the leakage in addition to (b). Note that by pushing all players down closer to 100, (a) will also be more prominent. Eventually there will be a new balance, that's true. But in this new balance, factors other than skill will also be included; for example, out of two players of equal skill, the one that plays more often will have a lower rating. That's an issue for multiple reasons, but an important issue is that it promotes playing less. Apart from that, the system as a whole will be weaker at measuring skill, because the same players will be crammed into a smaller scale (closer to the bottom). Also, convergence to the new balance might take a while, depending on how often the various players play, resulting in imbalances that might take months (maybe even years) until they eventually converge to the new dynamic balance.

Also, note that the stated intent of the power levelling fix was not to make things more 'volatile' or 'dynamic' at the top, but to stop cheaters from power levelling. We are actually discussing the side effects, none of which were intended (or at least not declared so).

IMO the only bad thing about this is that playing more can cause you to have a smaller score, which can be somewhat fixed by K-factor change instead of removing points from the winner. Otherwise, I don't see the changes as big of an issue.

Elo is a rough indicator of skill anyway, especially since the current system does not differentiate between game variants (different player count or specific expansions can change the game completely so it would actually need a separate elo calculation). So you would probably not get a completely honest indication when comparing one player to another anyway.

Why bother with nitpicking and just try it out in the long run... Even if the changes will take a long time to settle, I don't see how it would considerably change the current rankings order. And being so fixated on keeping your elo score does not really show to me that you actually got it by being a skilled player but rather you got it by gaming the system. And now that the system changes, you see that the only way to abuse the system is not to play... So if you'd rather not play then good riddance, TBH. Play for fun even if it hurts elo. Really.

[Siveure]

Okay I did more calculation on the threshold at which you cannot gain elo in the long run (or to put it another way, where resetting the penalty of a cumulative win streak cannot compensate you for losing a game against another person, even if you won the rest of your games against them forever).

The simple method of assuming you get a 6 win streak for every loss and that this perfectly balances puts it at ~300. this is a fairly large overestimate because streaks will not be this nice versus players 300 rating away, and i didn't factor in the intermediate rating losses.

Factoring in the intermediate rating losses (0 ,-3, -11, -25, -44, -69) you get 4.48 wins worth of rating for every 6 wins you get versus someone. That puts it closer to 260 rating difference for the point at which you can balance a single loss with a 6 game winstreak.

I understand why h_illes is here all the time, in century there are only 17 other players who he can potentially expect to be able to gain elo against in the long term. For everyone else if he plays enough he will either get a 6 game winstreak against that person and be unable to gain any more elo, or have lost elo overall.

This calculation also assumes that the variance of streaks lines up perfectly for the person higher on rating and that the person who is weaker does not get any upset multiple game winstreaks, which is bound to happen eventually in the long term, thus making the effective rating difference at which elo gain is impossible in the long term even lower.

[h_illes]

Elo is a rough indicator of skill anyway

Don't strawman. We are discussing a bunch of changes. Nobody claimed the old system was perfect, but it is better than the current one, so the changes are bad.

Why bother with nitpicking and just try it out in the long run

I prefer thinking in advance. Stuff like this can and shouid be analyzed without actually trying it out live.

And now that the system changes, you see that the only way to abuse the system is not to play... So if you'd rather not play then good riddance, TBH. Play for fun even if it hurts elo

We are discussing whether the changes are good or not. You essentially say 'OK, they made the system worse, but you should still keep playing,' which is fine, but this is not an argument for the changes. The changes are just bad and everyone would be better off if they reverted them.

[Patrick of the Isles]

Perhaps since Asmodee owns BGA we could contact leaders at Asmodee to tell them we are concerned that these changes will damage the site's value if they drive users away. Probably a long shot but if enough people contact them they might ask the BGA leadership to take a second look at the proposals? My experience with BGA staff has always been very positive, but for the sake of this site we enjoy so much we do need them to reconsider this particular endeavor.