Win probability calculation incorrect

[h_illes]

Based on their track record of decisions related to the ELO changes, I think you overestimate how much thought they put into this. In all likeliness, this aspect of the ELO changes was not considered at all. The win probability formula was already in place, and it still gives some result, so who cares if it is meaningless or not? They don't seem to, but I do, hence the bug report.

[DoctorFianchetto]

I suppose this explains why in Captain Flip I am frequently gaining small amounts of ELO (sometimes 0 points) for winning, while losing as much as I normally would (if not more) from losing. I am almost always playing people lower-rated than me, but obviously something isn't right.

Two examples:
https://boardgamearena.com/table?table=526759265
https://boardgamearena.com/table?table=526755753

The calculation mentions something about consecutive wins - so if I win enough games in a row I stop gaining ELO? What kind of sense does that make?

[h_illes]

The calculation mentions something about consecutive wins - so if I win enough games in a row I stop gaining ELO? What kind of sense does that make?

This is the main thread: https://boardgamearena.com/forum/viewto ... 96#p182096

In short: yes, you get less and less ELO for the longer win streak you have. After 6 wins, you would stop gaining ELO. It was implemented to hinder cheating by power-levelling (playing and winning against the same dummy account repeatedly).

The win streak count is only checked for games in the last 30 days, so one way to avoid the win streak penalty is to not play against the same opponent for 30 days, then the win streak count resets. I know that's a poor 'solution', but that's all we have right now. Also, this is all based on player observations, we never got an official detailed description of how the win streak penalty works.

[DoctorFianchetto]

The calculation mentions something about consecutive wins - so if I win enough games in a row I stop gaining ELO? What kind of sense does that make?

The win streak count is only checked for games in the last 30 days, so one way to avoid the win streak penalty is to not play against the same opponent for 30 days, then the win streak count resets. I know that's a poor 'solution', but that's all we have right now. Also, this is all based on player observations, we never got an official detailed description of how the win streak penalty works.

Thank you for the explanation.

You're saying that the win streak is calculated per opponent, rather than per game played? As in, winning six games in a row against six different opponents will have no adverse affect on ELO gain? That makes more sense, although it's a case where a cheating deterrent has a negative impact on a greater number of legitimate players. Especially for games where the player pool is small, so repeated matchups are very likely.

[h_illes]

You're saying that the win streak is calculated per opponent, rather than per game played? As in, winning six games in a row against six different opponents will have no adverse affect on ELO gain?

Yes, yes.

...although it's a case where a cheating deterrent has a negative impact on a greater number of legitimate players. Especially for games where the player pool is small, so repeated matchups are very likely.

Yes, small player pool games has been one of the main arguments against the win streak penalty. Another is Arena, where matchmaking is blind. A third argument is that this system rewards playing less frequently; on a site where subscriptions are for a given time period rather than number of games played, such a change is very unethical (at the very least).

[Jellby]

So you're suggesting that the win probability should be computed based on the "true" elo, and not on the "capped" elo where consecutive wins give reduced (or zero) change? What would be the expected effect? Suppose I'm a highly skilled player, but my elo has ben capped due to consecutive wins. Should my win probability be actually higher than currently displayed? Wouldn't that mean that my wins will cause lower elo gains and my defeats higher elo loss?

Anyway, if you start keeping one elo for display and another elo for win probability, the next step is to not count losses due to time expulsion for the win probability, and then not to count losses because of bad luck, and then not to count losses because of rookie mistakes...

[h_illes]

Anyway, if you start keeping one elo for display and another elo for win probability, the next step is to not count losses due to time expulsion for the win probability, and then not to count losses because of bad luck, and then not to count losses because of rookie mistakes...

Time expulsion, bad luck, and rookie mistakes all factor into the performance of a player, and regular ELO rating takes those into account accordingly. Assuming they continue to play in the future with those factors present to the same extent as before, their rating remains stationary. But, for example, if they improve their time management and stop getting time expulsions, their rating will naturally increase - just like it would increase if they improve their skill at the specific game.

So you're suggesting that the win probability should be computed based on the "true" elo, and not on the "capped" elo where consecutive wins give reduced (or zero) change?

I never suggested that, and I agree it is a bad idea. I had two suggestions in the original post:
- return to the regular ELO formula (without the win streak modifiers), or
- replace the win streak penalties and loss mitigation by a system where the K factor is reduced for frequent opponents, as that way, the win probability calculation remains intact.
I stand by either of those, and I am happy to explain further if necessary.

[Jellby]

OK, so then your complaint has nothing to do with the win probability itself, but with the fact that the displayed elo is not the "true" elo, because it adds some spurious conditions to the original formula. I can agree with that, if whatever is being used for the rankings does not obey the original elo formula, it should not be called elo. As for the win probability, let's just call it "base score" and forget about it. It's not a win probability any more than XP points are any indication of experience.

[Tisaac]

I am bit surprised that no-one has brought the fact that ELO was only designed for 2p games without luck involved anyway so the whole win probability thing does not make sense for many games and for any player count > 2, even with the original formula.

I would agree with Jellby and say that it should just be renamed to make it more sound like a base score or whatever, but I dont see that as a solid argument in favor of returning to the old ELO computation, really.

[h_illes]

OK, so then your complaint has nothing to do with the win probability itself, but with the fact that the displayed elo is not the "true" elo, because it adds some spurious conditions to the original formula. I can agree with that, if whatever is being used for the rankings does not obey the original elo formula, it should not be called elo. As for the win probability, let's just call it "base score" and forget about it. It's not a win probability any more than XP points are any indication of experience.

Using a rating system other than ELO (e.g. Glicko) would be fine, but the current rating system has serious issues: it takes into account playing patterns outside skill; it incentivizes playing less often; it incentivizes avoiding certain opponents - just to name a few. That said, those are not reportable bugs, but the incorrect win probability calculation is, even though it is just a side effect. So yes, my issue is essentially with the current rating system, and I am trying to solicit a response through the bug report system - but I do that by reporting an actual bug resulting from the source problem.

Your suggestion of removing the term 'win probability' would be a cheap solution, I hope you agree on that. Also, this whole situation of me (and many others in the ELO changes thread) trying to give feedback but falling on deaf ears from BGA is sad because there would be a proper solution (reducing the K factor for frequent opponents) that would achieve an effect similar to the win streak penalty, without the issues mentioned above.

I am bit surprised that no-one has brought the fact that ELO was only designed for 2p games without luck involved anyway so the whole win probability thing does not make sense for many games and for any player count > 2, even with the original formula.

I would agree with Jellby and say that it should just be renamed to make it more sound like a base score or whatever, but I dont see that as a solid argument in favor of returning to the old ELO computation, really.

ELO rating works fine for games with luck involved. For 3+ player games, it's not perfect, but it works reasonably well.

All that said, the current system is essentially a strictly worse version of ELO with a bunch of issues the original ELO does not have. That's not what you want to have, especially if there are better solutions available (including solutions different from returning to the original ELO).