Win probability calculation incorrect

[h_illes]

There is a win probability calculation displayed in the small popup window shown when you hover above the players' ratings (after a game is finished). That calculation is incorrect since it is based on the regular ELO formula which is no longer in place due to the ELO changes in Dec. 2023. That means that a wrong win probability calculation was displayed for literally every game played since then, meaning millions of games.

Moreover, there is no easy way to fix the win probabilty formula, since due to the changes, the new rating is no longer a clean indicator of skill; instead, it takes into account playing patterns like how often does a player play or even opponent selection (none of which would matter for regular ELO).

My suggestion would be to change the current ELO calculation with win streak penalties and loss mitigation to...
- either just the regular ELO formula used before Dec. 2023, or
- replace the win streak penalties and loss mitigation by a system where the K factor is reduced for frequent opponents, as that way, the win probability calculation remains intact. (I am happy to explain further if necessary.)

I also created a bug report for this. If you would like to see this bug addressed, please upvote it.

https://boardgamearena.com/bug?id=127835

[Mathew5000]

I also created a bug report for this. If you would like to see this bug addressed, please upvote it.

https://boardgamearena.com/bug?id=127835

What happened to the bug report? As of right now it has 26 green thumbs but it's not displaying properly, and it has been changed to yellow (i.e. a "suggestion" rather than "bug").

[h_illes]

I don't know why it was changed to a suggestion. I assume it was some BGA admin, but they are simply mistaken. Displaying incorrect information is a bug, period.

Edit: whoever changed it also took out all the details and took away the option to add comments. Whoever did it - care to explain yourself here?

[nik592]

As someone who voted for it, I got an email regarding the status change. The comment was it was intended behaviour so changed to a suggestion.

I suspect it may have been assumed that it was meaning the new way of calculating ELO is incorrect, rather than the calculation on screen (though I feel your report was perfectly clear about this). In other words, it was seen as a bug report version of all the comments in the other thread requesting the ELO changes be reverted. Which also would somewhat indicate that BGA admin are paying attention to the ongoing discussion there, and deliberately choosing not to respond. Not for the first time on a contentious issue. Not sure that's the best way of managing things either.

[h_illes]

Ah, I see. Thanks.

Alright, quoting time again.

We value your input and are committed to ensuring a fair and balanced ranking system for all players.

So, back to the bug report. Incorrect information is displayed. That is a bug by definition. I understand it is a side effect of other changes you made, but it is still incorrect, and it affects almost every game played on this site (excluding corner cases like solo games and whatnot).

Can we agree on the above? Waiting for your acknowledgment.

[Silene]

I don't like the elo-changes either, but tbh I think your effort on win probability is a bit exaggerated.

You're right that the percentage is more off after the changes than before. But you kinda pretend that before the changes, the percentages were accurate and only after the changes they're not and that's not the case. It was always just a rough estimate that did not consider many aspects that were also affecting it:

* elo in general is barely ever accurate as it tends to go up and down. If a player on their UP plays a player in their DOWN, the win-% is not accurate either (you could argue it's a good thing as it makes players get back to their place quicker, but still the percentage is wrong)
* new players often have a terribly false elo as their possible prior knowledge of the game isn't considered and neither their talent on board-games in general

Win-percentage was never "correct". So if you want to paint it's not being accurate as a bug, you should argue for changing the text into something like "win-percentage is probably somewhere around this but it could be far off:". Only fighting one of many reasons why it isn't accurate and ignoring the others seem a bit beneath the point to me.

Changing elo-system to what it was before will not lead to correct win-percentages being shown.

[h_illes]

Changing elo-system to what it was before will not lead to correct win-percentages being shown.

Of course, the rating of a player is subject to some random fluctuations, so you could argue that it's never really precise. But the idea for regular ELO is that assuming the ratings reflect the true skill of the players, the win probability percentage shown is correct, and those two players will maintain that win probability percentage in the long run. And if the current ratings do not reflect the true skills of the players, they will be updated, eventually converging to correct.

The modified ELO does not have this property. For example, it could easily happen that for players A and B, player A wins 30% of their games steadily, yet the win probability percentage shown fluctuates between 35% and 40%. I mean, in this situation, any displayed value is meaningless.

[Jellby]

Incorrect information is displayed. That is a bug by definition.

I'm sorry, but that's not the definition of a bug. Only if it's unintentional it approaches to the definition, but it still isn't.

[h_illes]

Incorrect information is displayed. That is a bug by definition.

I'm sorry, but that's not the definition of a bug. Only if it's unintentional it approaches to the definition, but it still isn't.

I am pretty sure it was unintentional in the sense that it is the side effect of an intentional change that noone even considered. In any case, this fits the wikipedia definition for 'Bug (Engineering)': In engineering, a bug is a design defect in an engineered system that causes an undesired result.

Edit: they somewhat fixed the bug report, so you can still vote on it, people!

[Jellby]

I'm pretty sure that when they modified the elo change from consecutive victories, they intended the resulting elo to be used for estimating win probabilities (maybe not as the primary objective, but still they intended it to behave as "raw" elo). Whether that's a good or bad decision is a different matter.