Statistically Improbable Win Rates

[ResonanceSM]

At what point does a statistically improbable win rate become its own evidence for "something" going on?

Not necessarily saying someone's cheating, although there's evidence that they did in the past. But they went from *allegedly* losing on purpose and kingmaking, to winning close to 80% of their games in a game where experts only win like 60%. The winning itself seems like evidence to show that their losing in the past was intentional, but that's not "proof." I guess I could still report it.

Statistically improbable, but I can't report someone for "being too good" or "winning too much."

And honestly, not sure how they're doing it. So maybe they're not. I have warned some other players to look at this player's history to make their own judgements, and I think a group of high-level players blocking them may have helped them find softer games as well, so who knows.

Still. But I guess this is more of a vent, but the question still exists... if someone is winning at a rate that is statistically improbable, over a statistically significant number of games, when does that become its own evidence? It seems like a lot of effort to be a "fake" champion, but I suppose people have done more for less. But there's also no "reason" to report them for it.

[Jellby]

To evaluate the statistical significance, you should take into account not only the number of games and winning rate, but also the number and strength of opponents and possibly other game settings. And compute the winning rate yourself, don't rely on what BGA reports as winning rate.

[Ceaseless]

At what point does a statistically improbable win rate become its own evidence for "something" going on?

Not necessarily saying someone's cheating, although there's evidence that they did in the past. But they went from *allegedly* losing on purpose and kingmaking, to winning close to 80% of their games in a game where experts only win like 60%. The winning itself seems like evidence to show that their losing in the past was intentional, but that's not "proof." I guess I could still report it.

Statistically improbable, but I can't report someone for "being too good" or "winning too much."

And honestly, not sure how they're doing it. So maybe they're not. I have warned some other players to look at this player's history to make their own judgements, and I think a group of high-level players blocking them may have helped them find softer games as well, so who knows.

Still. But I guess this is more of a vent, but the question still exists... if someone is winning at a rate that is statistically improbable, over a statistically significant number of games, when does that become its own evidence? It seems like a lot of effort to be a "fake" champion, but I suppose people have done more for less. But there's also no "reason" to report them for it.

If you don't bother to look at the facts and put something together, "higher win rate" could be something completely innocuous or even positive and you wouldn't even know. Sure, there are several ways a higher win rate is part of the case for cheating, and they're not all the same either, different facts can lead to different methods. On the other hand, a higher win rate could also just show a player has gotten better at the game or has weaker tables, among other reasons. That you don't even sound sure about something as basic as "maybe they have easier tables that are improving their win rate" isn't a good sign.

[dschingis27]

If this goes on for >300 games and they reach an Elo quite above everyone else's in a game where it is implausible, that means something is going on. I mean 80% win rate isn't fishy on its own but it would be if it was in a game like Can't Stop or Lucky Numbers where such win rates can't be maintaned over longer periods.

While you can't name the player in forums, I think naming the game title is no problem. It would help in assessing how easy a cheater could be detected.

[tbhp]

How would they be cheating? Do they have access to some hidden information? Or are they using some kind of AI? But if the AI is able to win 80% of the time, why is it unimaginable that a human player would have the same winrate? Why can't it just be that they have figured out the game in a way other players haven't?

In order to call a winrate improbable, first you need to have an understanding of the game that allows you to prove why it's improbable. Otherwise, you don't actually know if there exists a way to play the game that makes said winrate probable.

[FrankJones]

At what point does a statistically improbable win rate become its own evidence for "something" going on?

This depends on how you are defining the terms you used.

[Fuchur]

At what point does a statistically improbable win rate become its own evidence for "something" going on?

You will ALWAYS have the risque of a "type one error" or "false positif" result, no matter how long the winning streak, how high the win rate is.

IF the underlying statistical process is well know you can estimate the probability of a false accusation but it seems to me that you have no idea whether or not something fishy is going on. As you don't give any information ... for example about the game in question, the win rate versus the "normal" win rate and so one, nobody can help you with the calculations.