Reconsider Arena Rankings System

Discussions about BGA (all languages)
Forum rules
Warning: challenging a moderation in Forum = 10 days ban
More info & details about how to challenge a moderation: viewtopic.php?p=119756
Post Reply
1000esqrtn1
Posts: 24
Joined: 15 November 2023, 19:20

Reconsider Arena Rankings System

Post by 1000esqrtn1 »

I have been playing for some time on BGA and last months I ventured to try competitive play as well and I joint various arena seasons.

After playing over hundreds games on Arena competitive play, I realized that the way the ranking works is quite deceiving and incentivizes no play. I found out that the ranking works as an ELO (once you reach elite league), but for games where there’s chance involved I think that’s a bad approach.

Let me argue my points. In most board games (except some full strategic like chess) some portion of games will be won by luck, where players skills won’t matter as the luck factor was to big to overcome. That’s fine, is how board games work. However when you have this unlucky games against opponents with a significant lower ranking than yours, you take a massive hit, and you need many wins to overcome that single loss.

In my experience, I think that players on top of rankings tend to play very few games, afraid of losing many points. The top 10 ranking looks pretty much the same during several days. Hence, once you start reaching high rankings (Top 100 - Top 50) you play most of your games against weaker players and here is where the problem begins. If you win those players, you get little reward, while if you lose, you drop many points. I have done some series of 3-4 wins and then one loss, my ranking didn’t vary as the loss compensated all wins. Should I play those games then? I understand that we are playing just for fun, but if we do a competition it shouldn’t have a negative effect to play those games, otherwise I prefer to switch to regular play.

That way of handling rankings can lead to unfair outcomes as well, as what makes you climb is a streak of wins, not an overall good result. For example let’s imagine 2 players on Elite league, A and B who play 100 games against random players. Player A wins 3 games and then loses one every time, while player B loses the first 80 games and then gets a 20 win streak to close it up. What is the outcome? Player A is likely to reach a ceiling. When he gets high in the rankings the 3 wins will likely not be enough to compensate that loss, considering that most of their games will be played against lower ranked opponents, hence the loss will deduct more points than the ones won. Meanwhile player B will drop to bottom of the rankings at first, but when it starts to win it will climb very fast as it plays better opponents. When he catches player A maybe the winning streak is not over yet and those final wins will be sufficient to surpass the ceiling that player A was never able to break. Is it fair that a player with 20% of wins ends up higher than one with 75%?

The current method bases everything (or at least most of it) on “the last winning streak”. It seems that this is all that matters to get a good ranking. Hence, when players get that good streak they stop playing, which in my opinion is a big problem. It clearly shouldn’t be the way to handle a 3 month competition.

I consider that there are 3 main parameters when valuing player performance during an arena season.

1 Games played - A player needs to play a significant number of games to consider his ranking legit. Currently this is somehow adressed by playing multiple leagues before reaching elite, but you can reach elite in around 10-15 games, which might be a little bit low.

2 Winning percentage - The overall perfomance through the whole season is a defining parameter as it tracks all of your games, not just the most recent ones.

3 Average ELO of opponents played - This is an important parameter as well, if someone tends to play easier games it should be penalized for that.

If the ranking is a formula combining these 3 factors I think we can obtain a more fair outcome. Players would be incentivized to play games as this is positive for the rankings. All games would be worth playing, as if you play a weaker opponent you have greater chances of winning while if you play a stronger opponent your avg ELO of opponents will increase. We would avoid the “no play” scenario as losses could be overcomed.

We could also have a more objective way to track our own progress, if we have a winning % of x one season, targetting to improve that for the next looks like a realistic goal.

In my opinion, this switch is a must on games with a significant size of its games (I consider 10-20% already significant) decided by luck. Thanks!
User avatar
maveleo
Posts: 53
Joined: 23 May 2023, 08:22

Re: Reconsider Arena Rankings System

Post by maveleo »

You convinced me after the first two paragraphs.
I wish you good luck but I doubt any action will be taken on this regard.
nahikkara
Posts: 1
Joined: 25 February 2023, 22:17

Re: Reconsider Arena Rankings System

Post by nahikkara »

I completely agree.

I've been playing on Arena competitive games recently and the ranking system is really frustrating. When a player reaches a high ranking position, he just prefers not to play the game anymore, because if he plays against weaker players and has bad luck, the loss is huge. It isn't compensated by the wins the player gets, as it is needed 3-4 wins for the one single lost game. So, when winning streaks are needed to have a good ranking outcome, it feels deceiving and frustrating. It doesn't incentivize to continue playing competitive, as the player is afraid of losing against a lower ranked player and doesn't want to be extremely penalized for just a single loss.

I think a better ranking system would be really useful in terms of competition: it would increase the competition quality, would encourage players to continue playing even when a high level ranking is reached and, of course, would not be discouraging.
User avatar
h_illes
Posts: 381
Joined: 14 March 2022, 20:08

Re: Reconsider Arena Rankings System

Post by h_illes »

In any meaningful ranking system, a win streak will result in a higher rank, regardless if it was due to luck or skill. That's fine and it should work this way. The problem is that some players stop playing after a win streak to preserve their rank. That's a valid point, and indeed a drawback of the ELO ranking system.

That said, I think your points will not really solve that issue, or even if they help, they introduce other problems. Let me address your specific points.
1 Games played - implementing this as a single threshold will not do much to solve the issue, as aspiring players will first play enough games to meet the threshold, but after that, they are still free to stop at the end of a win streak. If it is implemented in a way that rewards more games played (with no cap on the number of games), then, instead of pure skill, the rating will at least partially measure the time each player dedicates to playing. I don't think that's what you want.
2 Winning percentage - this seems actually worse than the current ELO system. If someone gets many weaker opponents (maybe just due to randomness of Arena matching), their win percentage will be higher. For ELO rankings, this is balanced out by the fact that winning against weaker opponents results in a smaller rating increase than winning against stronger opponents.
3 Average ELO of opponents played - this idea is actually used, but mostly as secondary rankings for tie-breakers. Once again, considering this does not help in directly comparing the performance of players who win more against weaker opponents and players who win fewer games but against stronger opponents. ELO does a reasonably good job of making such situations comparable.

Any ideas for a change should be carefully analyzed before being put into production. To this end, I recommend to consider existing and proven rating systems first. ATP rankings and Glicko rating system are two examples; both discourage players from stopping. I recommend you look up both.
1000esqrtn1
Posts: 24
Joined: 15 November 2023, 19:20

Re: Reconsider Arena Rankings System

Post by 1000esqrtn1 »

Thanks for expressing your concerns and alternatives. I understand that every system has its flaws and I don't pretend to find a perfect magic solution, I aim to take a second look and reconsider our current system to one that may limit the most important flaws (specially incentivize no play). The current system may work for a short season (10-15 days) where the hottest period is awarded, but on a 3 month competition I would reconsider it.
h_illes wrote: 26 February 2024, 20:29 In any meaningful ranking system, a win streak will result in a higher rank, regardless if it was due to luck or skill. That's fine and it should work this way.
That's clear, however the winning streak shouldn't compensate all other results that happened during the tournament.
h_illes wrote: 26 February 2024, 20:29 Any ideas for a change should be carefully analyzed before being put into production. To this end, I recommend to consider existing and proven rating systems first. ATP rankings and Glicko rating system are two examples; both discourage players from stopping. I recommend you look up both.
I'm familiar with ATP rankings, and even though they work quite well, they are based on importance of tournaments and every tournament is played increasing difficulty game after game (using seeded players) so you can value more later stages of the tournament. That format is unrealistic to implement on a platform based on casual gameplay. I wasn't aware of Glicko rating, and it looks as an improvement of the ELO system using that inactivity parameter, but the core problem is the same than ELO. You are expected to win games against lower opponents so its hard to compensate unlucky losses. That's why I think that win % would tell more accurately which player is better. If everyone losses 30% of their games due luck, that's fine, but maybe a player is closer to 70% of wins, denoting that he is able to win almost all of his close games.

Winning percentages systems are not a random thing I invented. Important leagues as Premier League, NBA or NFL (to name a few) use a winning %/points based on wins system and the team being crowned (playoffs aside) is the one that performed better through the whole season (sometimes using strength of victory as a tiebreaker, as you said), not considering streaks or some specific games. Those leagues have the advantage of every team playing the same number of games, that's why they don't need to factor that parameter.
h_illes wrote: 26 February 2024, 20:29 That said, I think your points will not really solve that issue, or even if they help, they introduce other problems. Let me address your specific points.
1 Games played - implementing this as a single threshold will not do much to solve the issue, as aspiring players will first play enough games to meet the threshold, but after that, they are still free to stop at the end of a win streak. If it is implemented in a way that rewards more games played (with no cap on the number of games), then, instead of pure skill, the rating will at least partially measure the time each player dedicates to playing. I don't think that's what you want.
2 Winning percentage - this seems actually worse than the current ELO system. If someone gets many weaker opponents (maybe just due to randomness of Arena matching), their win percentage will be higher. For ELO rankings, this is balanced out by the fact that winning against weaker opponents results in a smaller rating increase than winning against stronger opponents.
My idea here is to establish different thresholds of a similar importance. For example (writing some numbers at random).

30 g + 80%w = 50 g + 75 %w = 70g + 70% win = 100 g + 65% win

After a minimum game required players can check their standings. If they keep playing they will be allowed to drop slightly on win %, as it is clear that sustaining a win % over a greater number of games is harder (and you will get more of those unavoidable losses). The players won't get anything by no play as their mark could be broken by a good player and they won't have the fear of losing many points on a single game. At the same time, a player that spends many time on the platform won't have an easier win, as sustaining a win % over that many games will be harder (if the numbers are well adjusted that's clear). Of course a player may play 300 games on the first days and already get a high win %, but if he does that, then he deserves to be champ as he proved many wins.

Thank you
User avatar
maveleo
Posts: 53
Joined: 23 May 2023, 08:22

Re: Reconsider Arena Rankings System

Post by maveleo »

h_illes wrote: 26 February 2024, 20:29 In any meaningful ranking system, a win streak will result in a higher rank, regardless if it was due to luck or skill. That's fine and it should work this way. The problem is that some players stop playing after a win streak to preserve their rank. That's a valid point, and indeed a drawback of the ELO ranking system.

That said, I think your points will not really solve that issue, or even if they help, they introduce other problems. Let me address your specific points.
1 Games played - implementing this as a single threshold will not do much to solve the issue, as aspiring players will first play enough games to meet the threshold, but after that, they are still free to stop at the end of a win streak. If it is implemented in a way that rewards more games played (with no cap on the number of games), then, instead of pure skill, the rating will at least partially measure the time each player dedicates to playing. I don't think that's what you want.
2 Winning percentage - this seems actually worse than the current ELO system. If someone gets many weaker opponents (maybe just due to randomness of Arena matching), their win percentage will be higher. For ELO rankings, this is balanced out by the fact that winning against weaker opponents results in a smaller rating increase than winning against stronger opponents.
3 Average ELO of opponents played - this idea is actually used, but mostly as secondary rankings for tie-breakers. Once again, considering this does not help in directly comparing the performance of players who win more against weaker opponents and players who win fewer games but against stronger opponents. ELO does a reasonably good job of making such situations comparable.

Any ideas for a change should be carefully analyzed before being put into production. To this end, I recommend to consider existing and proven rating systems first. ATP rankings and Glicko rating system are two examples; both discourage players from stopping. I recommend you look up both.
The analysis is not that simple and there will always be a margin of unpredictability.
In an healthy software platform you can test different ideas on a number of games over a period of time and see how it works. Is called canary release and prevents innovation paralysis. It would be nice to see that implemented in BGA with each of the suggestions you made and see the results before taking a call to spread the change
User avatar
ChiefPointThief
Posts: 745
Joined: 14 August 2020, 22:27

Re: Reconsider Arena Rankings System

Post by ChiefPointThief »

1000esqrtn1 wrote: 26 February 2024, 18:57 That way of handling rankings can lead to unfair outcomes as well, as what makes you climb is a streak of wins, not an overall good result. For example let’s imagine 2 players on Elite league, A and B who play 100 games against random players. Player A wins 3 games and then loses one every time, while player B loses the first 80 games and then gets a 20 win streak to close it up. What is the outcome? Player A is likely to reach a ceiling. When he gets high in the rankings the 3 wins will likely not be enough to compensate that loss, considering that most of their games will be played against lower ranked opponents, hence the loss will deduct more points than the ones won. Meanwhile player B will drop to bottom of the rankings at first, but when it starts to win it will climb very fast as it plays better opponents. When he catches player A maybe the winning streak is not over yet and those final wins will be sufficient to surpass the ceiling that player A was never able to break. Is it fair that a player with 20% of wins ends up higher than one with 75%?
I agree with your overall premise and have commented several times on this subject on other threads. From my own personal experience a few seasons ago I posted my stats and how I was losing arena to a player despite outperforming them in every category. My win percentage, amount of games, head to head match up, I had like 15 matches against the arena champion compared to the other player having 0, etc.

I thought your example was a little far fetched though so I did some rough math. If player z was to play players w/ a consistent rating of 1550 100x they could lose the 1st 80 games then win 20 and still finish with about 1620 aes :lol: I didn't do the math on if someone was to win their first 75 games vs players with a consistent rate of 1550 and then lose the rest but I believe they would fall below that 1620 mark which is interesting. I don't think their ceiling surpasses 2200 and they would lose in the upper 300s in the first 10 games and then they would lose another 300+ in the next 15 games. And it is very possible to win 75% of your games and fall below 1620 but that would also depend on your streaks and the elo of who you won/lost against.
User avatar
maveleo
Posts: 53
Joined: 23 May 2023, 08:22

Re: Reconsider Arena Rankings System

Post by maveleo »

In essence most of the players are in agreement but still no action is taken. Maybe using forums is not the right approach as only moderators are reading them so no one that count really something on the platform development side.
User avatar
Agentus
Posts: 14
Joined: 30 March 2014, 14:23

Re: Reconsider Arena Rankings System

Post by Agentus »

I do not really get the issue here. The fact that ELO allows one to be competitive even after a bad start into a season is a feature not a bug in my opinion. A rating based on win percentage would make the competition absolutely static after a good portion of the season is over with only those that haven't played much yet being able to make a run. This would also encourage players to run multiple accounts and definitely not encourage people to play more if a season is decided early by a good early run.
Also win percentage would encourage to select opponents and not play the best players. Even somehow correcting for ELO could not really help, additionally that would mean BGA would have to somehow invent a formula for this which is somehow deemed fair by everyone. A nigh impossible task - for this things like ELO, Trueskill etc. were invented by very smart people, getting a new system right is a gigantic task and not simple 'action taking'. I would really prefer them working on actual problems of the site.

Just to make a counterpoint and show that in fact not all players are in agreement.
User avatar
h_illes
Posts: 381
Joined: 14 March 2022, 20:08

Re: Reconsider Arena Rankings System

Post by h_illes »

I think a smaller K factor in ELO would somewhat mitigate some of these problems. A single loss to a lower rated player would not hurt that much, and win streaks would also be less prevalent.
Post Reply

Return to “Discussions”