Reconsider Arena Rankings System

[maveleo]

Yup. It isn’t even the elo system which is the biggest problem but the increased k factor in arena. In the op example the losing player still drops a significant amount but the rise is decreased and vice versa. It is designed this way on purpose to keep people playing.

I will be very surprised if anything will change at all.
Hope to be wrong here but the system is so clearly unbalanced, if nothing was done so far then maybe issue is not in the development of the system but in their developers

Yeah, maybe you are right that they just want it that way. It seems a bit dirty to make a ranking system too swingy so everybody can feel they can win at any moment and force them to keep playing. As said, for me it produces a counter effect of not wanting to play more Arena. I hope they can change it, at least reduce the k factor.

To me and you it may seem a bit dirty but to someone like agentus it isn't. At least you are aware of what's going on. Some popular games have as much as 30% of total games being played in arena. So even though a more fair system would be the right way to go if bga wanted arena to resemble a serious competition that isn't going to happen. Changing it would result in less games/traffic to the site.

Well, not really. If they're no Arena you still play with ELO or you play a tournament. Point is if you want to play you play. So they could even try and pull the famous finger out and find a way to stop piss people off

[ChiefPointThief]

Yeah, maybe you are right that they just want it that way. It seems a bit dirty to make a ranking system too swingy so everybody can feel they can win at any moment and force them to keep playing. As said, for me it produces a counter effect of not wanting to play more Arena. I hope they can change it, at least reduce the k factor.

To me and you it may seem a bit dirty but to someone like agentus it isn't. At least you are aware of what's going on. Some popular games have as much as 30% of total games being played in arena. So even though a more fair system would be the right way to go if bga wanted arena to resemble a serious competition that isn't going to happen. Changing it would result in less games/traffic to the site.

Well, not really. If they're no Arena you still play with ELO or you play a tournament. Point is if you want to play you play. So they could even try and pull the famous finger out and find a way to stop piss people off

I'm on your side about this issue but I'm just being realistic. You won't see threads from a weaker player upset because they just lost 5 games won 1 and gained aes but you constantly see the opposite. Everyone isn't pissed off. Only good players who play in games with a luck factor which is the minority. Good players will play more because we have to compensate for the bad system and weaker players will play more because they can lose a ridiculous amount of games like in the example posted and still bounce back.

I would love to see data and an actual study on numbers if they took all competitive aspects like arena and tournaments away. In arena you can play as many games as you want. In tournaments there is a cap and depending on the format if you aren't good you are bounced out early. I looked at the game history of arena leaders in 3 of the biggest games on this site. All the arena leaders play nothing but that game in arena several times a day. I guess you would have to ask them personally if they would still play that much if it weren't a competition but I'm sure several wouldn't. The leagues are actually pointless for people who intend to win arena but it gives weaker players goals as they only need a few consecutive wins to get to the next league.

For reference ark nova gets about 60,000 arena games a month. Now look at the upcoming ark nova tournaments. I bet the numbers don't come close to that.

[euklid314]

It could be the other way round, Chief.

Perhaps it is not the Arena leaders that decide to play a ton of games to get to the top.

But instead there is simply a large group of people that just prefer one luck-based game and play it massively. And of course one of them is in the end winning the Arena just by statistics - because you need a long winning streak to be on top and hardcore players simply have better chances.

I for instance like to play Cant Stop and Backgammon and I do not mind the Rankings fluctuating wildly. Of course a lower K-factor would be much better but there exists no fair ranking for luck-dependent games (in Backgammon one could use PR which depends on the quality of your decisions)...

[maveleo]

To me and you it may seem a bit dirty but to someone like agentus it isn't. At least you are aware of what's going on. Some popular games have as much as 30% of total games being played in arena. So even though a more fair system would be the right way to go if bga wanted arena to resemble a serious competition that isn't going to happen. Changing it would result in less games/traffic to the site.

Well, not really. If they're no Arena you still play with ELO or you play a tournament. Point is if you want to play you play. So they could even try and pull the famous finger out and find a way to stop piss people off

I'm on your side about this issue but I'm just being realistic. You won't see threads from a weaker player upset because they just lost 5 games won 1 and gained aes but you constantly see the opposite. Everyone isn't pissed off. Only good players who play in games with a luck factor which is the minority. Good players will play more because we have to compensate for the bad system and weaker players will play more because they can lose a ridiculous amount of games like in the example posted and still bounce back.

I would love to see data and an actual study on numbers if they took all competitive aspects like arena and tournaments away. In arena you can play as many games as you want. In tournaments there is a cap and depending on the format if you aren't good you are bounced out early. I looked at the game history of arena leaders in 3 of the biggest games on this site. All the arena leaders play nothing but that game in arena several times a day. I guess you would have to ask them personally if they would still play that much if it weren't a competition but I'm sure several wouldn't. The leagues are actually pointless for people who intend to win arena but it gives weaker players goals as they only need a few consecutive wins to get to the next league.

For reference ark nova gets about 60,000 arena games a month. Now look at the upcoming ark nova tournaments. I bet the numbers don't come close to that.

They can put a spinning wheel and play with that then... Arena is for the best players among a time period. Otherwise it's *

*Moderator edit: please refrain from swearing on BGA, this is a family-friendly platform.

[1000esqrtn1]

I for instance like to play Cant Stop and Backgammon and I do not mind the Rankings fluctuating wildly. Of course a lower K-factor would be much better but there exists no fair ranking for luck-dependent games (in Backgammon one could use PR which depends on the quality of your decisions)...

I would rephrase your sentence to "any game that is not 100% luck dependent can have a ranking that, with a substantial number of games played, will always benefit the best player".

In both of your examples, the games can be solved from a probabilistic point of view and there's always a move that will max out your probability to win. If you are able to always spot it while your opponent doesn't you should be above 50% of wins, which is a significant parameter and a fair ranking could reflect that.

[BackGa]

I have been playing for some time on BGA and last months I ventured to try competitive play as well and I joint various arena seasons.

After playing over hundreds games on Arena competitive play, I realized that the way the ranking works is quite deceiving and incentivizes no play. I found out that the ranking works as an ELO (once you reach elite league), but for games where there’s chance involved I think that’s a bad approach.

Let me argue my points. In most board games (except some full strategic like chess) some portion of games will be won by luck, where players skills won’t matter as the luck factor was to big to overcome. That’s fine, is how board games work. However when you have this unlucky games against opponents with a significant lower ranking than yours, you take a massive hit, and you need many wins to overcome that single loss.

In my experience, I think that players on top of rankings tend to play very few games, afraid of losing many points. The top 10 ranking looks pretty much the same during several days. Hence, once you start reaching high rankings (Top 100 - Top 50) you play most of your games against weaker players and here is where the problem begins. If you win those players, you get little reward, while if you lose, you drop many points. I have done some series of 3-4 wins and then one loss, my ranking didn’t vary as the loss compensated all wins. Should I play those games then? I understand that we are playing just for fun, but if we do a competition it shouldn’t have a negative effect to play those games, otherwise I prefer to switch to regular play.

That way of handling rankings can lead to unfair outcomes as well, as what makes you climb is a streak of wins, not an overall good result. For example let’s imagine 2 players on Elite league, A and B who play 100 games against random players. Player A wins 3 games and then loses one every time, while player B loses the first 80 games and then gets a 20 win streak to close it up. What is the outcome? Player A is likely to reach a ceiling. When he gets high in the rankings the 3 wins will likely not be enough to compensate that loss, considering that most of their games will be played against lower ranked opponents, hence the loss will deduct more points than the ones won. Meanwhile player B will drop to bottom of the rankings at first, but when it starts to win it will climb very fast as it plays better opponents. When he catches player A maybe the winning streak is not over yet and those final wins will be sufficient to surpass the ceiling that player A was never able to break. Is it fair that a player with 20% of wins ends up higher than one with 75%?

The current method bases everything (or at least most of it) on “the last winning streak”. It seems that this is all that matters to get a good ranking. Hence, when players get that good streak they stop playing, which in my opinion is a big problem. It clearly shouldn’t be the way to handle a 3 month competition.

I consider that there are 3 main parameters when valuing player performance during an arena season.

1 Games played - A player needs to play a significant number of games to consider his ranking legit. Currently this is somehow adressed by playing multiple leagues before reaching elite, but you can reach elite in around 10-15 games, which might be a little bit low.

2 Winning percentage - The overall perfomance through the whole season is a defining parameter as it tracks all of your games, not just the most recent ones.

3 Average ELO of opponents played - This is an important parameter as well, if someone tends to play easier games it should be penalized for that.

If the ranking is a formula combining these 3 factors I think we can obtain a more fair outcome. Players would be incentivized to play games as this is positive for the rankings. All games would be worth playing, as if you play a weaker opponent you have greater chances of winning while if you play a stronger opponent your avg ELO of opponents will increase. We would avoid the “no play” scenario as losses could be overcomed.

We could also have a more objective way to track our own progress, if we have a winning % of x one season, targetting to improve that for the next looks like a realistic goal.

In my opinion, this switch is a must on games with a significant size of its games (I consider 10-20% already significant) decided by luck. Thanks!

I fully agree and support your post, thanks

[qatman]

I think part of the issue is the expected winning percentage versus a much lower ranked player varies considerably by game type. For example in Chess it can be >99% but in something like Can't Stop the maximum expected should be something like 75% or 80% tops because there is so much luck involved. So in theory the Elo scores should be calibrated per game type. Perhaps they already are, but I have pretty much given up worrying about them after seeing the aforementioned ups and downs.

[ChiefPointThief]

I think part of the issue is the expected winning percentage versus a much lower ranked player varies considerably by game type. For example in Chess it can be >99% but in something like Can't Stop the maximum expected should be something like 75% or 80% tops because there is so much luck involved. So in theory the Elo scores should be calibrated per game type. Perhaps they already are, but I have pretty much given up worrying about them after seeing the aforementioned ups and downs.

Correct! The k factor in arena is double what it is in regular games. Considering arena is supposed to be the "competitive" format that is backwards. As mentioned the k factor should be adjusted considering the luck factor of the game. Expecting someone to maintain a 75% win percentage just to break even in arena in a game in which the #1 player on the site doesn't have a 75% win percentage doesn't make sense.

Also the elo scores are the same for all games and 75%-80% for a game like can't stop is still too high.

[Ceaseless]

The more luck in a game, the more luck is going to determine who has first at whatever arbitrary time has been set. The first place ranking matters less and less, with proper play it just gets shuffled around with more frequency.

In my experience, I think that players on top of rankings tend to play very few games, afraid of losing many points. The top 10 ranking looks pretty much the same during several days. Hence, once you start reaching high rankings (Top 100 - Top 50) you play most of your games against weaker players and here is where the problem begins. If you win those players, you get little reward, while if you lose, you drop many points. I have done some series of 3-4 wins and then one loss, my ranking didn’t vary as the loss compensated all wins. Should I play those games then? I understand that we are playing just for fun, but if we do a competition it shouldn’t have a negative effect to play those games, otherwise I prefer to switch to regular play.

Assuming the players are rated properly, there's no reason not to play those games. The reason you get so little vs them is because you're expected to beat them by that much. You should only expect to have your rating go up if you think you are underrated, (or I suppose if you happen to think your opponent is overrated). But the ideal outcome for players is no change on average, not without a change in their play level. Now, as rating gaps get further away, I think the rating's predictions get less accurate, and frankly I find the idea of players regularly playing rated games with such large gaps distasteful as a result. The higher rated players are either getting unfairly nabbed in the greater variance games or they're boosting in the lesser variance games. The main issue here is the lack of depth in the player pool for that game on BGA.  

That way of handling rankings can lead to unfair outcomes as well, as what makes you climb is a streak of wins, not an overall good result. For example let’s imagine 2 players on Elite league, A and B who play 100 games against random players. Player A wins 3 games and then loses one every time, while player B loses the first 80 games and then gets a 20 win streak to close it up. What is the outcome? Player A is likely to reach a ceiling. When he gets high in the rankings the 3 wins will likely not be enough to compensate that loss, considering that most of their games will be played against lower ranked opponents, hence the loss will deduct more points than the ones won. Meanwhile player B will drop to bottom of the rankings at first, but when it starts to win it will climb very fast as it plays better opponents. When he catches player A maybe the winning streak is not over yet and those final wins will be sufficient to surpass the ceiling that player A was never able to break. Is it fair that a player with 20% of wins ends up higher than one with 75%?

Yes, it is fair. Winning percentage is largely irrelevant to what rating you deserve, it just says how good your opposition was relative to you. So beat up a bunch of weak players and you get a high win rate, face quality opposition of your caliber for a 50/50 win rate, and get in over your head for a low win rate. If Player B broke Player A's ceiling, then it looks like they outperformed Player A. To break that ceiling they had to reach that same rating issue Player A had, and they got through it. If Player B's gameplay quality has made such a drastic improvement, I see no reason to judge them for their previously weaker play. Also, if Player A is losing rating for playing people, then it sounds like they weren't properly rated, as their rating isn't reflecting the high success expectations, or they're playing too many rated games with large rating gaps, which is also not a good idea. There would need to be issues specific to what happened with Player A/B, such as if someone thought something sketchy happened to cause those win rates, but the general case of an overall 20% win rate topping an overall 75% win rate in the ratings I have no theoretical problem with.

The current method bases everything (or at least most of it) on “the last winning streak”. It seems that this is all that matters to get a good ranking. Hence, when players get that good streak they stop playing, which in my opinion is a big problem. It clearly shouldn’t be the way to handle a 3 month competition.

That's if you're looking at high variance games. That's what you get for trying to refer to a best player in a game of luck. There's no real top player, the variance is going to shuffle players around too much. the top Arena player is just going to be a snapshot of whoever happened to be on top at the time. If you're worried about player activity levels stopping upon hitting a peak, then your best solution for Arena is probably to just shorten the season, as apparently the peak ratings don't need that long to hit in the first place.

I consider that there are 3 main parameters when valuing player performance during an arena season.

1 Games played - A player needs to play a significant number of games to consider his ranking legit. Currently this is somehow adressed by playing multiple leagues before reaching elite, but you can reach elite in around 10-15 games, which might be a little bit low.

2 Winning percentage - The overall perfomance through the whole season is a defining parameter as it tracks all of your games, not just the most recent ones.

3 Average ELO of opponents played - This is an important parameter as well, if someone tends to play easier games it should be penalized for that.

I agree with 1, there should be a minimum threshold of games played to get to an established rating rather than one still swinging from k factor, etc.

I think 2 is not only meaningless, but even detrimental. High win percentages generally mean you're not playing proper opposition. The ideal win percentage for players is roughly 50% after the initial set of games to determine rating have been smoothed out. That percentage should not meaningfully change even if your proper rating increases/decreases, as things should smooth out again upon hitting as proper rating. Also, if someone performs at a 300 level for half the season, then moves up to 400 level for the rest of the season, they're playing at a 400 level, there's no reason to penalize an improving player.

I think 3 is important, and this is something ratings do apply. In fact, that's one of the things you were complaining about, a player not moving up despite winning at a 3-1 rate is the penalty for facing such weak opposition being applied.

We could also have a more objective way to track our own progress, if we have a winning % of x one season, targetting to improve that for the next looks like a realistic goal.

In my opinion, this switch is a must on games with a significant size of its games (I consider 10-20% already significant) decided by luck. Thanks!

Trying to increase your winning percentage is not good. The way to do that is to play weaker opponents. In a proper rating system and player pool, improving performance doesn't improve your win rate, it improves your opponent quality. So improvement should be having a roughly 50% win rate vs 200s, but now you have a roughly 50% win rate vs 300s instead. A larger issue holding back game ratings on BGA is the player pool for a game not being deep enough to properly pair people looking for games.

[Ceaseless]

I think part of the issue is the expected winning percentage versus a much lower ranked player varies considerably by game type. For example in Chess it can be >99% but in something like Can't Stop the maximum expected should be something like 75% or 80% tops because there is so much luck involved. So in theory the Elo scores should be calibrated per game type. Perhaps they already are, but I have pretty much given up worrying about them after seeing the aforementioned ups and downs.

Correct! The k factor in arena is double what it is in regular games. Considering arena is supposed to be the "competitive" format that is backwards. As mentioned the k factor should be adjusted considering the luck factor of the game. Expecting someone to maintain a 75% win percentage just to break even in arena in a game in which the #1 player on the site doesn't have a 75% win percentage doesn't make sense.

Also the elo scores are the same for all games and 75%-80% for a game like can't stop is still too high.

I wouldn't use win percentage as an indicator to anything important, at least with respect to player skill. It needs more qualifiers to be meaningful. A high win rate means that player isn't getting good pairings. That said, the k factor concern in luck games is valid, but I think the issue is less with how large the k factor is and more with how soon they count the rating. I think one of the ideas listed in the OP could help, the idea to account for the number of games played. I'd suggest making ratings provisional until x number of games are played, x being however long that rating system needs to sort out the initial k factor issues. K factor is useful for players that don't want to slog through a bunch of bad players forever to get through the early ratings, so they increase/decrease faster to reach a proper rating so they can get back to a proper ~50% win rate sooner as they face proper opposition.