Suggestions to improve point allocation

[Travis Hall]

For Hanabi, I would sometimes even want to filter by 800 or 900 too. Some 700s still keep playing at 500-600 level.

That’s true. But being able to filter 700+ is a lot better than nothing at all.

[Malo77]

I've been thinking about this point a lot lately. I love Hanabi, but I have walked away playing it on BGA because most people I met are too perfectionist and not wanting to lose points, and wants to start dictating how other people played the game to the point it is no longer fun.

My suggestion:
Let's not make players lose points by not attaining a perfect score, but instead, like Just One or Bandido on BGA, reward very little amount of points depending on how they scored on the firework display - such as 5 points for a perfect score, and less thereafter. That way, people would be more welcoming to newcomers and mistakes made instead of being the incredibly toxic mess that this game has made the perfectionist players.

No. ELO is about measuring skill. Giving points and not taking them away is not a good indication of a good player. Only about how much you have played.

Here as Yukihimeko points out: scoring in different collab games is not done the same way in every collab game. This is annoying and shows that BGA should rethink and revamp the whole so called "ELO" system for collab games and apply probably the same to all collab games. At the very start ELO was designed for chess and by chess players so is mre fi for 1v1 with a winner.

Unfortunately i do not know who there on BGA could think and do this kind of thing ? this seems a bit pointless to talk about in in forums if no admin/no dev reads it and take it into consideration.
Maybee a BGA bug/improvement report exists about that ?

Maybee a system like:
- at avery game start: every player bet 1% of his ELO point into the game pot
- if the game is lost the points are lost (and BGA "keeps" thems)
- if the game is won: take the pot point x RATIO, split and and give them back to the winning players.
- the RATIO is automatically adjusted by BGA ... if the stats show that an hanabi game is won 33% of the time .. 1 time over 3 ... the RATIO multiplier would be x2 (3 games-1win=2x) .. either each players gets back his points x2 .. or split them another way ...
- this way the total ELO points of players is about the same, without points growing without limit

[Stroom]

I've been thinking about this point a lot lately. I love Hanabi, but I have walked away playing it on BGA because most people I met are too perfectionist and not wanting to lose points, and wants to start dictating how other people played the game to the point it is no longer fun.

My suggestion:
Let's not make players lose points by not attaining a perfect score, but instead, like Just One or Bandido on BGA, reward very little amount of points depending on how they scored on the firework display - such as 5 points for a perfect score, and less thereafter. That way, people would be more welcoming to newcomers and mistakes made instead of being the incredibly toxic mess that this game has made the perfectionist players.

No. ELO is about measuring skill. Giving points and not taking them away is not a good indication of a good player. Only about how much you have played.

Here as Yukihimeko points out: scoring in different collab games is not done the same way in every collab game. This is annoying and shows that BGA should rethink and revamp the whole so called "ELO" system for collab games and apply probably the same to all collab games. At the very start ELO was designed for chess and by chess players so is mre fi for 1v1 with a winner.

Unfortunately i do not know who there on BGA could think and do this kind of thing ? this seems a bit pointless to talk about in in forums if no admin/no dev reads it and take it into consideration.
Maybee a BGA bug/improvement report exists about that ?

Maybee a system like:
- at avery game start: every player bet 1% of his ELO point into the game pot
- if the game is lost the points are lost (and BGA "keeps" thems)
- if the game is won: take the pot point x RATIO, split and and give them back to the winning players.
- the RATIO is automatically adjusted by BGA ... if the stats show that an hanabi game is won 33% of the time .. 1 time over 3 ... the RATIO multiplier would be x2 (3 games-1win=2x) .. either each players gets back his points x2 .. or split them another way ...
- this way the total ELO points of players is about the same, without points growing without limit

You describe something that is very similar to elo calculation. The current system is sufficient as is.

When starting a game, elo system already basically wagers a portion of your elo. If you lose, you lose the elo. If you win, you gain elo. The ratio is adjusted based on the average skill vs an imaginary opponent. Since the "strength" of the imaginary opponent is unchanged then everyone is playing vs the same "opponent" and the calculation is fair (enough).

When players play on different game settings, the elo score will be somewhat different anyway. Base vs multicolors vs black powder vs flamboyants. You can't calculate one effective score that takes all of that into account. And it would be a pain in the ass to balance it. Meanwhile you get a very rough estimate with a known system that already works.

[HanabiGrandMaster]

I think with the current adjustment for cooperative games (abandoned games count as a loss), the ELO system can be used to depict people's skill in the game. I think most cooperative games have a very high chance for "master" players to win the game. So I would propose that ELO reflects a permillage that the player wins a cooperative game.
Example:
1000 ELO points = 100.0% chance;
300 ELO points = 30.0% chance;
50 ELO points = 0.50% chance. (etc)

This way, you can stay in the ELO-zones that BGA uses as default (no more players with 3,000 ELO points in Bandido). Since the highest level is "masters" from 700+ ELO points, the only assumption you need to make is that (every) cooperative game has a 70% chance or higher that the perfect strategy will win the game. I think that's not an unreasonable assumption.

Side note:
I would recommend that the TEAM ELO is not calculated as the average, but using the "least squares" method (smallest sum of distances, because in team games it's often not possible for one player to carry the team to victory).
For example, if three players play a coop game and their ELO's are {100, 200, 600} then I don't think there's a 30% chance for victory, but a 21%


In relation to Hanabi, I think the scoring table SHOULD be modified, despite a lot of players already dropping out of master league since the adjustment. I would propose the following:

Scoring table Hanabi
Five colour games

• If TEAM ELO =< 100 then 25 points gives +5ELO, non-perfect scores get +0ELO
• If TEAM ELO >100 then 25 points gives +0ELO, non-perfect score gets -5.88ELO

Six colour games
This includes the standard + blackpowder, tricky, and avalanche of colours

• For 30 points give 58.8*{1000 -TEAM ELO}/1000 points
• For 29 points give -58.8*TEAM ELO/1000 points
• For less than 29 points give -117.6*TEAM ELO/1000 points
• For 0 points (loss/abandon) give -294.1*TEAM ELO/1000 points

Difficult games
This is the variant with once of each multicolour, which is ca. 10% less likely to get perfect score

• For 30 points give 58.8*{1000 -TEAM ELO}/1000 points
• For 29 points give -58.8*TEAM ELO/1100 points
• For less than 29 points give -117.6*TEAM ELO/1100 points
• For 0 points (loss/abandon) give -294.1*TEAM ELO/1000 points

Seven colour games
This includes the variants trick + blackpowder, and avalance of colours + blackpowder, which are ca. 20% less likely to get perfect score

• For 30 points give 58.8*{1000 -TEAM ELO}/1000 points
• For 29 points give -58.8*TEAM ELO/1200 points
• For less than 29 points give -117.6*TEAM ELO/1200 points
• For 0 points (loss/abandon) give -294.1*TEAM ELO/1000 points

Side note:
I used {100/17=}5.88 as a default unit, to keep ELO points from converging to whole numbers. And this makes the last 17 games representative of the skill.

[Stroom]

These numbers seem to be taken out of thin air. And I do not have much confidence in it actually helping at all. The current elo calculation is good as is. You just need to play more.

[HanabiGrandMaster]

Your feedback isn't very specific. But I will try to elaborate.

I've tried to use the ELO calculation that is described in the BGA wiki (here: https://boardgamearena.com/faq?anchor=faq_account_elo)

I assume p(D), the probability of winning, to be 100%.
I use a K, the elasticity factor, of 58.8 (comes from the assumption that the last 17 games are representative of your abilities).

I wanted to make a distinction between a lost game with 1 point missing, multiple points missing, and a failed game. I wanted to punish missing multiple points by doubling the ELO loss, and punish a failed game by quintupling the ELO loss.

Difficult games are ~10% less likely to get perfect score. There is a 1/55 chance that m1 is at the bottom of the deck, resulting in a theoretical maximum of 26 points in 2% of the decks. The same goes for m2, m3, and m4; so 4/55 = ~7.5%. I added 2.5% for the difficulty of having 5 less tokens in the game (~20% less clue-tokens).

Seven colour games are ~20% less likely to get perfect score, because of limited hand space. From personal experience I found that 2-player variant with only 5 cards in hand (official variant) gets ca. 60% of perfect score with flamboyants. And 5-player variant with 4 cards in hand (official variant) gets perfect score pretty much all the time (100%), so I went with the middle ground of 80%.

If you still think these numbers are too "handwavey", please give specific critiques on a number. Do you dislike the assumption that ELO could indicate a players winrate (in permille)? Do you dislike the assumption that cooperative games can be won (as good as) 100% of the time? Do you dislike the elasticity factor? Do you like the correction for different variants?

[Stroom]

If you still think these numbers are too "handwavey", please give specific critiques on a number. Do you dislike the assumption that ELO could indicate a players winrate (in permille)? Do you dislike the assumption that cooperative games can be won (as good as) 100% of the time? Do you dislike the elasticity factor? Do you like the correction for different variants?

Elo does not indicate a "win rate". It's more like how good score you can get compared to a predetermined benchmark (which is predefined based on the average elo of the players). It's a somewhat imaginary rate of skill. In BGA Hanabi at least

Assuming 100% probability of winning is also flawed. Mathematically it is never 100%. K of 58 is also horrible as it will make elo fluctuate greatly. Normally it is about 16 for most games. I guess it's 20 based on the BGA wiki.

The distinction of losing by more points already exists. I guess you want to make it so that losing by more points would have an amplifier... but the most you can lose is 10 anyway at the moment (because of K = 20 I guess)... which at my score comes at about -5 points short of max score anyway. So it doesn't really matter much.

Your assumptions about percentage of how often games do not get a perfect score are nice but... they are a bit flawed since you only count max as a win and -1/-2 as a loss? At lower elo, that is also sometimes considered as a positive outcome. And by elo calculation it would also give you a smaller lose than 0 points.

Hanabi can not be won 100% of the time due to several factors. And that is OK. And these corrections that are based on your assumptions also feel weird.

Overall, it does not make sense to change the system that already exists for several other coop games and that keeps players scores in balance at different levels based on their apparent skills. Don't fix what isn't broken.

(I've been playing 2p games for 7 years and it does seem that there is a clear skill difference between 700+, 900+ and 1100+ players.)

[Jessi Jaymes]

I've been thinking about this point a lot lately. I love Hanabi, but I have walked away playing it on BGA because most people I met are too perfectionist and not wanting to lose points, and wants to start dictating how other people played the game to the point it is no longer fun.

My suggestion:
Let's not make players lose points by not attaining a perfect score, but instead, like Just One or Bandido on BGA, reward very little amount of points depending on how they scored on the firework display - such as 5 points for a perfect score, and less thereafter. That way, people would be more welcoming to newcomers and mistakes made instead of being the incredibly toxic mess that this game has made the perfectionist players.

Yuki, I think that it’s unfair to state that someone directing a play style is dictating or a perfectionist. We are in a co-op game and need to speak the same language. So it’s important that newer players recognize they are playing with conventions and make an effort to learn them. If you don’t like the conventions of most tables, make your own table and label your own convention as “secret” or whatever so that you can play “just for fun”

[HanabiGrandMaster]

Elo does not indicate a "win rate". It's more like how good score you can get compared to a predetermined benchmark (which is predefined based on the average elo of the players). It's a somewhat imaginary rate of skill. In BGA Hanabi at least

I agree that it's not. However, I'm suggesting that across ALL BGA cooperative games, we make it so. Now, there are some coop games that give you +1 point just for completing the game. This made me play with players with 3000+ ELO points, who still played a strategy that would only win less than half of the games. While playing with 100 ELO point players who play a strategy that wins >90% of the games.

When the benchmark is set relatively arbitrarily, like with hanabi, you get a lot of frustrations with the ability to filter games. (See your post mentioning this:

For Hanabi, I would sometimes even want to filter by 800 or 900 too. Some 700s still keep playing at 500-600 level.

That’s true. But being able to filter 700+ is a lot better than nothing at all.

In my sample size of ca 200 "master" players, I've found that 2/3 played games that dropped their odds of winning below 50% at a certain point. This, to me, is an indication that the predetermined benchmark ELO table for hanabi is set too low, and doesn't allow differentiation between mediocre players, good players, and great players. Limiting myself to the 0+, 100+, 200+, 300+, 500+, and 700+ filters of the BGA system, I think the winrate in permillage is a decent enough approximation that's instinctive and works without a giant overhaul of the BGA site.

Assuming 100% probability of winning is also flawed. Mathematically it is never 100%. K of 58 is also horrible as it will make elo fluctuate greatly. Normally it is about 16 for most games. I guess it's 20 based on the BGA wiki.

The distinction of losing by more points already exists. I guess you want to make it so that losing by more points would have an amplifier... but the most you can lose is 10 anyway at the moment (because of K = 20 I guess)... which at my score comes at about -5 points short of max score anyway. So it doesn't really matter much.

Your assumptions about percentage of how often games do not get a perfect score are nice but... they are a bit flawed since you only count max as a win and -1/-2 as a loss? At lower elo, that is also sometimes considered as a positive outcome. And by elo calculation it would also give you a smaller lose than 0 points.

Hanabi can not be won 100% of the time due to several factors. And that is OK. And these corrections that are based on your assumptions also feel weird.

Overall, it does not make sense to change the system that already exists for several other coop games and that keeps players scores in balance at different levels based on their apparent skills. Don't fix what isn't broken.

(I've been playing 2p games for 7 years and it does seem that there is a clear skill difference between 700+, 900+ and 1100+ players.)

The winrates (perfect score) for hanabi that have been simulated with a cheat-bot (https://github.com/Quuxplusone/Hanabi) are:
93.92%; 98.37%; 97.94%; 96.47%; for 2p, 3p, 4p, and 5p respectively. They are simulated without flamboyants, and are very similar to my full mathematical calculations (see google-doc on my profile). Again, this is very Hanabi specific; I've not run the numbers for other cooperative games. But it's close enough to 1000 ELO points as the maximum (or 900 ELO points) to still use the master-filter of BGA.

I respect your critique that my maths in the post is... let's say incomplete. But they are a decent approximation and served mostly to motivate my viewpoint on re-interpreting ELO as a winrate promillage.

The maximum ELO gain/loss one can get at the moment is 10 points. Under the assumption that 1000 ELO points is the max, it means that each game you play represents "only" 1% of your ELO, or that the last 100 games are representative of your skill. I don't think that's true, so the elasticity should go up.

Your point that you can see a difference between 700+, 900+ and 1100+ players, just means that the master-level needs more differentiation, which I wholeheartedly agree with. And I think you can achieve that by making a stricter ELO-table.

As an additional argument, in response to:

I've been thinking about this point a lot lately. I love Hanabi, but I have walked away playing it on BGA because most people I met are too perfectionist and not wanting to lose points, and wants to start dictating how other people played the game to the point it is no longer fun.

My suggestion:
Let's not make players lose points by not attaining a perfect score, but instead, like Just One or Bandido on BGA, reward very little amount of points depending on how they scored on the firework display - such as 5 points for a perfect score, and less thereafter. That way, people would be more welcoming to newcomers and mistakes made instead of being the incredibly toxic mess that this game has made the perfectionist players.

When the ELO reflects your perfection better, then it's easier to avoid "casuals" or "perfectionists". yukihimeko's suggestion to change it into a Bandido-style game, where you can achieve 3000+ ELO points, will achieve the complete opposite, making it harder to filter skilled players from frequent players.

Is is also why I recommended to have the basic variant as the training ELO. Skills people learn in 5 colour mode, do not translate well to blackpowder or avalance of colours (reflecting that 1 to 100 ELO points makes you an apprentice in BGA terminology).

[Stroom]

I agree that it's not. However, I'm suggesting that across ALL BGA cooperative games, we make it so. Now, there are some coop games that give you +1 point just for completing the game. This made me play with players with 3000+ ELO points, who still played a strategy that would only win less than half of the games. While playing with 100 ELO point players who play a strategy that wins >90% of the games.

A system that always gives you a point is definitely bad. Similarly, win rate is a bad system as you do not really grasp the actual skill. A player who accidentally wins the first game would be at 100%. Someone who plays thousands of games decently would not be able to get rid of the history of the initial games. And if you only count x latest games, the score might start jumping a lot if the player is trying out different strategies. But elo is actually a good system because you actually have to consistently play well to reach a higher score. With a low K value, it is harder to climb in score. Sure it's hard to fall too but that's actually good as your elo fluctuates less. Over time elo will adjust itself.

What the actual problem at the moment is that all the game modes you can play are aggregated into one general elo score. So a 2p player and 5p player have the same number but they mean different things. Similarly whether you use 5 colors or multicolors or black powder or flamboyants.

I'm sure your winrate proposal would not address that issue either. And it would require EACH game to be thoroughly analyzed for a specific scoring rule... while elo works well enough for every game with minimal simplistic asjustments.

In my sample size of ca 200 "master" players, I've found that 2/3 played games that dropped their odds of winning below 50% at a certain point. This, to me, is an indication that the predetermined benchmark ELO table for hanabi is set too low, and doesn't allow differentiation between mediocre players, good players, and great players. Limiting myself to the 0+, 100+, 200+, 300+, 500+, and 700+ filters of the BGA system, I think the winrate in permillage is a decent enough approximation that's instinctive and works without a giant overhaul of the BGA site.

I'm not really sure what "research" you have made but that seems like a flawed system based on generalized data. your calculation of probabilities and odds seems quite weird.
Again, winrate does not show much about a player. Especially for a coop game where your opponent can just bomb the games intentionally, ruining your "win rate". Elo is better as you can just play with non-bombers and gain back the loss in 3-4 games.

The winrates (perfect score) for hanabi that have been simulated with a cheat-bot (https://github.com/Quuxplusone/Hanabi) are:
93.92%; 98.37%; 97.94%; 96.47%; for 2p, 3p, 4p, and 5p respectively. They are simulated without flamboyants, and are very similar to my full mathematical calculations (see google-doc on my profile). Again, this is very Hanabi specific; I've not run the numbers for other cooperative games. But it's close enough to 1000 ELO points as the maximum (or 900 ELO points) to still use the master-filter of BGA.

This is for only 5-color games, I guess. No-one over 800 elo plays that. It is super easy (as your winning probabilities show >90% too).
I have looked at your google docs and I disagree with most of what you have written about the strategy there, especially for 2p games. And seems like in the one game we played, you decided to bomb intentionally too. So there's seems to be some hard delusion in all of that to me.

The maximum ELO gain/loss one can get at the moment is 10 points. Under the assumption that 1000 ELO points is the max, it means that each game you play represents "only" 1% of your ELO, or that the last 100 games are representative of your skill. I don't think that's true, so the elasticity should go up.

There is no need for elasticity to go up. Just play more. By the way, based on BGA wiki, elasticity is higher if you are new to the game. This way you can climb in elo faster to your skill level if you play well. If you are already experienced, your skill would probably not change much game after game. So why should each win or lose change the score more drastically?

Your point that you can see a difference between 700+, 900+ and 1100+ players, just means that the master-level needs more differentiation, which I wholeheartedly agree with. And I think you can achieve that by making a stricter ELO-table.

Sure, we could differentiate more for filtering. Or maybe the elo calculation parameters should be adjusted in a way that max theoretical elo is lower. This way, players who are currently at 700 would become ~500 and everyone else also goes down accordingly. Only playing the base game would then mean that reaching master is impossible.

Is is also why I recommended to have the basic variant as the training ELO. Skills people learn in 5 colour mode, do not translate well to blackpowder or avalance of colours (reflecting that 1 to 100 ELO points makes you an apprentice in BGA terminology).

What you are asking for is a lot of development work for something that does not really matter much. Different elo values for different play styles, different calculation logics, so many different implementations... for what? Current elo does work well enough. You can roughly judge people based on their value... or if you play often enough, you know people's skill levels by their names too.