Page 1 of 2

server crashed?

Posted: 15 June 2021, 23:55
by Espina
There seems to be a problem on one of the servers. For example I was playing at https://boardgamearena.com/table?table=180529202 and we both (myself and other player) are getting "gateway timeout" and are unable to access the board or make moves.

Re: server crashed?

Posted: 16 June 2021, 00:18
by aesche
Yes, server 4 is the culprit…

Re: server crashed?

Posted: 16 June 2021, 00:56
by pinkyandthebrain
Is anyone else stuck in limbo?

We have a Hanabi game that was going on during the blip (https://boardgamearena.com/4/hanabi?table=180520481), and it is still ongoing an hourish later but can't get into the game to stop it or continue it. As the game is ongoing we can't join or start another real time game. Can't seem to find a contact us button that allows stalled games to be reported and deblocked either, just the contact us button that leads to FAQ style help boxes.

Re: server crashed?

Posted: 16 June 2021, 01:26
by bendow
I was able to quit the game from the lobby (through "my games in progress"), though AFAICT it still exists, and I probably got a reputation penalty.

Re: server crashed?

Posted: 16 June 2021, 07:47
by Een
Our hosting provider had an internal network issue during a maintenance operation, making one of our game servers unavailable for a few hours. Players with a live game on this server at the moment of the issue have been impacted. Sorry about that, we'll cancel the penalties.

Re: server crashed?

Posted: 16 June 2021, 09:48
by bendow
Een wrote: 16 June 2021, 07:47 Our hosting provider had an internal network issue during a maintenance operation, making one of our game servers unavailable for a few hours. Players with a live game on this server at the moment of the issue have been impacted. Sorry about that, we'll cancel the penalties.
No problem. Just so you know, the page that came up telling me that the game had ended claimed that the service was stable at the time, so I suspect there may be a hole in your monitoring.

Re: server crashed?

Posted: 16 June 2021, 10:50
by Een
bendow wrote: 16 June 2021, 09:48 No problem. Just so you know, the page that came up telling me that the game had ended claimed that the service was stable at the time, so I suspect there may be a hole in your monitoring.
Stable overall, but yes, at a more fine grained level there was an issue. Many possible improvements all over the place. And it's a big place :)

Re: server crashed?

Posted: 16 June 2021, 19:50
by aesche
Thanks for the fix and explanation.
Een wrote: 16 June 2021, 07:47 Sorry about that, we'll cancel the penalties.
I‘ve been wondering about this on earlier occasions already, do you have an automated way of detecting and removing all penalties „awarded“ due to such down times, or does this need to be reported somehow by everyone affected?
Not asking for myself (and generally dont think a single penalty would be a real issue for anyone, so really just out of curiosity), but just seeing that my playing partner that ended up quitting the game yesterday was not fixed yet. No worries if that‘s just too impatient / still is planned only.

Re: server crashed?

Posted: 16 June 2021, 21:41
by Een
aesche wrote: 16 June 2021, 19:50
Een wrote: 16 June 2021, 07:47 Sorry about that, we'll cancel the penalties.
I‘ve been wondering about this on earlier occasions already, do you have an automated way of detecting and removing all penalties „awarded“ due to such down times, or does this need to be reported somehow by everyone affected?
Not asking for myself (and generally don't think a single penalty would be a real issue for anyone, so really just out of curiosity), but just seeing that my playing partner that ended up quitting the game yesterday was not fixed yet. No worries if that‘s just too impatient / still is planned only.
We have an automated detection and penalties are automatically cancelled if a crash is detected. For smaller crashes that don't degrade the service enough to trigger this, we cancel manually penalties over the period of the crash.

Of course this may take a little time as our priority is not the penalties, it's the crash, so it can lag a little behind. People tend to report penalties immediately or even contact us directly by chat. In general, advice would be to wait until the system is running smoothly again, and a little more. If it's been half a day since the system is fine and the penalty is still there and the player cares about it, then it's good to let us know, maybe we didn't make the manual cancellation broad enough.

Re: server crashed?

Posted: 17 June 2021, 01:08
by compte obelète
Een wrote: 16 June 2021, 21:41
aesche wrote: 16 June 2021, 19:50
Een wrote: 16 June 2021, 07:47 Sorry about that, we'll cancel the penalties.
I‘ve been wondering about this on earlier occasions already, do you have an automated way of detecting and removing all penalties „awarded“ due to such down times, or does this need to be reported somehow by everyone affected?
Not asking for myself (and generally don't think a single penalty would be a real issue for anyone, so really just out of curiosity), but just seeing that my playing partner that ended up quitting the game yesterday was not fixed yet. No worries if that‘s just too impatient / still is planned only.
We have an automated detection and penalties are automatically cancelled if a crash is detected. For smaller crashes that don't degrade the service enough to trigger this, we cancel manually penalties over the period of the crash.

Of course this may take a little time as our priority is not the penalties, it's the crash, so it can lag a little behind. People tend to report penalties immediately or even contact us directly by chat. In general, advice would be to wait until the system is running smoothly again, and a little more. If it's been half a day since the system is fine and the penalty is still there and the player cares about it, then it's good to let us know, maybe we didn't make the manual cancellation broad enough.
I know it's off the topic, but i feel it's way too much easy to kick someone of a table if they are in negative in a tournament. as you just need the negative and only one person to kick the guy, like, we should add like, the table remaining/percentage of the game done, the concentement of more players or the concentement of the admin of the tournament to allow the kick. can we restore destroyed elo/karma for being only at -55 mins in a 3/4 done game and being 3 days or more still to play?

like, i got my thurn at 3 of the morning, i was sleeping at this time, and at 7 or so, i got kicked, didn't have the chances to get my turn. Like, people's health is important too. Should we implement also a anti-"let a guy play at his night" fuction?