Studio server down

Game development with Board Game Arena Studio
Post Reply
User avatar
paramesis
Posts: 398
Joined: 28 April 2020, 05:00

Studio server down

Post by paramesis »

I've been getting a 504 Gateway Time-out error on all open tables of all games for the last hour.
I have been able to create and express stop tables, but not open any of them. Is this due to planned maintenance?

If not, there's a possible reason that would be concerning. The problem began immediately after I took an action which I now realize may have caused an infinite loop. On the previous studio server, this would produce a fatal error and halt after 1 second of execution. Is it possible that the execution timeout wasn't configured with the new server, and my infinite loop is still going, causing the studio server not to respond for anybody?

I've called Express Stop on that table, but all other tables are still experiencing the error.
Last edited by paramesis on 03 January 2021, 06:44, edited 1 time in total.
User avatar
paramesis
Posts: 398
Joined: 28 April 2020, 05:00

Re: Studio server down

Post by paramesis »

I was able to use the server again after a few hours. Given the exact timing of the issue, the infinite loop seems like a possible cause. Is there an execution timeout on the Studio server?
User avatar
Geoffster
Posts: 1
Joined: 21 April 2020, 14:53

Re: Studio server down

Post by Geoffster »

I was penalised due to "504 error". Is there a way to request the penalty overturned or cancelled?
User avatar
Een
Posts: 3861
Joined: 16 June 2010, 19:52

Re: Studio server down

Post by Een »

paramesis wrote: 03 January 2021, 06:44 I was able to use the server again after a few hours. Given the exact timing of the issue, the infinite loop seems like a possible cause. Is there an execution timeout on the Studio server?
The studio web server has not changed recently (only the database server).
So there is a php timeout.
I'll check the logs.
Geoffster wrote: 03 January 2021, 09:21 I was penalised due to "504 error". Is there a way to request the penalty overturned or cancelled?
On the studio, account reputation is not important, those are test accounts.
User avatar
Een
Posts: 3861
Joined: 16 June 2010, 19:52

Re: Studio server down

Post by Een »

I see database timeouts for your table 2322990. Not sure what happened exactly, but it looks more like a database lock on this table than a server wide issue.
User avatar
paramesis
Posts: 398
Joined: 28 April 2020, 05:00

Re: Studio server down

Post by paramesis »

Een wrote: 03 January 2021, 18:06 I see database timeouts for your table 2322990. Not sure what happened exactly, but it looks more like a database lock on this table than a server wide issue.
Some other developers have experienced similar symptoms after an infinite loop. Is it possible a database lock is preventing the php timeout from functioning normally? I never saw the timeout error, just a connection error "The server is not responding, are you connected?..." and was then unable to open any studio table for a few hours.
User avatar
lordalx
Posts: 205
Joined: 03 March 2015, 21:15

Re: Studio server down

Post by lordalx »

Hi.

Several and repeatable occurs on m'y side as well.
It don't lock only the current table but prevent any connexion to the studio.
User avatar
Een
Posts: 3861
Joined: 16 June 2010, 19:52

Re: Studio server down

Post by Een »

I found just one php timeout in the logs (for player actions, it's actually longer than just 1 second)
Indeed, external calls (such as database) are not counted by php for the timeout since it's not actually php running time.
There were not so many database timeouts either (maybe 10 or something like that).
Usually, infinite loops hit the memory allocation limit rather than the execution limit, but of course, it all depends on what was in the loop. Potentially if the loop as very little php running time, very long database query time or ext calls time, and doesn't allocate memory, I suppose it can run for a long time.
Still, it only blocks a thread of the web server, so other developers are fine (and I think other games timeout for you because we probably still have a config limiting to one current request per IP on the studio; we can look into lifting this as we lifted it for the mainsite)
Post Reply

Return to “Developers”