Exactly! But not "then", it's happening right now. The game is automatically starts and other players should wait until my timer expire. They become angry and kick inactive player from the game instead of aborting the game (I guess 1 of 20 stays calm and aborts the game).
Such games shouldn't even starts. As for me, correct behavior when some player haven't accept game for some time (15s?):
1. Don't start the game
2. Stop searching for inactive player
3. Continue searching for active player