I play on mobile and obviously this problem is even worse there.
You could even cap the take back to like 3 seconds like other games do f.e. Tigris&Euphrates.
But on the other hand this is a very quick game so if you loose by misclick you could just play again...