I think it would be a mistake to remove this from the game.
The amount of depth introduced because you have to consider that a servant could be stolen in tremendous. By removing that, it's simply puts more dependency on finding early prestige guests that will draw you more.
In other words, one of the only defenses you might have if you lost the first courtship and didn't get a good hit on your main gazebo, but your opponent did, is to find other ways to gain an edge.
Oftentimes your opponent may have gained that edge because they only hired once and they did not hire an extra footmen, or they only hired once and they did not get an extra valet.... In other words, they got greedy to push for the Fairchild, and in return (not always) you might be able to slightly hinder them by stealing a worker.
Further, if someone is truly in a state where they don't have an option for another worker type, it would indicate that they made a poor choice in their first hire, watch their opponent. A second hire, did not respond, watched their opponent gain a servant's quarters, and then found themselves in the loop of their only worker of a particular type being stolen each turn, with not even a double hire able to rectify the situation.
Board game arena consists of players who have played the game hundreds or even thousands of times. This nuanced play is extremely rare and is entirely dependent on an opponent who has not been in this situation before. Plus some lucky tiles early on.... Most folks playing at home are never going to play the game enough to run into this situation.
That said, I'm glad that it exists in the game.
A player can lose on turn 3 in chess.... That doesn't make just broken, it just means that one player has a lot of learning to do and is likely new at the game. That this exists in chess speaks to the depth and level of mastery that can be gained over time.
Removing strategic wrinkles in a game to reduce player interaction doesn't necessarily make a game better.
Back to the opponent who got the early courtship...
While they have the edge for getting Elizabeth and for you missing on your main gazebo, you may be able to gain an edge back by swiping a worker. Removing this option simply means your opponent will continue to snowball prestige guests and you'll be playing from behind the whole game, likely losing, and most likely, by a large margin.
The other way to potentially get back into the game of course is the use of the rumor Mill, and if you truly are behind after that first courtship it would be great to have both options: stealing rep AND workers.... It might be your only way back into the game.
By removing player interactions that keep your opponent in check, the game becomes two snowballs being rolled down a hill. One picks up a few big clumps along the way and continues to gain size and speed, while the other sputters along.
I understand some folks don't like aggressive player interaction or take-that mechanisms in a game, but sometimes they help maintain balance, offering players a way too adjust a game state.
By all means, include it as an option, but I don't think I would ever want to play using it.