Fair points. I'm a little confused by your statement that you have "less cards in hand." The forced 6 is in addition to your normal starting hand, so you start with 5 cards, not 4, which is actually MORE cards.
This setup is definitely not to everyone's taste, and certainly plays differently even from "standard" Brink of War. I personally find it an interesting difference, but will be glad to move to something else next season. Like you, I think Takeovers help balance some otherwise overpowered strategies (that #!%! Mediator/Trade Emissaries combo) and are a useful part of the expansions.
I'm interested that you feel the Imperium has the advantage. My own opinion is that the Cantina is a bit stronger; the Warlord is definitely not the most powerful Military start world (that'd be either New Sparta or my personal fave, Uplift Mercs), the Cantina's Production draw bonus is stronger than the Warlord's Explore bonus, the Cantina can settle green X windfalls without help and is all-around more versatile.
Even more tellingly, Rebel Alliance is just hands-down better than Imperium Seat. It will usually score more VP, has a more useful Settle bonus, and its Takeover is far more flexible; the Seat can only conquer Rebels, whereas the Alliance can conquer ANY military world in the Warlord's tableau. If the Cantina can get just a little bit of early military help, it can completely shut down the Warlord's military.
Anyway, each to his own. Sorry this season is one you're not enjoying. I'm sure we'll be doing something else in a couple of months; I think we'll all be tired of it by then.