No, they can do more. The nuclear option is to use Article 7 to suspend Hungary's voting rights at the European Council if they deem that Hungary is acting against the EU's values.

Stopping the flow of EU funds is part of the escalation route towards that but they won't do it unless they've exhausted other options - it's a tricky balancing act to cajole Orban into behaving like a European, and once they use Article 7 there will be no more escalation threats available.

There is no mechanism to throw Hungary out completely. The only mechanism for a member state to leave the EU is in the hands of that member state's government.

So once you've suspended their voting rights, he goes fully into Putin's arms and suddenly you've got an EU member state that is explicitly an agent of Russian foreign policy and doesn't even make the pretense of being European anymore. That may still be the endpoint they need to arrive at, but cajoling him into compromises such as they did on opening Ukrainian membership talks - where Orban agreed to leave the room when the vote took place so didn't need to vote in favour but wouldn't block the unanimous vote that was needed - is still a better outcome for the EU and Ukraine.