Forget the brig, forget having a pip removed from your collar so you're the same rank as Kim... the absolute worst punishment possible on Voyager is Janeway's "disappointed" look.