Sorry but you can't do that -- as much as you want to and as much as the guy deserved. That might be considered assault, so you've got to let your player take the verbal abuse.
As a rule of thumb I tend to walk. I have a 2 wheeled cart and am OK to push it and play without a caddie. The only courses I would prefer to have a cart is one where there are a lot of steep uphills. I find it more tiring to climb than it should be. I suppose being unfit and time catching up to me has that effect.
I have never objected to a cart but rarely see a need to pay separately for that. If it is built into the green fee or is mandatory, I will use the cart. Of course carts tend to be quicker and less exhausting so that is a big plus.