I'd have played my ball towards that guy.
Actually, something similar happened to me the other day. I was playinga match with a good friend and golfer. I was 1 down through 11 holes. We are waiting on the tee box for some old guy's to play they're way to the green just whacking the ball around, which was very funny to see, because they were loving it and laughing...
Then, has we are ready to tee it up, a guy comes up driving a cart, alone, and just say's this: "I think it is my time to play, get out of there!" Since it was me who was going to hit, I asked the guy if he wanted to die. The guy tried to smack me with a golf club because I had been rude to him and since he was much older than me I had to respect him. The guy goes out, straight to the reception and complains about me and my friend. The guy there called us and we told him what happened. When I told him the story, he just laughed and said, "Well, next time don't respond like that, but the guy was a real jerk wasn't him?"