OMG, same thing happened to me about a month ago. I was getting ready to tee off on my first hole, I was playing by myself in a cart, and as I was approaching the ball this foursome of ladies with absolutely no respect, ettiquette, drive right infront of the tee box to the womens tee and proceed to tee off. With my standing right there with driver in hand, I asked them what they think they were doing, and they gave this BS reason that they're all members to the golf course which somehow ALLOWS them to cut infront of me.
They got mad at me for trying to defend myself, I played behind them for 1 hole, slowest group of players I've ever seen, furthest shot was about 100 yards. I skipped a hole so I wasn't behind them anymore, finished my first 9 and noticed that they were only finishing #4.
No you weren't a jerk, but hey at least these people at least informed you they were going to try and play through, these people just assumed they had the god given right to cut infront of whomever they want on the golf course no matter how slow they're playing and how fast the other person is playing.