1) I hate playing with drunk players more than anything else. When they get loud, pushy, and reek of alcohol, it detracts from my enjoyment of my environment. I don't mind funny, talkative, or goofy, but drunk and annoying is not fun.
2) I hate partners who don't always verify their ball before they hit it. I always, always check the ball before I hit it to make sure it's mine, even if I'm playing alone. I hate watching someone go up to a ball and hit with hardly a second glance at it. I've had my ball hit more than once.
3) People behind you who get fed up with the pace of play and jump one or two holes ahead of you just to find out it's not any faster up there, causing you to wait on them. We may be a half hole behind the group ahead, but that doesn't mean you can sandwich yourself ahead of us, because then we just have to wait on you in turn.
4) Reading greens in late twilight hours. I'm not good at green reading to begin with, but reading greens with dim lighting and shadows cast on them makes a hard job almost impossible. When I play until it's dark, the green reading on the last 3 or so holes is usually guesswork. This is fixable by playing earlier in the day, but I don't always have that option.
5) Partners who smoke. It stinks and I'd rather not share a cart with you. I respect your right to smoke, but I don't like to be near it.
6) Partners who pick up your "gimmes". Please, please let me putt out. I live for the sound of the ball hitting plastic. At least this can be cured by talking to them, though, so they won't do it. Some do it just to speed up the pace of play, but putting out from 1 or 2 feet only takes an extra 20 seconds at most, and that's with a long walk and setup.
7) Mud bars in front of greens. I don't like chipping out of mud, or walking through mud just to get to the green.
8) Strong dogleg par-5s. Pulling a wood or hybrid on a par-5 teebox feels wrong. I'll probably have to pull it for the second shot as well.
9) Range ball spillover onto the fairway. Which of those 20 balls in the left rough is mine? Have fun!
I didn't think I had such a long list, but I think that's literally everything that detracts from the golfing experience for me. (Aside from poor play, of course.