It has always fascinated me how sensitive golfers are to any noise during their turn. It's one thing if someone blows an airhorn in your backswing, but I would think you would be focused enough to not even realize people were talking around you, etc. Maybe it's a lack of experience in those kinds of situations, who knows.
Personally, I couldn't care less whether people are talking or moving around during my shot because I never notice it. Once you experience trying to hit a baseball with thousands of fans going crazy and screaming their lungs out, you really don't need silence because you create your own.
Best example I can think of was when Mark McGwire was in a televised golf skills challenge and he would get the crowd into it and cheering during his shot.