At one time golf was for the wealthy and it still is, but not exclusively. In Northern NJ we have have some classic older and some extravagant new courses. To belong to a course like Baltusrol, Ridgewood, or Somerset takes money, connections, and the proper social status. There are some excellent public courses that have opened with several charging well over $100/round on weekends - that takes deep pockets also. However, there is some great golf to be had at $40-$60/round which is more affordable, but it is still not a poor man's pastime. Then there are some well-maintained county courses where a resident can play for $35 with a cart. And of course there are a few decent executive courses or out of the way courses where you can play for $15 - $25 on weekdays.
My point is that most people in this State can afford some level of golf if they are so moved, and on a regular basis. Certainly not a poor man's sport, but it can be had at a modest price. The availability of used, excellent equipment on-line is another bonus today. I have no statistics but just my observation, but I would put forward that the typical golfer in the US is nowhere near wealthy, and many of them are top notch players.