you are actually very wrong about this here. I have played on many baseball fields where houses are very close to the field and well within normal foul ball or home run balls. I think you are absolving the home owner of a little too much in these situations. While it is certainly not their fault that their house got hit, they are certainly responsible for buying a house on a golf course in range of an errant drive.
i've actually done some research into this topic because i have hit some very wayward drives that have come very close to hitting houses. the law varies by state, but in general, unless the golfer is being actively negligent (this means hitting a person or house that is in their intended target line...so trying to hit a ball over a house to cut a corner or hitting into slow playing groups and hitting somebody in that group) the golfer is not responsible for errant shots. the rulings on these cases have been made under the premise that since the best professional golfers cannot always control where they hit the ball, then the average player cannot be expected to hit a perfect shot every time. Houses on golf courses have special insurance for just these situations, unless the course was built after the houses were there in which case the golf course is responsible for having insurance to cover these errant shots.