I have only ever lived in So Cal. So I can only speak to what your money will get you here.
It definitely has the weather you are after and great golf. But it comes with a cost. It would probably require you to get a roommate or two to live in a relatively safe neighborhood. And your golf budget of roughly $38/week could get you on a decent course once a week depending the day you play and tee times.
For example.... if you were living in San Diego, you can get a resident card ($25/year) and play Torrey Pines North for $42 (weekday) of $32 (weekend twilight). Most other courses that are worth playing in San Diego are going to be more expensive than this.
Torrey Pines Golf Course Fees | Parks & Recreation | City of San Diego Official Website
Torrey Pines Golf Course Home Tee Time Reservations and Fees Course Information Tournaments and Special Events About Torrey Pines
But.... you would be playing all year round. Just not as often as 3-4 times a week.
By the way, if you find a place with year round golf, where you can play multiple rounds for ~$38/week, please let me know! It'll be something to keep in mind once I get closer to retirement.