I used to keep my ball marker on my hat*, on one of those magnetized little clips. LPGA players keep their ball marker there fairly often.
Yet no PGA Tour players do this, and even the guys you play with are highly unlikely to do it.
Why is this? Is it somehow "unmanly"? Cuz believe me, there are a lot more "unmanly" things many golfers will do.
* FWIW I stopped doing it because the TST ball markers are a little too big and they're also, duh, non-magnetic. :) I liked having the ball marker on my hat. Titleist, for one, even made hats with built-in magnets for awhile.