With such people I've said, "No thank you, I'd rather play by the rules of golf since I'll be using this round in my handicap." It (a) says what you believe, but (b) if you're worried about seeming uptight, which you shouldn't be, explains why you just want to play by the rules.
I don't care what a person writes down so long as I'm not betting him. I think the boasters (guys who improve their lies, take things inside the leather, etc) get exposed easily once they play with new guys. Golfers who know me know when I say 86 it means 86 just like when I say I shot 103, I really shot 103.
If I'm betting him, I'll be quick to point out his indiscretions. Just this Sunday I mentioned to an opponent in my men's league that he was wrong to remove leaves from behind his ball in a sand trap.
I've played with guys who always seem to shot in the 80s in rounds they don't play with me, but I've never seen them shot in the 80s. These people don't fool anyone.