I voted Leave It. No hesitation.
I was lucky this year to have my first hole in one, and it came immediately after an emergency call from my step-daughter, during which I excused myself for a couple holes to deal with her situation. I caught up with my group and struck a 6-iron well, but none of us saw it land or roll due to the lighting. It was a downhill par 3 with some wind in our face, so we had no idea if I was short, long, whatever. After a search in the front bunker and long in the rough, I was audacious enough to check the hole, and there it was.
I almost felt guilty, because the morning had started so poorly that I had written off the day. I wasn't even really focusing when I hit the tee shot, and I had decided to leave after the next couple holes (would be near the clubhouse), because I was shaken up by the call I had just had from my step-daughter for 20 minutes or so. I had just caught back up with my group who had just teed off, so I just nonchalantly let it fly to restart my round.
I enjoyed the rest of the day (finished the round) and played mediocre golf after that, but I will say that the best way to get a hole in one would include:
be playing in a round where you're in good shape mentally (I was not) and be playing somewhat decent that day (I was not)
see your ball go in (I did not)
don't have it happen on a mishit (but I had been playing so poorly that a GOOD shot was the mishit )
So I managed #3 above. At the time I was thrilled, but I must admit that I feel like it doesn't really count and isn't a big deal, in retrospect. It was luck and didn't even represent that I was playing well at the time. But I'll take this one. Next time, I want all 3 things above to be true.
If none of them are true, I'll definitely leave it.