There is intent in every rule, otherwise it's meaningless. Sports are full of rules which are regularly broken, but a referee doesn't call each infraction unless the player gains an unfair advantage. If the refs called every infraction, games would take days to play.
In golf, the player is often the referee and is relied upon to call the infraction based on his own judgment. Most rules are based on the premise of prohibiting an unfair advantage. Brushing a leaf is not gaining any advantage, so as far as I'm concerned, the player should have the option to use his own judgment as to whether to call it or not. By definition, the honor system is telling the player to use his own judgment.
Nothing is cut and dry in this world. There are always exceptions and mitigating circumstances which are outside the letter of the law.