I think it's worth experimenting and finding out what works best for you.
I'm an impatient person, typically aggressive and easily annoyed... so I like to swing fast and get it over and done with, lol.
Having said that, when I get too fast, I get all over the place and my ball striking sucks. Being the slow learner that I am, I'll got through a few weeks or months of this, and then just ease up and let it come.
Guess what!
When I ease up, my ball striking improves out of site - having gone through a few cycles of this, you think I'd learn.
I was playing with a trainee professional on the weekend and a middle age guy who was playing off 7.
The middle age guys swing was super slow on the backswing (position at top was way under play), but because he had a fairly long pause at the top, the young pro said he had plenty of time to correct on the way down.
This guy was a very consistent ball striker... watching him setup and swing back you'd think he had no chance.
As for myself, when I eased my swing up (closer to his "slowness") my ball striking immediately improved.
The more you practice or the better your hand eye coordination, balance etc - the more you can save a poor swing and keep the ball in play... otherwise, try easing up a bit on the takeaway and see how you go.
Sean