Please don't think I'm trying to brag at all here, because I'm certain that most of you would beat the crap out of me if we played a round together. I just wanted to comment on my own personal experience as it relates to this topic. I can attest to the fact that low handicap and long drives don't always correlate, although I agree that most of the time they do. It's certainly possible to have a low handicap without hitting 300 yard drives, and it's possible to drive 300 and have a high handicap.
Like some have mentioned, an athletic swing from a guy with good hand/eye coordination can result in lots of 300 yard drives. That doesn't necessarily translate into low scores by itself though. Low handicap results from a good short game more than anything else, and strength and/or swing speed has little to do with that. Touch, feel, proper reading of the green.....all of that kind of thing can be mastered without ever hitting the ball 300 yards.
I am just getting back into playing regularly after 4-5 years of only playing once or twice per year, and my short game is currently non-existent - especially my wedge play. Even when I was playing regularly before, my wedges were always the weakest part of my game. But I have always been able to hit a ball a long ways. I'm 43 now, but I can still hit it as far as I ever could before. I work out regularly and haven't gotten too large around the middle yet (certainly wearing a different belt size than I did 20 years ago though!). I'm sure that helps some. I'm 6' tall and weigh 215. Not a huge guy, but not small either. Just average sized, I guess.
I just downloaded Google Earth and measured a few drives from this past weekend as best I could trying to put the little dot as close to where the ball ended up as I could remember. If my margin of error on that is +/- 20 yards, the vast majority of the balls hit with a driver were 300 yards or a little more. Now that is with roll, and I'm in West Texas in the winter with dead grass. The ball rolls more here right now than it does on lush PGA courses in the tropics. We actually had very light winds this weekend, so wind wasn't a factor either way. Measuring a couple that I knew about where they carried to, carrying over a tree or a bunker or a creek, etc....the carries were 250-275. I wasn't always in the fairway with the drives, and 2 or 3 of them were well out of the fairway. I'm no pro-level ball striker at all. It just goes far - not always at my target.
On the first 2 or 3 holes, my brother-in-law who has played with me a lot before, kept telling me to wait on the group ahead of us to move off the green before hitting. My father-in-law, who I haven't played with much, kept saying, "Go ahead and hit. There's no way you can hit them from here." I waited. After #3, he quit saying that I should go ahead and hit.
Abilene Country Club is a short course - only about 6,300 yards (par 71), so on every par 4 I had a wedge shot that was much less than a full swing - the worst part of my game. Same thing for the 3rd shots on the par 5s. On some of the shorter par 4s, I was even with the green, just off to the side. Twice I was on the frog hair. I shot an 87 - I even bogeyed one of those par 4s where I drove to the frog hair. On probably 3 out of 4 of those holes, I pitched and chipped at least twice - sometimes 3 times. My putting wasn't bad, but my wedge play was horrible. Again, I'm not at all trying to toot my own horn here. I'm just saying all of this to show that it's possible for a sucky golfer to, in fact, consistently drive the ball fairly long and still end up with a crappy score.
Like some have said, there is a LOT more to golf than length, and I am a long ways from being a good golfer even though I usually knock the little ball a good ways down there off the tee box.