I played down to a 12 HI with no driver, I did have a 3w. My first time last year to shoot a sub 40 nine was with 4-PW on the 3352 yd front. IMO a bigger problem than being inconsistent with the driver is the overuse of it. Can't speak for anyone else but I play from 6400-6800 most of the time and for par 4's less than 400 yds driver is usually not a wise choice. The reason the high handicap guys get away with it, probably better to say attempt it, is because they're not very long. I use a driver now but I still hit a lot of 4w, no longer play the 3 w.
Example my former home course has a 348 yd par 4 dog leg left. The left side is guarded by water that juts out where the movement left starts near the 150 marker. Most steer away from the water and miss the fairway right or hit through it leaving a tough, uphill approach in deep rough to an elevated green protected by mounds on the right. Most miss it short and the pitch you're left with over the mound is a bear because the green slopes away from the mounds. Better players cut the corner over the water with a shorter club and it leaves a short pitch in the landing area where it expands around the lake. IMO it's a course management issue because those fearful of the water would be better off in the fairway even if they were 160 out. All it takes is 188 to get there and the angle to green is so much better than what you get missing long and right.