I have 2 nemesis shots: a tee shot where anything other than a solid drive simply won't give you a shot at reaching (about 480+yards in my case), or a fairway with a very tight dogleg or multiple doglegs requiring both an accurate yardage and a certain shape to give you a shot at the green.
The driver is a problem for me, especially on holes into the wind. It just puts a lot of pressure on a club I hit like crap 90% of the time and I know I'm leaving yards on the table. Luckily I hit my long irons very well, preferring my 3 iron on most holes, and it often gives me a wedge into the green on par 4s anyway. I don't mind hitting it off the deck either, but the math just doesn't add up on really long holes like par 5s. When I have more confidence in my 1 iron and 3 iron than my driver, being forced to use my driver due to distance is an issue. If it's a wide open hole and the driver can't hurt me, it doesn't bother me. I can get 240ish even if I hit it poorly, but I might get a solid one to carry 260+ or get some decent roll. That means I might have a longer iron into the green, but I still don't need to lay up or try to force an iron farther than it ought to go. I can easily hit my 3 iron 220 in the air with a few yards of roll, straighter than any of my other long clubs oddly enough. But that's about the longest shot I have in the bag right now. I hate this shot more as a reflection of hating my equipment than anything.
There's a 9 hole course near me with a 90 degree left turn about 200 from the tee, with about 180-200 more to the pin from there, depending on your position. And I've been in the left woods before; it ain't pretty. And it's really tough to hit the 50 yard deep green that slopes away from you into a dropoff. Not to mention the mini valley of sin before the green, and the speedy surface. Fun hole though. Tough one to make par, let alone reach in regulation. This hole is actually worse from the white tees for me, because it requires about 160 off the tee followed by a tough 190... Hitting more club on approach than the tee is a good way to crush a golfer's soul. It's honestly not bad for me because I'm both long with my irons and hit my longer irons well. Most players would take a fairway wood off the tee and maybe another or a hybrid into the green on this hole, where I'm hitting a 5 or 6 iron that I'm confident in. It's just annoying that I can't take a club I really like off the tee and everyone's first shot ends up in the same spot.
Another shot worth mentioning is the one through a tree. Whether it's a big shape or threading a shot through a gap, I always convince myself I have a shot, I usually manage OK, but it's really hard and requires an element of luck at times. I can often land the ball where I want, but when there's an obstacle 10 yards ahead of you it can jack with your brain because you can't get away with even the smallest error in shot shape. Plus the first 30 or so yards of my ball flight are the part I'm least familiar with. It takes some guts to pull that sort of shot off. Missing this shot by 6 inches is often worse than missing the green by 10 yards. And you could hit a perfect shot that gets knocked down by a branch.
Water/forced carries don't bother me much because I just take an aggressive low shot in these circumstances and often make better contact as a result. I don't mess around with OB though. Major uphill and downhill approaches aren't a big deal for me because I accept they won't be great shots in all likelihood rather than try to overplay them. They're tough but not overly penalizing in most cases. I often thrive on difficult shots personally, and often see my playing partners get in trouble from them; it's partially because I focus more and have a different objective than usual.
As for hardpan, just don't be afraid of it. Get an old wedge or something and hit shots off a plank or some hard dirt somewhere, just to teach you to accept the bad feedback and focus on good mechanics. I guarantee if you try to make the shot feel less bad, you'll mishit it. Just hit it aggressively and accelerate, best thing to do to overcome the vibration is just to either not hit into the stuff in the first place or to try to make the ground give instead of your hands. It's hard to hit a nice feeling shot off the stuff, but you can still hit good ones. Also consider using a low lofted club, one with less bounce, or a putter if possible. Finally, don't worry too much about spinning it or hitting it high and soft, just play what shot you can to a flat part of the green unless you're feeling confident.