I like to play on harder difficulties, but sometimes it's outright frustrating because enemies just become sponges. I remember encountering a Dwemer Sphere (or whatever it's called) and it took me literally 4-10 minutes to kill it. I just kept running away and occasionally hitting it. It wasn't really all that much fun and I could have saved time playing on easy.
I mentioned it in another thread...but I really wish we had a hard mode where you could die quickly, but enemies die quickly too. That way you're not wasting time finding some cheap tactic to kill a sponge.
Of course, enemies like giant metal Centurions should remain hard to kill. They're metal! I kinda wish they made it so that they were immune to swords or something, and electricity or warhammers were their weakness. It would switch things up a bit.
So, in the end, I think I like to stick with hard difficulties if I'm playing a stealth role. That way, I can take enemies out with multiplied damage, and getting caught is a huge consequence because I can be taken out really quickly. Otherwise, getting caught is just an inconvenience as I can just kill anyone regardless of stealth.