Latin h tended to be dropped even in classical times, particularly in the middle of words. Thus nihil ‘nothing’ has an alternative form nīl, and mihi an alternative mī, while dē- plus habeo yields dēbeo ‘I owe’.
In initial position it was more tenacious, though even here by classical times it was only the educated classes who pronounced h. At Pompeii, destroyed 79 CE, there are inscriptional forms such as ic for hic ‘this (m.)’, and conversely hire for ire ‘to go’. In his poem about Arrius, Catullus pokes fun at hypercorrections such as hinsidias for insidias. Even the educated sometimes got confused: the letter h in the regular spelling of humor, humerus, and humidus is apparently unetymological.
The Romance languages inherited no phonetic h from Latin. The h that we pronounce nowadays in English words of Romance or Latin origin reflects a spelling pronunciation: habit, hesitate, horror and for most speakers humo(u)r, humid. As we all know, in various other Latin-derived words we have not restored h despite the spelling: there is no h in heir, hono(u)r, honest. In herb Brits and Americans agree to differ.
I was thinking about this because I have been noticing people pronouncing adhere, adherent, adhesion, adhesive without h, thus əˈdɪə etc. In LPD I give only forms that include h — əd ˈhɪə etc. In this I follow Daniel Jones’s EPD, though I notice that the Cambridge EPD now includes the h-less forms. Rightly so; on reflection, I think they are widespread enough to warrant inclusion, at least for BrE.
I have long been aware of the corresponding h-less pronunciation of abhor, which both LPD and the current EPD (but not the DJ EPD) include.
I don’t think there is any tendency towards a spelling-inspired restoration of h in words with the prefix ex-, as exhaust, exhibit, exhilarate, exhort, which all have -gˈz-. But exhale is a notable exception, always having -ksˈh-, and so sometimes is exhume.
You sometimes encounter the hypercorrect spelling exhorbitant for exorbitant. I can’t say I’ve ever heard the corresponding hypercorrect pronunciation, but presumably it exists.