I'm a little confused by your first sentence. Did you engage 4HI, then find that you had no drive whatsoever to any wheel (engine revs, nothing happens), then smoke--or did you engage 4HI, the rears spun but the fronts didn't, and then smoke?
If it's the former, I suspect your transmission overheated and boiled fluid out of its vent tube. Maybe that hose was damaged, wasn't on its barb right, or just blew off. The boiling fluid prevented the pump from making line pressure, so the clutches failed to engage, and you had no drive until the transmission cooled and the fluid settled down. How hot was it where you were, and how hard were you working it? Are you running an external transmission cooler?
If it's the latter, that would suggest a transfer case issue. Park it on a surface you don't care about (gravel, neighbor's driveway) and try to put it in 4HI again. If you're right about a seal being bad and releasing hydraulic pressure meant for the clutches, and the transfer case hasn't already pumped itself dry (check the level first), it should start spraying again when the pump comes on. I wouldn't expect the case seal to be holding back line pressure, but I don't know the ATX14A very well. If it doesn't spray, check that it's actually in 4HI.
Either way, the first thing I'd check is the fluid level and condition in your trans and transfer. Then I'd have a look at the breather to the transmission, see if it's still attached.
The breathers shouldn't need hose clamps, but that doesn't mean it's a bad idea. I've heard of ATF puking onto exhaust components and starting fires.