Being unhittable is better then stamina but once you're past 102,4% stamina is certainly good. I'm still fully mastery gemmed but that's to reach 102,4 with 1 stamina trinket and flask (390 itemlvl) and I'm going to switch over to a few stamina gems when i get more upgrades.
It might take some time until you got enough wiggle room for a stamina trinket (8+ epic mastery gems or 10+ regular ones), until then start replacing your gems one by one with stamina and switch back when you got enough upgrades. Final goal should be something like that: souldrinker + 1-2 stam trinkets + stam flask.
Ps: get the boe boots ASAP, no reason not to use them over stompers.