I've asked around and found that my dodge, parry, and especially block are pretty low. I do have a few yellow gem slots in my gear so i was wondering if I should throw some +defense in there instead of stam or would the defense gems give diminishing returns and not be worth it.
BTW my defense is 550, and i have both the hit and expertise soft caps.
Don't worry about your dodge, parry and block, enough will come with gear. What really kills you is unpredictable physical burst or unavoidable damage. Most of the time in current encounters the only way you can prevent a death is by adding more EH.
Along with what Squats said you can do these things to maximize your EH:
Shoulder enchant to the pvp one
Cloak enchant to 225 armor
Glove to 240 armor
Fervor of the Frostborn to the Glyph of Indom.