For PvE don't worry too much about armor class. It's useful in some circumstances, and you should probably pass on cloth for a clothie or mail for a shammy (etc.) just to be fair, but if the stats are superior, use it. Very rarely will you be taking mitigatable physical damage, and even that won't be too hard to heal through (if clothies can survive it in cloth, you can too).
PvP is a different story but beyond the scope of this thread :-P