I'd say that the easiest point to move to vigilance would be the one point in Safeguard.
I personally don't want to go that deeply into arms, but I can see a lot of advantages to doing it. If you're looking to find the points for deep wounds (which a lot of people seem to), the most common place to take them from appears to be Cruelty in most cases. Again, I personally have a hard time sacrificing Cruelty in any build, but there's nothing wrong with it and I have to assume that the people that are doing it are seeing a threat increase for doing so. I may go that way eventually, but as it is I rarely main tank 25-man bosses and would rather have the utility talents in prot (i.e. spell reflect, safeguard) than Deep Wounds/Impale.
If you're trying to dig out points, another common place to get them is only put two points in Shield Specialization. If you're going for a threat build (which you appear to be), this could be another place to get the points you are looking for to put into abilities such as deep wounds or improved revenge.
To give direct feedback, I sort of see Safeguard as one of the (many imo) all-or-nothing talents. Safeguard vs. Vigilance is an entirely personal choice for you. The answer to your real question is, "No, you don't need to respec." If you want to respec to pick up vigilance, then you really don't need to change too much.