I keep Def higher just because of switching gear and such with new gear that may or may not have high def.
I am just under your HP rating and have 455 def with Hit maxed out as well.
Make sure you have Hit maxed (8%) before you re-gem though based on the threads and guides within this forum. I can't remember remember the actual number for 1% hit it is like 30 somthing I believe. Anyone have that stat off the top of their head?