Add the correct amount of padding to the non-hover state. For example, if you are adding 5px of padding on each side on the hover, add 5px of paddng on each side on non-hover.

Edit: Since it appears you are just using width, add the correct width to the non-hover state instead of padding.