Form button focus

Annoyingly, when a form element gets focus in IE, the respective submit button changes slightly. Combine this with custom styles for buttons and they gain a black border:

Any ideas?