I managed to remove the state field from checkout with this code:

add_filter( 'woocommerce_checkout_fields' , 'custom_override_checkout_fields' );
function custom_override_checkout_fields( $fields ) {
    unset($fields['billing']['billing_state']);
    return $fields;
}

But this does not remove the same field from Ship to a Different Address.
I can not find this anywhere, not even in official docs, please advise.