get_terms() Admin sorting order for Woocommers attribute and categorie

So I’m working on a custom widget for Woocommerce, and I’m trying to get all the terms of the product categories and product attributes.

And the goal is to sort these terms in the same way as you would see them in the admin.
(Where you can sort them with the drag and drop system)

I found some pretty similar topics, that suggest using menu_order or term_order as the orderby value with an extra hook function.

function wpcf_filter_terms_order( $orderby, $query_vars, $taxonomies ) {
    return $query_vars['orderby'] == 'term_order' ? 'term_order' : $orderby;
}

add_filter( 'get_terms_orderby', 'wpcf_filter_terms_order', 10, 3 );

But this didn’t solve my problem, so I’m Still looking for a working solution.

At this point I’m using the get_terms() functions to get all the terms of a product attribute or product category.

$args = array(
    'orderby'    => 'term_order',
    'order'      => 'ASC',
    'hide_empty' => TRUE
);

$term_results = get_terms($taxonomy, $args);

Could somebody point me in the right direction here?
Or is this impossible for product categories and product attributes?