I managed to modify an existing answer from a closely related thread to achieve this.
You can find the answer from the related thread here.
https://wordpress.stackexchange.com/a/365334/185141

add_action('woocommerce_before_single_product', 'elron_prev_next_product');

// and if you also want them at the bottom...
add_action('woocommerce_after_single_product', 'elron_prev_next_product');

function elron_prev_next_product() {
    
   global $post;
    
   echo '<div class="prev-next-buttons">';
   $terms = get_the_terms( $product->id, 'product_cat' );  
   foreach ($terms as $term) {
   if ($term->ID == $category->cat_ID) {
   $all_posts = new WP_Query(
      array(
         'post_type' => 'product',
         'meta_key' => '_price', // <-- CHANGE THIS
         'orderby' => 'meta_value',
         'order' => 'ASC',
         'post_status' => 'publish',
         'posts_per_page' => -1,
         'tax_query' => array(
                array(
                        'taxonomy' => 'product_cat',
                        'field'      => 'term_id',
                        'terms'    => $terms[0]->term_id
                     )
         )
      )
   );
       break;
   }
   }
   foreach ($all_posts->posts as $key => $value) {
      if ($value->ID == $post->ID) {
         $nextID = $all_posts->posts[$key + 1]->ID;
         $prevID = $all_posts->posts[$key - 1]->ID;
         break;
         wp_reset_postdata();
      }
   }

   if ($prevID) : ?>
      <a href="<?= get_the_permalink($prevID) ?>" rel="prev" class="prev" title="<?= get_the_title($prevID) ?>"><?= esc_attr__('Previous product') ?></a>
   <?php endif; ?>
   <?php if ($nextID) : ?>
      <a href="<?= get_the_permalink($nextID) ?>" rel="next" class="next" title="<?= get_the_title($nextID) ?>"><?= esc_attr__('Next product') ?></a>
   <?php endif; ?>
   <?php

   echo '</div>';
}

I know i was searching for a way to make next/prev button on my product pages that were sorted by price and gave products only inside the product category i was on, for a long time, if you did as well give an upvote! 🙂