How can I include ALL post types in the_post_navigation() links, not just current post type?

I have a site with several Custom Post Types: blog posts, movie reviews & book reviews. In my single.php template I call the_post_navigation().

When viewing a blog post the previous and next links only navigate to other posts of type blog, completely ignoring movies and books. If my previous post happens to be a book review, I want the previous link will take the user to that book review and not another blog post.

Same for when on a movie or book page. I don’t want to limit previous and next to those post types. How can I accomplish this?