Unable to get wordpress+nginx+multisite to work. Child sites return 404

I am doing a multisite installation with WordPress 5.5 on nginx 1.17. I am using this config based on what I read here.

My wp-config.php has following settings:

define( 'WP_ALLOW_MULTISITE', true );
define( 'MULTISITE', true );
define( 'SUBDOMAIN_INSTALL', false );
$base = '/';
define( 'DOMAIN_CURRENT_SITE', 'my-site.com' );
define( 'PATH_CURRENT_SITE', '/' );
define( 'SITE_ID_CURRENT_SITE', 1 );
define( 'BLOG_ID_CURRENT_SITE', 1 );

and I can access the base URL fine. But if I try to access any child site I get 404 from NGINX. I inserted a error_log statement in the /var/www/html/index.php file just to check if the request is even getting to wordpress (i.e., php-fpm) from nginx.

define( 'WP_USE_THEMES', true );
error_log("received request");
/** Loads the WordPress Environment and Template */
require __DIR__ . '/wp-blog-header.php';

I can see a request received message in the WP log when I access the base URL but there is no output when a child site is accessed showing that nginx is not even forwarding the request to wordpress. I checked MySQL tables and I can see tables for child sites:

mysql> show tables;
+----------------------------+
| Tables_in_wordpress        |
+----------------------------+
| wp_2_commentmeta           |
| wp_2_comments              |
          ...
| wp_3_term_taxonomy         |
| wp_3_termmeta              |
       ...
| wp_options                 |
| wp_postmeta                |
       ...
43 rows in set (0.01 sec)

How can I fix this? Nginx seems to be the culprit. I created my child sites using the wp-cli.

bash-5.0# wp site create --slug=childsite --path=/var/www/html --allow-root
Success: Site 3 created: http://my-site.com/childsite/

I have done nginx -s reload many times btw so that is not an answer.