I have a WordPress price comparison website for affiliate purposes. I use WoocCommerce to store and update the product data. Currently, I use a Python script with Selenium on my PC to scrape product prices from different websites. This creates a CSV file which I then manually import into WooCommerce to update the product data. I am looking for a solution whereby the scraping can be done automatically e.g every hour in the cloud/on a server and then automatically update the WooCommerce product data and finally publish the changes. All this without having my PC on and avoiding any manual intervention. Essentially, I am trying to create a ‘live’ price comparison website.

Please can someone point me in the right direction what the options are on how to do this? I am really struggling to know where to start, and I am happy to explore any feasible option.

More detailed breakdown of the steps required:

Step 1. Scrape the price from a website

Step 2. Update the Regular price field on WooCommerce

Step 3. Using this price, perform a calculation to calculate e.g. price per kg. The kg mass is already in a WooCommerce custom field and does not change, so the automated process would need to use the mass to perform the calculation.

Step 4. Update the ‘price per kg’ WooCommerce custom field.

Thanks in advance,

Charlie