I would recommend a different web crawling method, if you're getting flagged for scraping then you might have to change up how it works, are you using a headless browser with a user-agent or are you just reading html? Without disclosing exactly how it works would you be able to identify why it got you flagged?
Thanks for the input, one of the issues was how often we scraped, initially, it was set at 30minute intervals to ensure the price was recent, then I changed it to once a day. Still, I think it was too often and therefore they band us (probably just an automatic WAF rule) through resource utilisation - after all, we try and collect data on over 7k items. So we had to change the schedule to weekly and we are going to add a couple of rules, which should have been in place. Check first then update.