Deepcrawl is now Lumar. Read more.
DeepcrawlはLumarになりました。 詳細はこちら

How to Whitelist Lumar on Cloudflare

If you are using Cloudflare to improve your website’s security and performance, our crawler may be blocked by Cloudflare’s firewall, hence stopping you from performing a crawl on your website. You can lift this restriction by following the simple steps below.

1. Log into your Cloudflare account

2. On your left sidebar, click on Security, then click on WAF

3. On that page, click on the “Create firewall rule” button

4. Name your firewall rule and select either of the two options below:

A. For IP address whitelisting, you’ll need to enter the Lumar IP addresses for crawling. To do this, choose IP Source Address as the field, equals as the Operator and as the Value. You will then need to click Or to add a second line for IP address

Cloudflare - Create Firewall Rule - IP Whitelisting

B. If you want to crawl with a regional IP or custom proxy, add ‘deepcrawl’ as an allowed user agent. Please note that while we have changed our name to Lumar, our user agent still (for the time being) references ‘deepcrawl’ to avoid creating issues for existing customers. You therefore need to choose User Agent as the field, Contains as the Operator and enter deepcrawl as the Value.

Cloudflare - Create Firewall Rule - User Agent

5. Once that information is entered, select Allow from the Action dropdown and click on the “Deploy firewall rule” button


Get the best digital marketing & SEO insights, straight to your inbox