Deepcrawl is now Lumar. Read more.
DeepcrawlはLumarになりました。 詳細はこちら

How to Whitelist Lumar on Cloudflare

If you are using Cloudflare to improve your website’s security and performance, our crawler may be blocked by Cloudflare’s firewall, hence stopping you from performing a crawl on your website. You can lift this restriction by following the simple steps below.

1. Log into your Cloudflare account

2. On your left sidebar, click on Security, then click on WAF

seBT77yPE04Q38oLzbery5cQ2N3SIRK8QTCbx65_NQo8IIcXb5FB_2IyaRbPIq0mkaH5DYAOtJOZu1dZz_2YixQeBHIoJb6Bq2Xi6KivfD5AHO_vD5PANhJEEUn0hi5xNsnOxZCjC 7Ngu2a7VPPjtvNfXQC0lJPA406EgT8AmxfCtAAsM9OC48fpA
3. On that page, click on the “Create firewall rule” button

xvaDAWkKQrMrMDM9lfb791aVHqfGirOozlUWTYksDChAXF4Xk4KYSg3LYYacVDHjpEDINbNrZ2lmRBnIKxNEohuqb8LCDEs08CyyV1Mn1bg_gIqod8nfjo63L2rmf XbSdnHubojapvVFlJ9Pu zWL8ezxaKr9Son69sGwTBebz2F yoJ6GCfv5lMQ

4. Name your firewall rule and select either of the two options below:

A. For IP address whitelisting, you’ll need to enter the Lumar IP addresses for crawling. To do this, choose IP Source Address as the field, equals as the Operator and as the Value. You will then need to click Or to add a second line for IP address

Cloudflare - Create Firewall Rule - IP Whitelisting

B. If you want to crawl with a regional IP or custom proxy, add ‘deepcrawl’ as an allowed user agent. Please note that while we have changed our name to Lumar, our user agent still (for the time being) references ‘deepcrawl’ to avoid creating issues for existing customers. You therefore need to choose User Agent as the field, Contains as the Operator and enter deepcrawl as the Value.

Cloudflare - Create Firewall Rule - User Agent

5. Once that information is entered, select Allow from the Action dropdown and click on the “Deploy firewall rule” button


Get the best digital marketing & SEO insights, straight to your inbox