Deepcrawl is now Lumar. Read more.
DeepcrawlはLumarになりました。 詳細はこちら

Feature Release: API Log File Integration

DeepCrawl Splunk integration

Feature Release: API Log File Integration

Spring is now upon us and in keeping with the theme of new beginnings we’ve got a new feature release to shout about: brand spanking new integration with!

You might remember at the back end of 2017, we launched our much-loved log file integration. This update allowed you to upload log file data as our source in our ever-accommodating crawler, so you can see how search engines crawl your site.

Fabulous – :):):) – DeepCrawl’s Log File Integration: The Latest Addition to Your Search Universe – DeepCrawl #SEO

— Dawn Anderson (@dawnieando) September 16, 2017

Then, earlier this year, we integrated with Splunk’s log file analyzer, allowing you to automatically pull in log file data directly into your crawl.

Well, now we’ve gone and partnered with another massively popular log file analyzer:


What are the benefits?

In a similar way to our Splunk integration, DeepCrawl users can now integrate their accounts to automatically pull in log file data as a source in their crawls.

With log file data integrated you can benefit by:

  1. Identifying where crawl budget is being wasted crawling pages returning 4xx and 5xx status codes
  2. Knowing where search engines focus their time crawling your site, allowing you to optimize your crawl budget – you can even break not hits down by device
  3. Uncovering which of your non-indexable pages are being crawled by search engine bots
  4. Flagging URLs which are included in your sitemap but aren’t being crawled by search engine bots

How can I integrate in my DeepCrawl account?

Integrating your account as a crawl source is a simple process.

In the second stage of the crawl setup, add in your crawl sources as usual and when you get to the ‘Log Summary’ section, click on the tab and hit the ‘Request Access to API’.

Once you’ve requested access a member of our Customer Success team will be in touch with you shortly afterwards to assist you with the setup of the integration. integration

What if i don’t have a account?

What if you want to gain all the benefits of bringing log file data into a crawl but don’t have a account that will enable you to do this?

Don’t worry! We’ve got you covered!

Simply click the request access button as outlined above and a member of our Customer Success team will get in touch to give you a demo of the integration and help you set up an account with

If you’ve got any questions about our exciting new integration we’d be happy to answer them. Just shoot us over a message or get in touch with your account manager.

We hope you enjoy our latest feature release. Happy crawling!

Avatar image for Sam Marsden
Sam Marsden

SEO & Content Manager

Sam Marsden is Lumar's former SEO & Content Manager and currently Head of SEO at Busuu. Sam speaks regularly at marketing conferences, like SMX and BrightonSEO, and is a contributor to industry publications such as Search Engine Journal and State of Digital.


Get the best digital marketing & SEO insights, straight to your inbox