Deepcrawl is now Lumar. Read more.
DeepcrawlはLumarになりました。 詳細はこちら

5xx Server Errors: A Guide to Finding & Fixing 5xx Issues on Your Website

What are 5xx server errors? And how do you find and fix 5xx errors on your website?

Have you ever clicked on a search result only to get a notification stating that “this page isn’t working”? Usually, these errors come in the form of 5xx or 4xx error codes. Having an HTTP availability error or server error on your website can be detrimental for both SEO and conversions. But what are server errors anyway? 

There are several types of server errors that can be issued as a response to a client’s request to access your site. If server errors occur quite frequently on your site, it could have a negative impact on your SEO efforts. This is because some HTTP status codes (like 5xx errors) could signal to search engines that they should crawl those pages less frequently, or could lead to a page being dropped from the search engine index. But not all HTTP status codes are created equal, so it’s worth understanding the different types of errors — and how they might affect your search ranking.   

In this article, we will dig into the different types of 5xx server errors, what each one means, and how to find and fix them.

 

What is a 5xx server error?

A 5xx error message occurs when a website’s server can not successfully meet the client’s request to access a specific page.

 Often, you will see 5xx status codes appear in your browser like this:

An example of how a 5xx server error message might appear in your browser
How a 5xx server error may appear in your browser.

5xx simply refers to any HyperText Transfer Protocol (HTTP) status code in the fifth class of errors. HTTP status codes range from 1xx to 5xx. The first number—which signifies the class—allows us to understand the type of response the website is returning:

As you can see from the list above, only 4xx and 5xx codes are considered errors

 

What causes 5xx errors?

A server error can happen at multiple points during the response and can happen across multiple layers of the server environment. The most common causes on websites are:

  • Bugs on a web server like Nginx or Apache
  • Backend failure of a content management system (CMS) 
  • Incompatible plugins running on the CMS
  • The website development framework (like PHP) not being compatible with the CMS
  • A content distribution network (CDN) like Cloudflare misconfiguration
 

Are 5xx server errors bad for SEO?

There are several reasons why 5xx errors may impact your SEO efforts.

First and foremost, if users are unable to access pages on your site, it results in a bad user experience for your website visitors. For example, if you run a major eCommerce website and your users keep encountering errors when they are looking for different products, they are unlikely to stay on the site. Not only will this impact how users interact with your site (and whether they bother to return), it will impact metrics like bounce rate and, more importantly, conversions. With the launch of Core Web Vitals last year, it’s clear that Google is increasingly looking at page experience in its search algorithms.

Additionally, if a 5xx code is returned when Googlebot attempts to access a page on your website, it could result in Google dropping that page from its index. 

Google’s John Mueller has said that Google will initially retry a page that has a 5xx error, but “if we continue to see …the 500 errors then we will …slow down crawling. And if we continue to see that there are 500 errors, then we will drop those URLs from the index.”

 

What are the different types of 5xx server errors?

As mentioned above, there is a range of 5xx errors that can be returned by the server. Although the 5 indicates it is a server-side issue, the numbers that follow allow us to understand the exact issue in order to rectify it. 

500 – Internal server error

A 500 error code is generated when the server encounters an unexpected issue. This is probably the most commonly seen 5xx error and can happen for numerous reasons. Often, this is the server’s way of saying “we are currently experiencing technical difficulties. Please try again later.”

501 – Not implemented

A 501 error happens when the server can not support the functions needed to make the request happen. 

502 – Bad gateway

A 502 response occurs when the server is acting as a gateway or proxy. If the upstream server you are expecting a response from does not return a valid response while the server in question is trying to process the request, this results in a 502 error.

503 – Service unavailable

This error is returned when the server is unable to process your request at a particular time. Usually, this is only temporary (often due to server maintenance) and is nothing to worry about.

504 – Gateway timeout

Similar to the 502 response above, this happens when your server is acting as a proxy or gateway. On this occasion, the upstream service did not return a response in a timely manner. 

505 – HTTP version not supported

If a web server does not support (or refuses to support) the HTTP protocol used in the request then it will return a 505 response code. When this happens, the response should explain why it is not supported.

506 – Variant also negotiates

Content negotiation is the HTTP mechanism that allows websites to serve content in different formats. For example, if you have an image that can be served in PNG or GIF format (variants), this mechanism allows a browser to show whichever one it can support. 

If there is an internal error within the configuration of the chosen variant, that means there is no proper endpoint in the content negotiation process. In which case, you will receive a 506 error response.

507 – Insufficient storage

Similar to a 503 error response, this is often temporary. A 507 response is served when there is not enough storage available to store the representation it needs in order to complete the request. 

508 – Loop detected

A 508 code indicates the operation has failed due to the server encountering an infinite loop when attempting to process the request. 

509 – Bandwidth limit exceeded

This code appears most often on shared hosting environments and means your website has now exceeded its allotted bandwidth on the server and is no longer accessible to the public. 

510 – Not extended

A 510 response indicates the server needs further extensions in order to fulfill the request. 

511 – Network authentication required

This happens when the client needs to authenticate themselves before being granted full access to the website. Often, this can happen due to the location you are sending the request from or if you are using a virtual private network (VPN).

When this response is returned it should include an accessible link to allow the user to enter their credentials and gain authorization. 

524 – A timeout occurred

This error was created by the content delivery network (CDN) Cloudflare and shows when your website doesn’t respond in time. That means Cloudflare can’t provide a response to visitors or crawlers and results in this familiar screen:

Example of a 524 server error
 

How to find 5XX errors on your site

Often, it can be difficult to find 5xx errors unless a website user points one out or you happen to stumble on one yourself. This is especially true for large websites where an error may only occur on one particular page.

However, permanent 5xx errors are a serious SEO issue. That’s why regularly auditing your website to find these errors (and any other issues affecting your website’s health) is so important. 

The best way to do this is to use a technical SEO tool like Lumar Analyze to automatically audit your site at regular intervals. Let’s look at how to find 5xx issues with Lumar.

Find 5xx errors with Lumar

First, add your website as a project within the tool and fill in all of the required settings. Then let the website crawler do its magic.

how to find 5xx errors using Deepcrawl's technical SEO tool - step 1

Once your crawl is complete you will get a detailed overview of your site and any errors that have been identified as part of the audit.

To find 5xx errors, go to the left-hand menu and select Indexation > Overview:

how to find 5xx errors using Deepcrawl's technical SEO tool - step 2

On the Indexation dashboard, if you scroll down on the right-hand side, you will see the HTTP status breakdown. This will show you all of the pages by their HTTP response code, and you can see if there are any pages showing errors:

https status code breakdown report in Deepcrawl's SEO analytics tool

As you can see above, you can see how many pages are associated with each HTTP status code type. If you have any 5xx errors showing, select it from the menu and you will see a detailed breakdown of each URL.

Find 5xx errors with Google Search Console

You can also find 5xx errors in the coverage report within Google Search Console. To do this, head over to your Google Search Console account and select the correct property.

Then, in the left-hand menu under the “Index” tab, select “Coverage”.

how to find 5xx errors in google search console - step 1

Scroll down and you will see the 5xx errors listed below:

how to find 5xx errors in google search console - step 2

If you click “Server error 5xx” you will be taken to an overview of the affected pages:

how to find 5xx errors in google search console - step 3

You can also use GSC to find 5xx errors by checking the crawl responses in the Crawl Stats Report. In the left-hand menu, select “Settings”. Once on the settings screen, select “Open report” under “Crawling”

how to find 5xx errors in google search console - step 4

Once the report is open, scroll down to Crawl Requests Breakdown > By Response

how to find 5xx errors in google search console - step 5

If you have any 5xx pages on your site, they will show here and you can select “Server error (5xx)” to view a detailed report.

 

Find 5xx errors in your log files

Considering Google Search Console will only show you 5xx server errors that Google receives, this may leave you in the dark about certain pages Google hasn’t stumbled upon. That’s why one of the best ways to look for 5xx errors is directly in your log files.

Log files are usually found directly from your web host, although in some cases, these can have strict security measures and be difficult to access. 

How you access your log files through your host differs depending on what service you are using. However, as an example, with Siteground, I can access my error logs simply by opening Site tools > Statistics > Error Logs

how to find 5xx errors through log file analysis

One great way to analyze your log files is with Lumar’s file integration with Logz.io which allows you to perform log file analysis directly in Lumar’s Analyze Tool.

 

How to fix 5xx errors

It’s important to fix 5xx errors as soon as you are aware of them. Here are some of the best steps to take in order to fix 5xx issues:

Reload the page:  Reloading the page usually works if the server issue is only temporary. You can do this by hitting F5 or Ctrl-R on your keyboard or submitting the URL again from the address bar in your browser. 

Clear cookies: Occasionally, 5xx issues are caused by cookies associated with the website; so, clearing these cookies and reloading the browser may fix the issue. Enter the History section of your web browser and choose Delete to do this. You might need to check the box next to Cookies before pressing the delete key on some devices in order to clear the cookies. Check to see whether the error code still appears after refreshing your page.

Rollback your site: If you have just made any updates or changes to the site, roll them back and see if that fixes the problem (this can often be the case if a newly updated plugin is incompatible with your CMS).

Contact your host: If all of the above fails, it may be time to call in the troops. Contact your developer or create a support ticket with your hosting provider so that they can help you resolve the server issue. 

 

Final thoughts on 5xx errors

Every website is bound to run into 5xx errors at some point, whether they are temporary or more serious. For anyone who wants to ensure a good user experience and avoid traffic losses from organic search, it’s worth understanding how to monitor your site for server errors so that you can take the action needed to fix them when they happen — before they have a chance to negatively impact your website’s health and performance.

To learn more about these server errors, check out Lumar’s SEO Office Hours notes on 5xx errors.


Website Health Article Series - HTTP Status Code Availability Topic

This article is part of Lumar’s educational series on Website Health. In this series, we are diving deep into each of the 7 categories of the SEO-Revenue Funnel to help digital marketing teams learn more about the many elements of search engine optimization that contribute to a high-performing, healthy website.

Learn more about HTTP status code availability in our other Availability posts, and in our guide to HTTP status codes, as well as in our SEO Office Hours library collections for 3xx redirects, 4xx errors, and 5xx errors.

Start building better online experiences today

Lumar is the intelligence & automation platform behind revenue-driving websites

Avatar image for Jenny Abouobaia
Jenny Abouobaia

SEO Consultant & Lumar Contributing Writer

Jenny is a contributing writer for Lumar, an SEO consultant, and a regular speaker at search industry events. Her specialties include E-A-T best practices to build topical authority and the interplay between digital PR and organic search. She was recently commended as SEO Consultant of the Year in the 2021 MENA Search Awards.

Newsletter

Get the best digital marketing & SEO insights, straight to your inbox