Indexing
In order for web pages to be included within search results, they must be in Google’s index. Search engine indexing is a complex topic and is dependent on a number of different factors. Our SEO Office Hours Notes on indexing cover a range of best practices and compile indexability advice Google has released in their Office Hours sessions to help ensure your website’s important pages are indexed by search engines.
Soft 404 Pages May Be Indexed then Later Dropped Out of the Index
Google initially indexes pages which then might be classified as Soft 404 pages, and then drops them from the index when they have processed the content.
Sitemap Errors don’t Impact Rankings but can Slow Down Indexing
Sitemaps help Google improve crawling and indexing of sites. If a sitemap can’t be properly processed, Google may take longer to index pages as have to rely on normal crawling and indexing to find those pages.
PubSubHubbub is the fastest way to get content into Google
RSS feeds with PubSubHubbub are a quickest way to get content updated in Google.
301 and 302 Redirects Only Determine Which URL is Indexed
A 301 indicates to Google that the destination URL should be indexed, a 302 indicates to Google that the original URL should be indexed but they always use the content on the destination page.
Google Chooses the HTTPS Version of a Page if Both Exist
Google will choose the HTTPS version of a page instead of HTTP when both exist, but there is no change in ranking position.
Contact Google for Sites Incorrectly Filtered by Safe Search
If your website is incorrectly being filtered by safe search, you can submit a form in the help centre to ask Google to reclasify it.
URL Parameters Help Crawling and Indexing
URL parameters in URLs make it easier for Google to understand URLs for crawling and indexing. If you put everything into the path of the URL it can be harder for Google to crawl them properly.
Disallowed URLs may Show in Search Due to Internal Linking
If you’re seeing disallowed URLs showing up in search results, it may be because of internal linking to these pages, so you should change links to point to pages you want to be shown.
Sitemap Index Counts Report the Exact Submitted URLs
Sitemap Index counts report the exact URL you sumbit including trailing slashes. If Google chooses to index a different copy of the same page, the submitted URL wouldn’t be reported as indexed.
Cross Linking Deep Pages Can Help Indexing
Cross linking between deep pages linked through paginated category pages can help them be discovered more easily.