Indexing
In order for web pages to be included within search results, they must be in Google’s index. Search engine indexing is a complex topic and is dependent on a number of different factors. Our SEO Office Hours Notes on indexing cover a range of best practices and compile indexability advice Google has released in their Office Hours sessions to help ensure your website’s important pages are indexed by search engines.
It Will be Difficult For Google to Improve its Rendering Services at Scale
There are plans for Google to improve its rendering services, but this will take a long time as it will be very difficult to make rendering improvements at scale, especially for websites with millions of pages that each require rendering.
Use Pre-rendering For New JavaScript Frameworks so Google Can Index Them
Google uses Chrome 41 for rendering which is an old version, so it may struggle to render new frameworks such as JavaScript ES6. To get around this issue, use pre-rendering.
Ecommerce Websites Should Avoid Serving Product Page Content Via JavaScript
Large ecommerce websites should steer clear of serving product page content via JavaScript as this can delay indexing of new products by weeks, as opposed to minutes when you serve content in the HTML.
For Mobile-first, Ranking Fluctuations Are Caused by Google Recrawling and Reprocessing a Site
If a site experiences ranking fluctuations after being switched to mobile-first indexing, this is because Google will need to recrawl and reprocess the site to update the index.
Block Ads From Being Crawled to Avoid Ranking For Unintended Queries
Ads which are inline with the main text of a page can be picked up by Google as part of the content of that page. This could cause the page to rank for queries related to the text in the ad. John recommends blocking the ads from passing pagerank and using JavaScript to block them by robots.txt.
Google May Fold Together Similar or Duplicate Hreflang Versions
Google may fold together hreflang versions of a page if the content is similar or the same, as it doesn’t make sense for both versions to be indexed.
After Indexing Google Checks to See if a More Relevant Page Can Be Swapped in
After pages are added into Google’s index, it then checks if there are any different URLs that should be swapped in because they are a better fit for the user.
Hreflang Alternates Can Still be Shown in Search While Technically Non-indexable Duplicates
Pages identified as duplicates which are not technically indexed may still be shown in search if they are an alternate hreflang version which is similar to the canonical version.
Dynamic Rendering Can be Used to Show Googlebot Fully Rendered Pages
You can use dynamic rendering to serve Googlebot with pages that are already fully rendered, meaning there won’t be a gap between the initial indexing and rendering.
There Will Continue to be a Delay Between Indexing & Rendering Due to Resource Issues
John explained that for the foreseeable future there will continue to be a delay between initial indexing of HTML and rendering, because JavaScript requires resource to be rendered and this can’t happen immediately with the current system.