Internal Site Search
Internal search offers the ability for users to search within a website and find further relevant content. While onsite search can provide insights into what users are looking for, it can also lead to some issues with search engine crawling. Within our SEO Office Hours notes below, we cover Google’s advice for implementing internal search with further recommendations and use cases.
For more on internal site search & best practices for SEO, we recommend reading: SEO Best Practices for Faceted Search (Faceted Navigation)
Internal Search Results Pages Should be Blocked Unless They Provide Unique Value
Internal search result pages should be blocked from crawling because it could overload the site’s server and they tend to be low quality. However, there may be instances where it makes sense to have these pages indexed if they provide value.
Use Query Parameters For Onsite Search
John recommends using a separate query parameter for on-site search as it’s a lot easier for Google to recognise as something that might vary. If there are additional parameters also added to the URL e.g. for pagination or filtering, utilising a query parameter makes it easier for Google to learn what the individual path parts are for as well as helping to optimize the crawling of these pages.
Googlebot Doesn’t Use Sites’ Internal Search Features to Find Pages
Googlebot doesn’t know what to search for on a site, so doesn’t use a site’s internal search for content discovery. The rare exception to this will be if a site isn’t crawlable normally and pages can only be discovered through internal search.
Internal Search Pages Are Indexed if Google Sees Them as More Useful Than Linked Pages
Internal search pages may be indexed if they’re seen as a way of finding category pages or if they form a clear navigation for Google to find related products/pages. If you don’t want them to be indexed you should make your navigation clearer.
Multiple Options for Internal Search Pages with External Links
Internal search pages are commonly blocked from indexing in robots.txt. However, if you have external links pointing to internal search pages you can block and allow URLs in a few ways to maximise link equity.
Internal Search Pages Should Not Be Indexable
Google recommends you block internal search from being indexed as will likely increase number of pages indexed for that site and can be be inefficient for crawling and indexing
Disallowing Internal Search Pages won’t Impact the Sitelinks Search Box Markup
Internal search pages on a site do not need to be crawlable for the Sitelinks Search Box markup to work. Google doesn’t differentiate desktop and mobile URLs, so you might want to set up a redirect to the mobile search pages for mobile devices.
Markup Doesn’t Make the Sitelinks Search Box Appear
If Google doesn’t show a searchlink search box, the markup won’t make it appear. It just allows searches to take place on the site instead of Google.
Sitelinks Search Box Markup Doesn’t Make the Search Box Appear
The Sitelinks Search box markup is only used when Google is already showing the Sitelinks search box, to direct Google to a specific search URL. Without the markup, Google will do a site: query in Google’s search results.
Noindex Search Results Pages and Nofollow Search Navigation
You should noindex your own search results pages as they are generally lower quality, but you can disallow them if they are causing a problem with the server load from being crawled. You can nofollow links on the search navigation to prevent crawling but you should probably have followed links on the results pages which can be useful for finding new pages if they are crawled.