Faceted navigation is the set of filters on a shop or listing page, such as colour, size and price. Each combination can create a new URL, so a small catalogue can turn into thousands of near-identical pages. Google suggests blocking filter URLs you do not need in search with robots.txt, and keeping the rest consistent.
| Approach | What it does | Good for |
|---|---|---|
| robots.txt disallow | Stops Google crawling those filter URLs | Filters nobody searches for: sort, price slider, in stock |
| URL fragments (#) | Filters live after the #; Google generally does not crawl fragments | New builds where you control the code |
| rel=canonical | Points filtered pages to the main category; may reduce crawling of them | Filtered views close to the category |
| rel=nofollow on filter links | Hints not to follow; must be on every link to that URL | A supporting measure only |
| Keep indexable | Consistent URL, & separator, fixed order, 404 when empty | Filters people do search for, like a colour or brand |
Updated
Filters are good for shoppers and risky for crawling. Google calls the result an infinite URL space and says it leads to overcrawling and slower discovery of new pages. Use the planner below to list your filters, mark the ones worth a search result, and get robots.txt lines for the rest.
Category path: /collections/dresses
Filter parameter names, one per line
color
size
sort
price
in_stockURL shape for kept filters (fixed order)
/collections/dresses?color={value}robots.txt lines for the other filters
User-agent: *
Disallow: /*?size=
Disallow: /*&size=
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?price=
Disallow: /*&price=
Disallow: /*?in_stock=
Disallow: /*&in_stock=Copy robots.txt lines
| Filter | Usually | Reason |
|---|---|---|
| Brand | Keep | People search for brand plus product type |
| Colour | Keep for main colours | Searches like black boots are common |
| Size | Block | Results change with stock; few size-only searches |
| Price range | Block | Endless combinations, little search demand |
| Sort order | Block | Same products, different order |
| In stock only | Block | Temporary, changes daily |
The * wildcard matches any path. Two lines per parameter catch it whether it comes first after ? or later after &.
User-agent: *
Disallow: /*?sort=
Disallow: /*&sort=
Disallow: /*?price=
Disallow: /*&price=In the head of /collections/dresses?size=m. Only use this on URLs you have not blocked in robots.txt, so Google can read it.
<link rel="canonical" href="https://example.com/collections/dresses">Take a shop with 6 colours, 5 sizes and 4 sort orders on one category. Picking one of each alone gives 6 x 5 x 4 = 120 URLs, before price ranges or multiple selections. Across 20 categories that is 2,400 addresses for what is really 20 pages.
Search engines have to spend time on each one before they can decide it is a duplicate, and new products wait longer to be found.
Your platform may already ship rules. Open yoursite.com/robots.txt and look for lines that mention filter or sort parameters, and view the source of a filtered page to see whether it already carries a canonical tag, before adding anything of your own.
It is the filter panel on a category or search page that lets visitors narrow results by properties such as size, colour, brand or price. Each filter usually adds a parameter to the URL, like ?color=red&size=m.
Mostly no. Keep a filter crawlable only when it matches a real search with demand, and make sure that page has unique, useful content. Sort orders, price sliders and stock toggles almost never need a search result of their own.
Google's faceted navigation page lists robots.txt first for filter URLs you do not need in search, because it stops the crawling itself. A canonical tag still lets Google crawl the page, and Google describes it as possibly decreasing, not stopping, crawling of the other versions.
Yes. Google asks that the logical order of filters always stays the same and that no duplicate filters can exist. Otherwise ?color=red&size=m and ?size=m&color=red are two URLs for one page.
Related: Robots.txt generator, Robots.txt tester, Canonical tag checker, Ecommerce SEO guide, Keyword cannibalization.
Start for free or see all free tools. We.Inc is free to start, with no card needed. AI building is on paid plans from $20 a month.
We.Inc is an AI-powered website builder you can resell under your own brand. Launch a branded client dashboard, bill on Stripe Connect, and deliver AI-generated websites in minutes. White-label plans start at $99 a month for 25 client sites, with a 7-day free trial and no per-site fees.