Canonical URL grouper
Four URLs, one page: with and without www, with and without a trailing slash, plus a version carrying ?utm_source=. Each rule you toggle here decides whether two URLs count as the same page, and the tool groups the list accordingly, picks the canonical you would have picked, and gives you the duplicate → canonical pairs ready for a redirect map.
Summary
How to use
- Paste the list — a sitemap export, a crawl, a set of URLs from an analytics report, or the log of pages that received traffic.
- Turn the equivalence rules on and off. The defaults match how search engines actually treat URLs; the two off by default are the ones that are only sometimes true.
- Read the table. Each row is a group of URLs that collapse to one normalized key, with the canonical the tool would keep and the duplicates listed.
- The Summary tells you how many URLs could be removed — that number is the size of your duplicate-content problem.
- Copy duplicate → canonical pairs gives you two tab-separated columns, which is exactly the input the redirect map generator wants.
Which rules are safe to assume
Almost always safe. Host case (DNS is case-insensitive), the port when it is the scheme default, the fragment (never sent to the server), and parameter order (servers parse into a map). Tracking parameters too: utm_*, gclid, fbclid and friends never change what page is served.
Safe if your site is configured correctly, and worth verifying. http versus https, and www versus non-www — both should be a single 301 to one canonical form. Trailing slash is the same story: /page and /page/ are technically different URLs and a server may legitimately serve different content, but on virtually every real site they are one page, and one of them should redirect.
Off by default because it is genuinely site-dependent. Path case: on Linux servers paths are case-sensitive, so /About and /about can be two different pages — or a 404 and a page. Only turn it on when you know your routing lowercases. And ignoring all query parameters is a blunt instrument: it will happily group ?product=1 with ?product=2, which are different pages. It is useful for a first pass on a URL list from analytics, not for generating redirects.
What to do with the groups
Duplicates are not a penalty, but they do split signals: links pointing at four variants of a page each carry a quarter of the authority, and search engines have to guess which version to index. The fix, in order of preference: make the duplicates 301-redirect to the canonical, or if they must stay reachable, put a rel="canonical" on each pointing at the canonical version. Parameter variants that only add tracking do not need redirects — a self-referencing canonical on the page is enough, since the parameter does not change the content.
The canonical this tool picks is the one a human would: HTTPS over HTTP, no query string over one with parameters, non-www over www, and the shortest as a tiebreak. If your site's convention differs — plenty of sites are canonically www — pick the other member of the group; the grouping is the useful output, and the suggestion is a default.
FAQ
Is this the same as the URL sorter and deduper?
Related but different in output. The sorter / deduper gives you a clean deduplicated list — it throws the duplicates away. This one keeps them and shows you the groups, because the mapping from duplicate to canonical is what you need for redirects and for understanding how the duplicates arose.
Why is a URL with a different parameter value grouped with another?
Only if Ignore all query parameters is on — that setting deliberately ignores values. Turn it off and use Ignore tracking parameters, which only drops parameters known not to change the content.
Can it tell me which variant search engines have indexed?
No — that needs Search Console or a live query, and neither is reachable from a browser tab without your credentials. This works on the list you supply.
How do I add my own tracking parameters to ignore?
The built-in list covers utm_*, the major click IDs, and the common email and analytics prefixes. For custom parameters, strip them first with the param keep / drop tool, then group the result here.