A user agent is any piece of software that requests web pages on someone's behalf: a browser, a search engine crawler, an app, a command-line tool. With each request it sends a "user agent string", a line of text that tells the server roughly what it is, such as Chrome on Windows, Safari on iPhone, or Googlebot.
What a user agent string looks like
The string travels in the User-Agent HTTP header. A typical desktop Chrome string reads something like Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36. It looks confusing because of history: browsers copied each other's tokens for decades so that sites would not shut them out, which is why Chrome still claims to be "Mozilla" and mentions Safari.
Browsers are now reducing how much detail the string reveals, freezing parts of it to limit fingerprinting. Chrome offers more precise information through a separate mechanism called User-Agent Client Hints, which sites have to request.
Bots identify themselves the same way
Crawlers send their own user agent. Googlebot includes "Googlebot" in its string, and Bing's crawler says "bingbot". Your robots.txt file uses these names in its User-agent: lines to give different rules to different crawlers. AI crawlers such as OpenAI's GPTBot can be addressed the same way.
The string is self-reported, so anyone can fake it. Google documents a reverse DNS lookup to confirm that a visitor claiming to be Googlebot really comes from Google.
An example from a server log
A shop owner notices thousands of hits a night in their hosting logs. The user agent column shows a scraper tool's name rather than a browser. They add a rule blocking that user agent at the firewall and the load disappears. On another site, an owner blocks everything containing "bot" and accidentally shuts out Googlebot, so pages slowly drop from search. Both cases are user agents at work.
Does a small business need to care?
Mostly no. Analytics tools read user agents for you to report devices and browsers. It becomes relevant when you read server logs, write robots.txt rules, debug a problem that only one browser has, or deal with unwanted bot traffic.
User agent mistakes
- Serving different content to Googlebot than to people. That is cloaking and breaks Google's spam policies.
- Browser sniffing for layout. Use responsive design and feature detection instead of guessing from the string.
- Trusting the string for security. It is trivial to spoof.
- Overbroad blocks that catch legitimate crawlers along with bad ones.
Related terms
- Googlebot: Google's crawler and its user agent.
- robots.txt: rules addressed to user agents by name.
- Web browser: the most common user agent.
- Crawling: what bot user agents do on your site.