URL encoding explained: percent-encoding, + vs %20, and the reserved characters

A URL can only contain a small, safe set of characters, so everything else must be percent-encoded — but the rules change between the path, the query and form submissions. This guide maps the territory: which characters are reserved, when a space is + and when it is %20, and how UTF-8 does the heavy lifting underneath.

Try putting a search for fish & chips? into a URL by hand and you will meet URL encoding whether you planned to or not: the & starts a new parameter, the ? was already spent starting the query, and the spaces are illegal outright. Percent-encoding is the escape system that lets arbitrary text live inside a URL — each unsafe byte written as % followed by two hex digits, so & becomes %26 and a space becomes %20.

The complications are real but finite: different parts of a URL play by slightly different rules, forms use a historical variant in which space is +, and non-English text passes through UTF-8 before it is encoded at all. Master those three facts and broken links become diagnosable instead of mysterious.

Anatomy: the same URL, five rulebooks

shop.example.com:8443/menu/fish%20%26%20chips?sort=price&note=hot+fresh#reviews
└──── host ────┘└port┘└────── path ──────┘└────── query ─────┘└ fragment ┘

The scheme, host and port are restrictive alphabets of their own. The path is a hierarchy of segments separated by real slashes; a slash inside a segment's data must be %2F, and many servers reject even that. The query is a list of key=value pairs joined by &, so any & or = inside a value must be encoded or it becomes structure. The fragment (after #) is never sent to the server at all — it is consumed by the browser — which is why analytics and routing behave differently around it.

The character table worth memorising

That last row causes the subtlest bugs: a value that already contains percent-encoded text, encoded again, produces double-encoding — %2520 — which decodes once to %20, not to a space. When a link looks right in the address bar but the server swears the file is not found, count the encoding layers.

Full URL vs component encoding: the JavaScript distinction that matters

JavaScript's two built-in functions embody the classic choice. encodeURI() encodes a whole URL and leaves structural characters (& ? = / : #) intact, because it assumes you are handling structure deliberately. encodeURIComponent() encodes one value and escapes those same characters, because a value must not smuggle in structure. Passing user data through encodeURI is the standard bug: a search term containing & silently amputates itself and invents a new parameter.

const q = "fish & chips";
// wrong:  ?q=fish%20&%20chips   (& splits the value)
// right:
const url = "/search?q=" + encodeURIComponent(q);
//        /search?q=fish%20%26%20chips

The same distinction exists in every stack — Python's quote versus quote_plus, and so on — and the question is always the same: am I encoding a whole URL whose structure I trust, or a value whose contents I must not trust at all?

How servers and frameworks unravel it

Servers percent-decode path and query once, automatically, before your code sees them — so application code should work with decoded values and encode only when constructing URLs, never double-handle. Form submissions arrive as application/x-www-form-urlencoded, where the body follows query rules including + for space. The mismatches to watch: a plus in a path segment is a literal plus, not a space (paths are not form data); a query built by hand with %20 works everywhere; and a redirect chain that re-encodes an already-encoded URL manufactures the double-encoding bug one hop at a time.

Mistakes and pitfalls

Encoding the entire URL you are about to request

Running a full URL — scheme, slashes, question mark and all — through a component encoder escapes the structure itself, producing a single unreadable segment. Encode the values; assemble the structure yourself.

Using + for spaces in paths and file names

/my+file.pdf names a file with a plus in it, not a space; servers will 404 while the link looks plausible. In paths, space is always %20 (better: name files with hyphens and skip the whole problem).

Decoding user input 'to be safe' before validating it

Extra decoding passes are how encoded traversal sequences like %2e%2e%2f turn back into ../ inside your application after the web server already checked the decoded-once form. Validate in the form the server gave you; do not keep unwrapping.

Common questions

Should I encode the ~ or the apostrophe?

The tilde is unreserved in modern rules and safe raw (ancient encoders escaped it unnecessarily). The apostrophe is a sub-delimiter: legal in queries, but encoding it as %27 is harmless and avoids trouble with sloppy parsers and HTML attributes — a pragmatic encode-always candidate.

Why does my URL work in the browser but fail from a script?

Browsers are forgiving mechanics: they auto-encode spaces and some unsafe characters when you type or paste a URL. HTTP libraries send exactly what you gave them. Test links in the strictest consumer you have.

Do URLs support Arabic, Chinese or emoji?

Yes, via the UTF-8 pipeline described in this article — browsers display the characters and transmit percent-encoded bytes. Internationalised domain names use a related system (punycode, the xn-- names). Long encoded strings are normal and not a sign of breakage.

To wrap up

Percent-encoding is a small system with sharp edges in predictable places: structure versus data, path versus form, once versus twice. Learn the character table and the two encoding modes and every broken link becomes a short investigation instead of a superstition.

Keep a small personal test string for any system that builds URLs — something in the spirit of: café & crème? 100% +tax. It packs an accented word, an ampersand, a question mark, a percent sign and a plus into one value. Pass it through your stack and read the final URL. If it survives as exactly one parameter, decodes back to the identical string, and the accented words arrive intact, your encoding layers are in the right order. If it does not, the shape of the failure will point at the guilty layer — which is all debugging ever is, once the test case is honest.