In-depth guide
URL, URI and percent-encoding
How URI and URL components are structured, which characters carry syntax and when percent-encoding is required.
URI, URL and components
URI is the general term for resource identifiers; URL describes URIs that also provide a means or location of access. A web URL is commonly decomposed into scheme, authority, path, query and fragment, each with different rules.
Component boundaries matter because the same character may be syntax in one position and data in another. Blindly encoding a complete URL can therefore destroy delimiters that are required for parsing.
Reserved and unreserved characters
RFC 3986 distinguishes unreserved characters, which can normally appear literally, from reserved characters that may act as delimiters. ASCII letters, digits, hyphen, dot, underscore and tilde are unreserved.
Characters such as :, /, ?, #, [, ], @ and sub-delimiters may carry structure. Whether they should be encoded as literal data depends on the specific component in which they appear.
From Unicode text to UTF-8 bytes
Percent-encoding represents bytes: each byte becomes % followed by two hexadecimal digits. A non-ASCII Unicode character is first encoded as UTF-8, often yielding multiple bytes and therefore multiple %HH triplets.
This is why one visible character can expand substantially. Correct decoding reconstructs bytes first and then interprets them using the agreed character encoding, normally UTF-8.
Path, query and form encoding
In a path, slashes and segments have structural meaning. In queries, ampersand and equals are widely used by application conventions to separate name/value pairs, although URI syntax itself does not impose one universal query data model.
application/x-www-form-urlencoded adds a separate convention in which spaces are commonly represented by + and a literal plus must be encoded. It should not be confused with generic URI percent-encoding.
Normalization and equivalence
Hexadecimal digits in percent-encoded triplets are case-insensitive, and encoded unreserved characters can often be normalized to their literal form. Still, textually different URLs are not automatically equivalent in every application.
Host normalization, default ports, paths and query treatment depend on protocol and server behavior. Avoid aggressive rewriting when exact resource identity or a cryptographic signature depends on the original representation.
Decoding errors and security
Incomplete percent sequences, non-hex digits or byte sequences that are invalid UTF-8 should be handled explicitly. Overly permissive decoders can make proxies, frameworks and applications disagree about the same request target.
Canonicalization differences can contribute to routing, cache or validation bypasses. Decode at the correct layer, avoid repeated decoding and validate the resulting component according to its semantics.
