URL ( PR #763 )

Goals

The URL standard takes the following approach towards making URLs fully interoperable:

• Align RFC 3986 and RFC 3987 with contemporary implementations and obsolete the RFCs in the process. (E.g., spaces, other "illegal" code points, query encoding, equality, canonicalization, are all concepts not entirely shared, or defined.) URL parsing needs to become as solid as HTML parsing. [RFC3986] [RFC3987]

• Standardize on the term URL. URI and IRI are just confusing. In practice a single algorithm is used for both so keeping them distinct is not helping anyone. URL also easily wins the search result popularity contest .

• Supplanting Origin of a URI [sic] . [RFC6454]

• Define URL’s existing JavaScript API in full detail and add enhancements to make it easier to work with. Add a new URL object as well for URL manipulation without usage of HTML elements. (Useful for JavaScript worker environments.)

• Ensure the combination of parser, serializer, and API guarantee idempotence. For example, a non-failure result of a parse-then-serialize operation will not change with any further parse-then-serialize operations applied to it. Similarly, manipulating a non-failure result through the API will not change from applying any number of serialize-then-parse operations to it.

As the editors learn more about the subject matter the goals might increase in scope somewhat.

1. Infrastructure

This specification depends on Infra . [INFRA]

Some terms used in this specification are defined in the following standards and specifications:

• Encoding [ENCODING]
• File API [FILEAPI]
• HTML [HTML]
• Unicode IDNA Compatibility Processing [UTS46]
• Web IDL [WEBIDL]

To serialize an integer , represent it as the shortest possible decimal number.

1.1. Writing

A validation error indicates a mismatch between input and valid input. User agents, especially conformance checkers, are encouraged to report them somewhere.

const

url

=


new
 URL


URL


(


"https:foo.html"

,


"https://example.org/"


);


· 

const

url

=


new
 URL


URL


(


"💩"


);


Input’s 

const

url

=


new
 URL


URL


(


"💩"

,


"mailto:user@example.org"


);


Yes 

const

url

=


new
 URL


URL


(


"/c:/path/to/file"

,


"file:///c:/"


);


· 

1.2. 1.2. Parsers The EOF code point A

The EOF code point is a conceptual code point that signifies the end of a string or code point stream.

A pointer for a string input code point is an integer that points to a code point within input . Initially it points to the start of input . If it is −1 it points nowhere. If it is greater than or equal to input ’s code point length EOF code point When a ’s code point length , it points to the EOF code point .

When a pointer is used, c code point references the code point the pointer points to as long as it does not point nowhere. When the pointer points to nowhere c When a cannot be used.

When a pointer is used, remaining code point substring references the code point substring from the pointer + 1 to the end of the string, as long as c EOF code point is not the EOF code point . When c EOF code point is the EOF code point remaining cannot be used.

If " If " mailto:username@example " is a " is a string being processed and a pointer points to @, c is U+0040 (@) and remaining is " is " example ". ".

If the empty string is being processed and a If the empty string is being processed and a pointer points to the start and is then decreased by 1, using c or remaining would be an error.

1.3. 1.3. Percent-encoded bytes Percent-encoded bytes A percent-encoded byte ASCII hex digits It is generally a good idea for sequences of percent-encoded bytes

A percent-encoded byte is U+0025 (%), followed by two ASCII hex digits .

It is generally a good idea for sequences of percent-encoded bytes to be such that, when percent-decoded UTF-8 decode without BOM or fail percent-encoded bytes host parser URL rendering percent-encoded bytes and then passed to UTF-8 decode without BOM or fail , they do not end up as failure. How important this is depends on where the percent-encoded bytes are used. E.g., for the host parser not following this advice is fatal, whereas for URL rendering the percent-encoded bytes would not be rendered percent-decoded To .

The C0 control percent-encode set C0 controls code points The fragment percent-encode set C0 control percent-encode set The query percent-encode set C0 control percent-encode set The query percent-encode set fragment percent-encode set The special-query percent-encode set query percent-encode set The path percent-encode set are the C0 controls and all code points greater than U+007E (~).

The fragment percent-encode set is the C0 control percent-encode set and U+0020 SPACE, U+0022 ("), U+003C (<), U+003E (>), and U+0060 (`).

The query percent-encode set is the C0 control percent-encode set and U+0020 SPACE, U+0022 ("), U+0023 (#), U+003C (<), and U+003E (>).

The query percent-encode set cannot be defined in terms of the fragment percent-encode set due to the omission of U+0060 (`).

The special-query percent-encode set is the query percent-encode set and U+0027 (').

The path percent-encode set query percent-encode set The userinfo percent-encode set is the query percent-encode set and U+003F (?), U+0060 (`), U+007B ({), and U+007D (}).

The userinfo percent-encode set path percent-encode set The component percent-encode set userinfo percent-encode set This is used by is the path percent-encode set and U+002F (/), U+003A (:), U+003B (;), U+003D (=), U+0040 (@), U+005B ([) to U+005E (^), inclusive, and U+007C (|).

The component percent-encode set is the userinfo percent-encode set and U+0024 ($) to U+0026 (&), inclusive, U+002B (+), and U+002C (,).

This is used by HTML for registerProtocolHandler() , and could also be used by other standards to percent-encode data that can then be embedded in a URL ’s ’s path , query , or fragment ; or in an opaque host UTF-8 percent-encode ; or in an opaque host . Using it with UTF-8 percent-encode gives identical results to JavaScript’s encodeURIComponent() [sic] [sic] . [HTML] [ECMA-262] The

The application/x-www-form-urlencoded percent-encode set component percent-encode set The percent-encode set is the component percent-encode set and U+0021 (!), U+0027 (') to U+0029 RIGHT PARENTHESIS, inclusive, and U+007E (~).

The application/x-www-form-urlencoded percent-encode set ASCII alphanumeric To percent-encode after encoding percent-encode set contains all code points, except the ASCII alphanumeric , U+002A (*), U+002D (-), U+002E (.), and U+005F (_).

2. 2. Security considerations Security considerations The security of a

The security of a URL is a function of its environment. Care is to be taken when rendering, interpreting, and passing URLs When rendering and allocating new around.

When rendering and allocating new URLs "spoofing" needs to be considered. An attack whereby one host or URL code points can be confused for another. For instance, consider how 1/l/I, m/rn/rri, 0/O, and а/a can all appear eerily similar. Or worse, consider how U+202A LEFT-TO-RIGHT EMBEDDING and similar code points are invisible. [UTR36] When passing a

When passing a URL from party A to B , both need to carefully consider what is happening. A might end up leaking data it does not want to leak. B might receive input it did not expect and take an action that harms the user. In particular, B should never trust A , as at some point URLs from A can come from untrusted sources.

3. 3. Hosts (domains and IP addresses) Hosts (domains and IP addresses) At a high level, a

At a high level, a host valid host string host parser host serializer The host parser scalar value string , valid host A string , host A valid host string validation error host parser The host serializer parser , and host ASCII string serializer relate as follows:

• The host parser takes an arbitrary scalar value string and returns either failure or a host .

• A host can be seen as the in-memory representation.

• A valid host string defines what input would not trigger a validation error or failure when given to the host parser . I.e., input that would be considered conforming or valid.

• The host serializer takes a host and returns an ASCII string . (If that string is then parsed , the result will equal the host that was serialized .)

3.1. 3.1. Host representation Host representation A

A host is a domain IP address opaque host empty host , an IP address , an opaque host , or an empty host . Typically a host serves as a network address, but it is sometimes used as opaque identifier in URLs where a network address is not necessary.

A typical A typical URL whose host opaque host is an opaque host is git://github.com/whatwg/url.git The RFCs referenced in the paragraphs below are for informative purposes only. They have no influence on .

The RFCs referenced in the paragraphs below are for informative purposes only. They have no influence on host A writing, parsing, and serialization. Unless stated otherwise in the sections that follow.

A domain ASCII string is a non-empty ASCII string that identifies a realm within a network. [RFC1034] The domain labels

The domain labels of a domain strictly splitting domain The are the result of strictly splitting domain on U+002E (.).

The example.com and example.com. domains An IP address IPv4 address IPv6 address An IPv4 address are not equivalent and typically treated as distinct.

An IP address is an IPv4 address or an IPv6 address .

An IPv4 address is a 32-bit unsigned integer that identifies a network address. [RFC791] An IPv6 address

An IPv6 address is a 128-bit unsigned integer that identifies a network address. For the purposes of this standard it is represented as a list IPv6 pieces of eight 16-bit unsigned integers, also known as IPv6 pieces . [RFC4291] Support for

Support for <zone_id> intentionally omitted An opaque host ASCII string An empty host is intentionally omitted .

An opaque host is a non-empty ASCII string that can be used for further processing.

An empty host is the empty string.

3.2. 3.2. Host miscellaneous Host miscellaneous A forbidden host code point A forbidden domain code point forbidden host code point C0 control To obtain the public suffix

A forbidden host code point is U+0000 NULL, U+0009 TAB, U+000A LF, U+000D CR, U+0020 SPACE, U+0023 (#), U+002F (/), U+003A (:), U+003C (<), U+003E (>), U+003F (?), U+0040 (@), U+005B ([), U+005C (\), U+005D (]), U+005E (^), or U+007C (|).

A forbidden domain code point is a forbidden host code point , a C0 control , U+0025 (%), or U+007F DELETE.

Specifications should prefer the origin concept for security decisions. The notion of " concept for security decisions. The notion of " public suffix public suffix " and " " and " registrable domain registrable domain " cannot be relied-upon to provide a hard security boundary, as the public suffix list will diverge from client to client. Specifications which ignore this advice are encouraged to carefully consider whether URLs' schemes ought to be incorporated into any decisions made, i.e. whether to use the same site schemelessly same site " cannot be relied-upon to provide a hard security boundary, as the public suffix list will diverge from client to client. Specifications which ignore this advice are encouraged to carefully consider whether URLs' schemes ought to be incorporated into any decisions made, i.e. whether to use the same site or schemelessly same site concepts.

3.3. 3.3. IDNA The domain to ASCII

The domain to ASCII algorithm, given a string domain and a boolean beStrict Let , runs these steps:

1. Let result Unicode ToASCII be the result of running Unicode ToASCII with domain_name set to domain , UseSTD3ASCIIRules set to beStrict , CheckHyphens set to false, CheckBidi set to true, CheckJoiners set to true, Transitional_Processing set to false, and VerifyDnsLength set to beStrict . [UTS46] If

   If beStrict is false, domain ASCII string strictly splitting is an ASCII string , and strictly splitting domain on U+002E (.) does not produce any item starts with ASCII case-insensitive match for " that starts with an ASCII case-insensitive match for " xn-- ", this step is equivalent to ASCII lowercasing ", this step is equivalent to ASCII lowercasing domain If .

2. If result is a failure value, domain-to-ASCII validation error If validation error , return failure.

3. If result is the empty string, domain-to-ASCII validation error Return validation error , return failure.

4. Return result This document and the web platform at large use Unicode IDNA Compatibility Processing .

This document and the web platform at large use Unicode IDNA Compatibility Processing and not IDNA2008. For instance, ☕.example becomes xn--53h.example and not failure. [UTS46] [RFC5890] The domain to Unicode

The domain to Unicode algorithm, given a domain domain and a boolean beStrict Let , runs these steps:

1. Let result Unicode ToUnicode be the result of running Unicode ToUnicode with domain_name set to domain , CheckHyphens set to false, CheckBidi set to true, CheckJoiners set to true, UseSTD3ASCIIRules set to beStrict , and Transitional_Processing set to false. [UTS46] Signify

2. Signify domain-to-Unicode validation errors validation errors for any returned errors, and then, return result .

3.4. 3.4. Host writing Host writing A valid host string

A valid host string valid domain string valid IPv4-address string valid IPv6-address string A must be a valid domain valid domain Let string , a valid IPv4-address string , or: U+005B ([), followed by a valid IPv6-address string , followed by U+005D (]).

A domain is a valid domain if these steps return success:

1. Let result domain to ASCII be the result of running domain If to ASCII with domain and true.

2. If result Set is failure, then return failure.

3. Set result domain to Unicode to the result If of running domain to Unicode with result Return success. Ideally we define this in terms of a sequence of code points that make up a valid domain issue 245 A valid domain string and true.

4. If result contains any errors, return failure.

5. Return success.

Ideally we define this in terms of a sequence of code points that make up a valid domain rather than through a whack-a-mole: issue 245 .

A valid domain string valid domain A valid IPv4-address string must be a string that is a valid domain .

A valid IPv4-address string ASCII digits A valid IPv6-address string must be four shortest possible strings of ASCII digits , representing a decimal number in the range 0 to 255, inclusive, separated from each other by U+002E (.).

A valid IPv6-address string "Text Representation of Addresses" chapter of IP Version 6 Addressing Architecture is defined in the "Text Representation of Addresses" chapter of IP Version 6 Addressing Architecture . [RFC4291] A valid opaque-host string one or more URL units forbidden host code points U+005B ([), followed by a valid IPv6-address string This is not part of the definition of valid host string

A valid opaque-host string must be one of the following:

• one or more URL units excluding forbidden host code points

• U+005B ([), followed by a valid IPv6-address string , followed by U+005D (]).

This is not part of the definition of valid host string as it requires context to be distinguished.

3.5. 3.5. Host parsing Host parsing The host parser scalar value string

3.6. 3.6. Host serializing Host serializing The host serializer

The host serializer takes a host ASCII string If host IPv4 address IPv4 serializer and then runs these steps. They return an ASCII string .

1. If host Otherwise, if is an IPv4 address , return the result of running the IPv4 serializer on host IPv6 address IPv6 serializer .

2. Otherwise, if host Otherwise, is an IPv6 address , return U+005B ([), followed by the result of running the IPv6 serializer on host , followed by U+005D (]).

3. Otherwise, host is a domain opaque host empty host , opaque host The IPv4 serializer IPv4 address , or empty host , return host .

The IPv4 serializer takes an IPv4 address ASCII string address and then runs these steps. They return an ASCII string . Let

1. Let output Let be the empty string.

2. Let n be the value of address .

3. For each For each i Prepend in the range 1 to 4, inclusive:

   1. Prepend n % 256, serialized , to output If .

   2. If i is not 4, then prepend U+002E (.) to output Set .

   3. Set n to floor( to floor( n Return / 256).

4. Return output The IPv6 serializer IPv6 address .

The IPv6 serializer takes an IPv6 address ASCII string Let address and then runs these steps. They return an ASCII string .

1. Let output Let be the empty string.

2. Let compress IPv6 piece be an index to the first IPv6 piece in the first longest sequences of address ’s IPv6 pieces ’s IPv6 pieces that are 0.

   In In 0:f:0:0:f:f:0:0 If there is no sequence of it would point to the second 0.

3. If there is no sequence of address ’s IPv6 pieces ’s IPv6 pieces that are 0 that is longer than 1, then set compress Let to null.

4. Let ignore0 be false.

5. For each For each pieceIndex If in the range 0 to 7, inclusive:

   1. If ignore0 is true and address [ pieceIndex ] is 0, then ] is 0, then continue Otherwise, if .

   2. Otherwise, if ignore0 is true, set ignore0 If to false.

   3. If compress is pieceIndex Let , then:

      1. Let separator be " be " :: " if " if pieceIndex Append is 0, and U+003A (:) otherwise.

      2. Append separator to output Set .

      3. Set ignore0 to true and continue Append .

   4. Append address [ pieceIndex ], represented as the shortest possible lowercase hexadecimal number, to ], represented as the shortest possible lowercase hexadecimal number, to output If .

   5. If pieceIndex is not 7, then append U+003A (:) to output Return .

6. Return output This algorithm requires the recommendation from A Recommendation for IPv6 Address Text Representation. .

This algorithm requires the recommendation from A Recommendation for IPv6 Address Text Representation. [RFC5952]

3.7. 3.7. Host equivalence Host equivalence To determine whether a

To determine whether a host A equals host B , return true if A is B Certificate comparison requires a host equivalence check that ignores the trailing dot of a domain (if any). However, those hosts have also various other facets enforced, such as DNS length, that are not enforced here, as URLs do not enforce them. If anyone has a good suggestion for how to bring these two closer together, or what a good unified model would be, please file an issue. , and false otherwise.

Certificate comparison requires a host equivalence check that ignores the trailing dot of a domain (if any). However, those hosts have also various other facets enforced, such as DNS length, that are not enforced here, as URLs do not enforce them. If anyone has a good suggestion for how to bring these two closer together, or what a good unified model would be, please file an issue.

4. 4. URLs At a high level, a

At a high level, a URL valid URL string URL parser URL serializer The URL parser scalar value string , valid URL validation errors A string , URL A valid URL string validation error URL parser The URL serializer parser , and URL ASCII string serializer relate as follows:

• The URL parser takes an arbitrary scalar value string and returns either failure or a URL . It might also record zero or more validation errors .

• A URL can be seen as the in-memory representation.

• A valid URL string defines what input would not trigger a validation error or failure when given to the URL parser . I.e., input that would be considered conforming or valid.

• The URL serializer takes a URL and returns an ASCII string . (If that string is then parsed , the result will equal the URL that was serialized URL serializer valid URL string .) The output of the URL serializer is not always a valid URL string .

4.1. 4.1. URL representation URL representation A

A URL is a struct valid URL string URL record A that represents a universal identifier. To disambiguate from a valid URL string ’s it can also be referred to as a URL record .

A URL ’s scheme ASCII string is an ASCII string that identifies the type of URL and can be used to dispatch a URL for further processing after parsing A . It is initially the empty string.

A URL ’s ’s username ASCII string A is an ASCII string identifying a username. It is initially the empty string.

A URL ’s ’s password ASCII string A is an ASCII string identifying a password. It is initially the empty string.

A URL ’s ’s host is null or a host The following table lists allowed . It is initially null.

A URL ’s ’s port A is either null or a 16-bit unsigned integer that identifies a networking port. It is initially null.

A URL ’s ’s path URL path segment is either a URL path segment or a list URL path segments A of zero or more URL path segments , usually identifying a location. It is initially « ».

A special URL ’s ’s path is always a list , i.e., it is never opaque A .

A URL ’s ’s query ASCII string A is either null or an ASCII string . It is initially null.

A URL ’s ’s fragment ASCII string is either null or an ASCII string that can be used for further processing on the resource the URL ’s other components identify. It is initially null. A ’s other components identify. It is initially null.

A URL blob URL entry blob URL entry This is used to support caching the object a " also has an associated blob URL entry that is either null or a blob URL entry . It is initially null.

This is used to support caching the object a " blob " URL refers to as well as its origin. It is important that these are cached as the " URL blob URL store refers to as well as its origin. It is important that these are cached as the URL might be removed from the blob URL store between parsing and fetching, while fetching will still need to succeed.

A URL path segment is an ASCII string . It commonly refers to a directory or a file, but has no predefined meaning.

A single-dot URL path segment URL path segment that is " is a URL path segment that is " . " or an ASCII case-insensitive match for " " or an ASCII case-insensitive match for " %2e ". A double-dot URL path segment ".

A double-dot URL path segment URL path segment that is " is a URL path segment that is " .. " or an ASCII case-insensitive match for " " or an ASCII case-insensitive match for " .%2e ", " ", " %2e. ", or " ", or " %2e%2e ". ".

4.2. 4.2. URL miscellaneous URL miscellaneous A special scheme ASCII string default port special scheme default port ASCII string

A special scheme is an ASCII string that is listed in the first column of the following table. The default port for a special scheme is listed in the second column on the same row. The default port for any other ASCII string is null.

A URL is special is special if its scheme special scheme is a special scheme . A URL is not special is not special if its scheme special scheme A is not a special scheme .

A URL includes credentials includes credentials if its username or password A is not the empty string.

A URL opaque path has an opaque path URL path segment A if its path is a URL cannot have a username/password/port path segment .

A URL cannot have a username/password/port if its host is null or the empty string, or its scheme is " is " file ". A ".

A URL base URL A base URL URL parser relative-URL string A Windows drive letter ASCII alpha A normalized Windows drive letter Windows drive letter As per the URL writing normalized Windows drive letter A string starts with a Windows drive letter its can be designated as base URL .

A base URL is useful for the URL parser when the input might be a relative-URL string .

A Windows drive letter is two code points, of which the first is an ASCII alpha and the second is either U+003A (:) or U+007C (|).

A normalized Windows drive letter is a Windows drive letter of which the second code point is U+003A (:).

As per the URL writing section, only a normalized Windows drive letter is conforming.

A string starts with a Windows drive letter if all of the following are true:

• its length its first two code points are a Windows drive letter its is greater than or equal to 2
• its first two code points are a Windows drive letter
• its length is 2 or its third code point is U+002F (/), U+005C (\), U+003F (?), or U+0023 (#).

To shorten a url ’s path ’s path :

1. Assert : url opaque path Let does not have an opaque path .

2. Let path be url ’s ’s path If .

3. If url ’s ’s scheme is " is " file ", ", path ’s ’s size is 1, and path [0] is a normalized Windows drive letter [0] is a normalized Windows drive letter , then return.

4. Remove path ’s last item, if any. ’s last item, if any.

4.3. 4.3. URL writing URL writing A valid URL string

A valid URL string relative-URL-with-fragment string absolute-URL-with-fragment string An absolute-URL-with-fragment string must be either a relative-URL-with-fragment string or an absolute-URL-with-fragment string .

An absolute-URL-with-fragment string absolute-URL string URL-fragment string An absolute-URL string must be an absolute-URL string , optionally followed by U+0023 (#) and a URL-fragment string .

An absolute-URL string a URL-scheme string ASCII case-insensitive special scheme ASCII case-insensitive match for " must be one of the following:

• a URL-scheme string that is an ASCII case-insensitive match for a special scheme and not an ASCII case-insensitive match for " file ", followed by U+003A (:) and a scheme-relative-special-URL string a URL-scheme string ", followed by U+003A (:) and a scheme-relative-special-URL string

• a URL-scheme string that is not ASCII case-insensitive special scheme relative-URL string a URL-scheme string ASCII case-insensitive match for " an ASCII case-insensitive match for a special scheme , followed by U+003A (:) and a relative-URL string

• a URL-scheme string that is an ASCII case-insensitive match for " file ", followed by U+003A (:) and a scheme-relative-file-URL string any optionally followed by U+003F (?) and a URL-query string A URL-scheme string ", followed by U+003A (:) and a scheme-relative-file-URL string

any optionally followed by U+003F (?) and a URL-query string .

A URL-scheme string ASCII alpha ASCII alphanumeric must be one ASCII alpha , followed by zero or more of ASCII alphanumeric , U+002B (+), U+002D (-), and U+002E (.). Schemes IANA URI [sic] Schemes should be registered in the IANA URI [sic] Schemes registry. [IANA-URI-SCHEMES] [RFC7595] A relative-URL-with-fragment string

A relative-URL-with-fragment string relative-URL string URL-fragment string A relative-URL string must be a relative-URL string , optionally followed by U+0023 (#) and a URL-fragment string .

A relative-URL string base URL must be one of the following, switching on base URL ’s ’s scheme A special scheme that is not " :

A special scheme that is not " file " a scheme-relative-special-URL string a path-absolute-URL string a path-relative-scheme-less-URL string "

a scheme-relative-special-URL string

a path-absolute-URL string

a path-relative-scheme-less-URL string

" file " a scheme-relative-file-URL string a path-absolute-URL string base URL "

a scheme-relative-file-URL string ’s

a path-absolute-URL string if base URL ’s host empty host a path-absolute-non-Windows-file-URL string base URL ’s is an empty host empty host a path-relative-scheme-less-URL string Otherwise a scheme-relative-URL string a path-absolute-URL string a path-relative-scheme-less-URL string any optionally followed by U+003F (?) and a URL-query string A non-null base URL

a path-absolute-non-Windows-file-URL string if base URL ’s host is not an empty host

a path-relative-scheme-less-URL string

Otherwise

a scheme-relative-URL string

a path-absolute-URL string

a path-relative-scheme-less-URL string

any optionally followed by U+003F (?) and a URL-query string .

A non-null base URL is necessary when parsing relative-URL string A scheme-relative-special-URL string must be " a relative-URL string .

A scheme-relative-special-URL string must be " // ", followed by a valid host string URL-port string path-absolute-URL string A URL-port string ", followed by a valid host string , optionally followed by U+003A (:) and a URL-port string , optionally followed by a path-absolute-URL string .

A URL-port string the empty string one or more ASCII digits representing a decimal number no greater than 2 must be one of the following:

• the empty string

• one or more ASCII digits representing a decimal number no greater than 2 ¹⁶  − 1. A scheme-relative-URL string  − 1.

A scheme-relative-URL string must be " must be " // ", followed by an opaque-host-and-port string path-absolute-URL string An opaque-host-and-port string valid opaque-host string URL-port string A scheme-relative-file-URL string ", followed by an opaque-host-and-port string , optionally followed by a path-absolute-URL string .

An opaque-host-and-port string must be either the empty string or: a valid opaque-host string , optionally followed by U+003A (:) and a URL-port string .

A scheme-relative-file-URL string must be " must be " // ", followed by one of the following: a valid host string path-absolute-non-Windows-file-URL string a path-absolute-URL string A path-absolute-URL string ", followed by one of the following:

• a valid host string , optionally followed by a path-absolute-non-Windows-file-URL string

• a path-absolute-URL string .

A path-absolute-URL string path-relative-URL string A path-absolute-non-Windows-file-URL string must be U+002F (/) followed by a path-relative-URL string .

A path-absolute-non-Windows-file-URL string path-absolute-URL string Windows drive letter A path-relative-URL string must be a path-absolute-URL string that does not start with: U+002F (/), followed by a Windows drive letter , followed by U+002F (/).

A path-relative-URL string URL-path-segment strings A path-relative-scheme-less-URL string must be zero or more URL-path-segment strings , separated from each other by U+002F (/), and not start with U+002F (/).

A path-relative-scheme-less-URL string path-relative-URL string URL-scheme string A URL-path-segment string must be a path-relative-URL string that does not start with: a URL-scheme string , followed by U+003A (:).

A URL-path-segment string zero or more URL units single-dot URL path segment double-dot URL path segment a single-dot URL path segment a double-dot URL path segment A URL-query string must be one of the following:

• zero or more URL units excluding U+002F (/) and U+003F (?), that together are not a single-dot URL path segment or a double-dot URL path segment .

• a single-dot URL path segment

• a double-dot URL path segment .

A URL-query string URL units A URL-fragment string must be zero or more URL units .

A URL-fragment string URL units The URL code points ASCII alphanumeric code points must be zero or more URL units .

The URL code points are ASCII alphanumeric , U+0021 (!), U+0024 ($), U+0026 (&), U+0027 ('), U+0028 LEFT PARENTHESIS, U+0029 RIGHT PARENTHESIS, U+002A (*), U+002B (+), U+002C (,), U+002D (-), U+002E (.), U+002F (/), U+003A (:), U+003B (;), U+003D (=), U+003F (?), U+0040 (@), U+005F (_), U+007E (~), and code points in the range U+00A0 to U+10FFFD, inclusive, excluding surrogates and noncharacters Code points greater than U+007F DELETE will be converted to percent-encoded bytes URL parser In HTML, when the document encoding is a legacy encoding, code points in the URL-query string percent-encoded bytes using the document’s encoding .

Code points greater than U+007F DELETE will be converted to percent-encoded bytes by the URL parser .

In HTML, when the document encoding is a legacy encoding, code points in the URL-query string that are higher than U+007F DELETE will be converted to percent-encoded bytes using the document’s encoding . This can cause problems if a URL that works in one document is copied to another document that uses a different document encoding. Using the UTF-8 encoding everywhere solves this problem.

<!doctype html>
<meta charset="windows-1252">

<


a


href


=


"?sm&ouml;rg&aring;sbord"


>

Test

</


a


>


Since the document encoding is windows-1252, the link’s 

The URL units are URL code points and percent-encoded bytes .

Percent-encoded bytes URL code points There is no way to express a Percent-encoded bytes can be used to encode code points that are not URL code points or are excluded from being written.

There is no way to express a username or password URL record valid URL string of a URL record within a valid URL string .

4.4. 4.4. URL parsing URL parsing The URL parser scalar value string

The URL parser takes a scalar value string input base URL , with an optional null or base URL base (default null) and an optional encoding encoding (default UTF-8 ), and then runs these steps: Non-web-browser implementations only need to implement the basic URL parser How user input in the web browser’s address bar is converted to a URL record URL rendering requirements Let ), and then runs these steps:

Non-web-browser implementations only need to implement the basic URL parser .

How user input in the web browser’s address bar is converted to a URL record is out-of-scope of this standard. This standard does include URL rendering requirements as they pertain trust decisions.

1. Let url basic URL parser be the result of running the basic URL parser on input with base and encoding If .

2. If url If is failure, return failure.

3. If url ’s ’s scheme is not " is not " blob ", return ", return url Set .

4. Set url ’s blob URL entry resolving the blob URL ’s blob URL entry to the result of resolving the blob URL url Return , if that did not return failure, and null otherwise.

5. Return url The basic URL parser scalar value string .

The basic URL parser takes a scalar value string input base URL , with an optional null or base URL base (default null), an optional encoding encoding (default UTF-8 ), an optional ), an optional URL url , and an optional state override state override state override The , and then runs these steps:

1. If url Set is not given:

   1. Set url to a new URL If .

   2. If input C0 control or space contains any leading or trailing C0 control or space , invalid-URL-unit validation error Remove any leading and trailing C0 control or space validation error .

   3. Remove any leading and trailing C0 control or space from input If .

2. If input ASCII tab or newline contains any ASCII tab or newline , invalid-URL-unit validation error Remove all ASCII tab or newline validation error .

3. Remove all ASCII tab or newline from input Let .

4. Let state state override scheme start state Set be state override if given, or scheme start state otherwise.

5. Set encoding getting an output encoding to the result of getting an output encoding Let from encoding .

6. Let buffer Let be the empty string.

7. Let atSignSeen , insideBrackets , and passwordTokenSeen Let be false.

8. Let pointer be a pointer for input Keep running the following state machine by switching on .

9. Keep running the following state machine by switching on state . If after a run pointer EOF code point points to the EOF code point , go to the next step. Otherwise, increase pointer by 1 and continue with the state machine.

   scheme start state If scheme start state

   1. If c ASCII alpha is an ASCII alpha , append c , lowercased , to buffer , and set state scheme state Otherwise, if state override to scheme state no scheme state .

   2. Otherwise, if state override is not given, set state to no scheme state and decrease pointer Otherwise, return failure. This indication of failure is used exclusively by the by 1.

   3. Otherwise, return failure.

      This indication of failure is used exclusively by the Location object’s protocol setter.

   scheme state If scheme state

   1. If c ASCII alphanumeric is an ASCII alphanumeric , U+002B (+), U+002D (-), or U+002E (.), append c , lowercased , to buffer Otherwise, if .

   2. Otherwise, if c If state override If is U+003A (:), then:

      1. If state override is given, then:

         1. If url ’s ’s scheme special scheme is a special scheme and buffer special scheme If is not a special scheme , then return.

         2. If url ’s ’s scheme special scheme is not a special scheme and buffer special scheme If is a special scheme , then return.

         3. If url includes credentials includes credentials or has a non-null port , and buffer is " is " file ", then return. If ", then return.

         4. If url ’s ’s scheme is " is " file " and its " and its host empty host Set is an empty host , then return.

      2. Set url ’s ’s scheme to buffer If state override If .

      3. If state override is given, then:

         1. If url ’s ’s port is url ’s ’s scheme ’s default port ’s default port , then set url ’s ’s port Return. Set to null.

         2. Return.

      4. Set buffer If to the empty string.

      5. If url ’s ’s scheme is " is " file ", then: If ", then:

         1. If remaining does not start with " does not start with " // ", ", special-scheme-missing-following-solidus validation error Set validation error .

         2. Set state file state Otherwise, if to file state .

      6. Otherwise, if url is special is special , base is non-null, and base ’s ’s scheme is url ’s ’s scheme :

         1. Assert : base is special opaque path is is special ). Set (and therefore does not have an opaque path ).

         2. Set state special relative or authority state Otherwise, if to special relative or authority state .

      7. Otherwise, if url is special is special , set state special authority slashes state Otherwise, if to special authority slashes state .

      8. Otherwise, if remaining starts with an U+002F (/), set state path or authority state to path or authority state and increase pointer Otherwise, set by 1.

      9. Otherwise, set url ’s ’s path to the empty string and set state opaque path state Otherwise, if state override to opaque path state .

   3. Otherwise, if state override is not given, set buffer to the empty string, state no scheme state to no scheme state , and start over (from the first code point in input ). Otherwise, return failure. This indication of failure is used exclusively by the ).

   4. Otherwise, return failure.

      This indication of failure is used exclusively by the Location object’s protocol setter. Furthermore, the non-failure termination earlier in this state is an intentional difference for defining that setter.

   no scheme state If no scheme state

   1. If base is null, or base opaque path has an opaque path and c is not U+0023 (#), missing-scheme-non-relative-URL validation error Otherwise, if validation error , return failure.

   2. Otherwise, if base opaque path has an opaque path and c is U+0023 (#), set url ’s ’s scheme to base ’s ’s scheme , url ’s ’s path to base ’s ’s path , url ’s ’s query to base ’s ’s query , url ’s ’s fragment to the empty string, and set state fragment state Otherwise, if to fragment state .

   3. Otherwise, if base ’s ’s scheme is not " is not " file ", set ", set state relative state to relative state and decrease pointer Otherwise, set by 1.

   4. Otherwise, set state file state to file state and decrease pointer by 1.

   special relative or authority state If special relative or authority state

   1. If c is U+002F (/) and remaining starts with U+002F (/), then set state special authority ignore slashes state to special authority ignore slashes state and increase pointer Otherwise, by 1.

   2. Otherwise, special-scheme-missing-following-solidus validation error validation error , set state relative state to relative state and decrease pointer by 1.

   path or authority state If path or authority state

   1. If c is U+002F (/), then set state authority state Otherwise, set to authority state path state .

   2. Otherwise, set state to path state , and decrease pointer by 1.

   relative state Assert: relative state

   1. Assert: base ’s ’s scheme is not " is not " file ". Set ".

   2. Set url ’s ’s scheme to base ’s ’s scheme If .

   3. If c is U+002F (/), then set state relative slash state Otherwise, if to relative slash state .

   4. Otherwise, if url is special is special and c is U+005C (\), invalid-reverse-solidus validation error validation error , set state relative slash state Otherwise: Set to relative slash state .

   5. Otherwise:

      1. Set url ’s ’s username to base ’s ’s username , url ’s ’s password to base ’s ’s password , url ’s ’s host to base ’s ’s host , url ’s ’s port to base ’s ’s port , url ’s ’s path to a clone of base ’s ’s path , and url ’s ’s query to base ’s ’s query If .

      2. If c is U+003F (?), then set url ’s ’s query to the empty string, and state query state Otherwise, if to query state .

      3. Otherwise, if c is U+0023 (#), set url ’s ’s fragment to the empty string and state fragment state Otherwise, if to fragment state .

      4. Otherwise, if c EOF code point Set is not the EOF code point :

         1. Set url ’s ’s query to null.

         2. Shorten url ’s ’s path Set .

         3. Set state path state to path state and decrease pointer by 1.

   relative slash state If relative slash state

   1. If url is special is special and c If is U+002F (/) or U+005C (\), then:

      1. If c is U+005C (\), invalid-reverse-solidus validation error Set validation error .

      2. Set state special authority ignore slashes state Otherwise, if to special authority ignore slashes state .

   2. Otherwise, if c is U+002F (/), then set state authority state Otherwise, set to authority state .

   3. Otherwise, set url ’s ’s username to base ’s ’s username , url ’s ’s password to base ’s ’s password , url ’s ’s host to base ’s ’s host , url ’s ’s port to base ’s ’s port , state path state to path state , and then, decrease pointer by 1.

   special authority slashes state If special authority slashes state

   1. If c is U+002F (/) and remaining starts with U+002F (/), then set state special authority ignore slashes state to special authority ignore slashes state and increase pointer Otherwise, by 1.

   2. Otherwise, special-scheme-missing-following-solidus validation error validation error , set state special authority ignore slashes state to special authority ignore slashes state and decrease pointer by 1.

   special authority ignore slashes state If special authority ignore slashes state

   1. If c is neither U+002F (/) nor U+005C (\), then set state authority state to authority state and decrease pointer Otherwise, by 1.

   2. Otherwise, special-scheme-missing-following-solidus validation error validation error .

   authority state If authority state

   1. If c is U+0040 (@), then:

      1. Invalid-credentials validation error If validation error .

      2. If atSignSeen is true, then prepend " is true, then prepend " %40 " to " to buffer Set .

      3. Set atSignSeen For each to true.

      4. For each codePoint in buffer If :

         1. If codePoint is U+003A (:) and passwordTokenSeen is false, then set passwordTokenSeen to true and continue Let .

         2. Let encodedCodePoints UTF-8 percent-encode be the result of running UTF-8 percent-encode codePoint userinfo percent-encode set If using the userinfo percent-encode set .

         3. If passwordTokenSeen is true, then append encodedCodePoints to url ’s ’s password Otherwise, append .

         4. Otherwise, append encodedCodePoints to url ’s ’s username Set .

      5. Set buffer Otherwise, if one of the following is true: to the empty string.

   2. Otherwise, if one of the following is true:

      • c EOF code point is the EOF code point , U+002F (/), U+003F (?), or U+0023 (#)

      • url is special is special and c then: If is U+005C (\)

      then:

      1. If atSignSeen is true and buffer is the empty string, invalid-credentials validation error Decrease validation error , return failure.

      2. Decrease pointer by buffer ’s code point length ’s code point length + 1, set buffer to the empty string, and set state host state Otherwise, append to host state .

   3. Otherwise, append c to buffer .

   host state host state

   hostname state If state override hostname state

   1. If state override is given and url ’s ’s scheme is " is " file ", then decrease ", then decrease pointer by 1 and set state file host state Otherwise, if to file host state .

   2. Otherwise, if c is U+003A (:) and insideBrackets If is false, then:

      1. If buffer is the empty string, host-missing validation error If state override state override hostname state Let validation error , return failure.

      2. If state override is given and state override is hostname state , then return.

      3. Let host host parsing be the result of host parsing buffer with url is not special If is not special .

      4. If host Set is failure, then return failure.

      5. Set url ’s ’s host to host , buffer to the empty string, and state port state Otherwise, if one of the following is true: to port state .

   3. Otherwise, if one of the following is true:

      • c EOF code point is the EOF code point , U+002F (/), U+003F (?), or U+0023 (#)

      • url is special is special and c then decrease is U+005C (\)

      then decrease pointer If by 1, and then:

      1. If url is special is special and buffer is the empty string, host-missing validation error Otherwise, if state override validation error , return failure.

      2. Otherwise, if state override is given, buffer is the empty string, and either url includes credentials includes credentials or url ’s ’s port Let is non-null, return.

      3. Let host host parsing be the result of host parsing buffer with url is not special If is not special .

      4. If host Set is failure, then return failure.

      5. Set url ’s ’s host to host , buffer to the empty string, and state path start state If state override Otherwise: If to path start state .

      6. If state override is given, then return.

   4. Otherwise:

      1. If c is U+005B ([), then set insideBrackets If to true.

      2. If c is U+005D (]), then set insideBrackets Append to false.

      3. Append c to buffer .

   port state If port state

   1. If c ASCII digit is an ASCII digit , append c to buffer Otherwise, if one of the following is true: .

   2. Otherwise, if one of the following is true:

      • c EOF code point is the EOF code point , U+002F (/), U+003F (?), or U+0023 (#)

      • url is special is special and c is U+005C (\)

      • state override then: If state override is given

      then:

      1. If buffer Let is not the empty string, then:

         1. Let port be the mathematical integer value that is represented by buffer ASCII digits If in radix-10 using ASCII digits for digits with values 0 through 9.

         2. If port is greater than 2 is greater than 2 ¹⁶  − 1,  − 1, port-out-of-range validation error Set validation error , return failure.

         3. Set url ’s ’s port to null, if port is url ’s ’s scheme ’s default port ’s default port ; otherwise to ; otherwise to port Set .

         4. Set buffer If state override Set to the empty string.

      2. If state path start state override is given, then return.

      3. Set state to path start state and decrease pointer Otherwise, by 1.

   3. Otherwise, port-invalid validation error validation error , return failure.

   file state Set file state

   1. Set url ’s ’s scheme to " to " file ". Set ".

   2. Set url ’s ’s host If to the empty string.

   3. If c If is U+002F (/) or U+005C (\), then:

      1. If c is U+005C (\), invalid-reverse-solidus validation error Set validation error .

      2. Set state file slash state Otherwise, if to file slash state .

   4. Otherwise, if base is non-null and base ’s ’s scheme is " is " file ": Set ":

      1. Set url ’s ’s host to base ’s ’s host , url ’s ’s path to a clone of base ’s ’s path , and url ’s ’s query to base ’s ’s query If .

      2. If c is U+003F (?), then set url ’s ’s query to the empty string and state query state Otherwise, if to query state .

      3. Otherwise, if c is U+0023 (#), set url ’s ’s fragment to the empty string and state fragment state Otherwise, if to fragment state .

      4. Otherwise, if c EOF code point Set is not the EOF code point :

         1. Set url ’s ’s query If the code point substring to null.

         2. If the code point substring from pointer to the end of input start with a Windows drive letter does not start with a Windows drive letter , then shorten url ’s ’s path Otherwise: .

         3. Otherwise:

            1. File-invalid-Windows-drive-letter validation error Set validation error .

            2. Set url ’s ’s path This is a (platform-independent) Windows drive letter quirk. Set to « ».

            This is a (platform-independent) Windows drive letter quirk.

         4. Set state path state to path state and decrease pointer Otherwise, set by 1.

   5. Otherwise, set state path state to path state , and decrease pointer by 1.

   file slash state If file slash state

   1. If c If is U+002F (/) or U+005C (\), then:

      1. If c is U+005C (\), invalid-reverse-solidus validation error Set validation error .

      2. Set state file host state Otherwise: If to file host state .

   2. Otherwise:

      1. If base is non-null and base ’s ’s scheme is " is " file ", then: Set ", then:

         1. Set url ’s ’s host to base ’s ’s host If the code point substring .

         2. If the code point substring from pointer to the end of input start with a Windows drive letter does not start with a Windows drive letter and base ’s ’s path [0] is a normalized Windows drive letter [0] is a normalized Windows drive letter , then append base ’s ’s path [0] to [0] to url ’s ’s path This is a (platform-independent) Windows drive letter quirk. Set .

            This is a (platform-independent) Windows drive letter quirk.

      2. Set state path state to path state , and decrease pointer by 1.

   file host state If file host state

   1. If c EOF code point is the EOF code point , U+002F (/), U+005C (\), U+003F (?), or U+0023 (#), then decrease pointer If state override by 1 and then:

      1. If state override is not given and buffer Windows drive letter is a Windows drive letter , file-invalid-Windows-drive-letter-host validation error validation error , set state path state This is a (platform-independent) Windows drive letter quirk. to path state .

         This is a (platform-independent) Windows drive letter quirk. buffer path state Otherwise, if is not reset here and instead used in the path state .

      2. Otherwise, if buffer Set is the empty string, then:

         1. Set url ’s ’s host If state override Set to the empty string.

         2. If state path start state Otherwise, run these steps: Let override is given, then return.

         3. Set state to path start state .

      3. Otherwise, run these steps:

         1. Let host host parsing be the result of host parsing buffer with url is not special If is not special .

         2. If host If is failure, then return failure.

         3. If host is " is " localhost ", then set ", then set host Set to the empty string.

         4. Set url ’s ’s host to host If state override Set .

         5. If state override is given, then return.

         6. Set buffer to the empty string and state path start state Otherwise, append to path start state .

   2. Otherwise, append c to buffer .

   path start state If path start state

   1. If url is special If is special , then:

      1. If c is U+005C (\), invalid-reverse-solidus validation error Set validation error .

      2. Set state path state If to path state .

      3. If c is neither U+002F (/) nor U+005C (\), then decrease pointer Otherwise, if state override by 1.

   2. Otherwise, if state override is not given and c is U+003F (?), set url ’s ’s query to the empty string and state query state Otherwise, if state override to query state .

   3. Otherwise, if state override is not given and c is U+0023 (#), set url ’s ’s fragment to the empty string and state fragment state Otherwise, if to fragment state .

   4. Otherwise, if c EOF code point Set is not the EOF code point :

      1. Set state path state If to path state .

      2. If c is not U+002F (/), then decrease pointer Otherwise, if state override by 1.

   5. Otherwise, if state override is given and url ’s ’s host is null, append the empty string to url ’s ’s path .

   path state If one of the following is true: path state

   1. If one of the following is true:

      • c EOF code point is the EOF code point or U+002F (/)

      • url is special is special and c is U+005C (\)

      • state override state override is not given and c then: If is U+003F (?) or U+0023 (#)

      then:

      1. If url is special is special and c is U+005C (\), invalid-reverse-solidus validation error If validation error .

      2. If buffer double-dot URL path segment is a double-dot URL path segment , then:

         1. Shorten url ’s ’s path If neither .

         2. If neither c is U+002F (/), nor url is special is special and c is U+005C (\), append the empty string to url ’s ’s path This means that for input .

            This means that for input /usr/.. the result is / Otherwise, if and not a lack of a path.

      3. Otherwise, if buffer single-dot URL path segment is a single-dot URL path segment and if neither c is U+002F (/), nor url is special is special and c is U+005C (\), append the empty string to url ’s ’s path Otherwise, if .

      4. Otherwise, if buffer single-dot URL path segment If is not a single-dot URL path segment , then:

         1. If url ’s ’s scheme is " is " file ", ", url ’s ’s path is empty is empty , and buffer Windows drive letter is a Windows drive letter , then replace the second code point in buffer This is a (platform-independent) Windows drive letter quirk. with U+003A (:).

            This is a (platform-independent) Windows drive letter quirk.

         2. Append buffer to url ’s ’s path Set .

      5. Set buffer If to the empty string.

      6. If c is U+003F (?), then set url ’s ’s query to the empty string and state query state If to query state .

      7. If c is U+0023 (#), then set url ’s ’s fragment to the empty string and state fragment state Otherwise, run these steps: If to fragment state .

   2. Otherwise, run these steps:

      1. If c URL code point is not a URL code point and not U+0025 (%), invalid-URL-unit validation error If validation error .

      2. If c is U+0025 (%) and remaining ASCII hex digits does not start with two ASCII hex digits , invalid-URL-unit validation error validation error .

      3. UTF-8 percent-encode UTF-8 percent-encode c path percent-encode set using the path percent-encode set and append the result to buffer .

   opaque path state If opaque path state

   1. If c is U+003F (?), then set url ’s ’s query to the empty string and state query state Otherwise, if to query state .

   2. Otherwise, if c is U+0023 (#), then set url ’s ’s fragment to the empty string and state fragment state Otherwise: If to fragment state .

   3. Otherwise:

      1. If c EOF code point URL code point is not the EOF code point , not a URL code point , and not U+0025 (%), invalid-URL-unit validation error If validation error .

      2. If c is U+0025 (%) and remaining ASCII hex digits does not start with two ASCII hex digits , invalid-URL-unit validation error If validation error .

      3. If c EOF code point UTF-8 percent-encode is not the EOF code point , UTF-8 percent-encode c C0 control percent-encode set using the C0 control percent-encode set and append the result to url ’s ’s path .

   query state If query state

   1. If encoding is not UTF-8 and one of the following is true:

      • url is not special is not special

      • url ’s ’s scheme is " is " ws " or " " or " wss " then set "

      then set encoding to UTF-8 If one of the following is true: .

   2. If one of the following is true:

      • state override state override is not given and c is U+0023 (#)

      • c EOF code point then: Let is the EOF code point

      then:

      1. Let queryPercentEncodeSet special-query percent-encode set be the special-query percent-encode set if url is special is special ; otherwise the query percent-encode set ; otherwise the query percent-encode set .

      2. Percent-encode after encoding Percent-encode after encoding , with encoding , buffer , and queryPercentEncodeSet , and append the result to url ’s ’s query This operation cannot be invoked code-point-for-code-point due to the stateful ISO-2022-JP encoder Set .

         This operation cannot be invoked code-point-for-code-point due to the stateful ISO-2022-JP encoder .

      3. Set buffer If to the empty string.

      4. If c is U+0023 (#), then set url ’s ’s fragment fragment state Otherwise, if to the empty string and state to fragment state .

   3. Otherwise, if c EOF code point If is not the EOF code point :

      1. If c URL code point is not a URL code point and not U+0025 (%), invalid-URL-unit validation error If validation error .

      2. If c is U+0025 (%) and remaining ASCII hex digits does not start with two ASCII hex digits , invalid-URL-unit validation error Append validation error .

      3. Append c to buffer .

   fragment state If fragment state

   1. If c EOF code point If is not the EOF code point , then:

      1. If c URL code point is not a URL code point and not U+0025 (%), invalid-URL-unit validation error If validation error .

      2. If c is U+0025 (%) and remaining ASCII hex digits does not start with two ASCII hex digits , invalid-URL-unit validation error validation error .

      3. UTF-8 percent-encode UTF-8 percent-encode c fragment percent-encode set using the fragment percent-encode set and append the result to url ’s ’s fragment Return .

10. Return url To set the username .