URL ( PR #486 )

PR Preview — Last Updated

Participate:
GitHub whatwg/url ( new issue , open issues )
IRC: #whatwg on Freenode
Commits:
GitHub whatwg/url/commits
Go to the living standard
@urlstandard
Tests:
web-platform-tests url/ ( ongoing work )
Translations (non-normative) :
日本語
Not Ready For Implementation

This spec is not yet ready for implementation. It exists in this repository to record the ideas and promote discussion.

Before attempting to implement this spec, please contact the editors.

Abstract

The URL Standard defines URLs, domains, IP addresses, the application/x-www-form-urlencoded format, and their API.

Goals

The URL standard takes the following approach towards making URLs fully interoperable:

As the editors learn more about the subject matter the goals might increase in scope somewhat.

1. Infrastructure

This specification depends on the Infra Standard. [INFRA]

Some terms used in this specification are defined in the following standards and specifications:

To serialize an integer , represent it as the shortest possible decimal number.

1.1. Writing

A validation error indicates a mismatch between input and valid input. User agents, especially conformance checkers, are encouraged to report them somewhere.

A validation error does not mean that the parser terminates. Termination of a parser is always stated explicitly, e.g., through a return statement.

It is useful to signal validation errors as error-handling can be non-intuitive, legacy user agents might not implement correct error-handling, and the intent of what is written might be unclear to other developers.

1.2. Parsers

The EOF code point is a conceptual code point that signifies the end of a string or code point stream.

Within a parser algorithm that uses a pointer variable, c references the code point the pointer variable points to.

Within a string-based parser algorithm that uses a pointer variable, remaining references the substring after pointer in the string being processed.

If " mailto:username@example " is a string being processed and pointer points to @, c is U+0040 (@) and remaining is " example ".

1.3. Percent-encoded bytes

A percent-encoded byte is U+0025 (%), followed by two ASCII hex digits . Sequences of percent-encoded bytes , after conversion to bytes, should not cause UTF-8 decode without BOM or fail to return failure.

To percent encode a byte into a percent-encoded byte , return a string consisting of U+0025 (%), followed by two ASCII upper hex digits representing byte .

To percent decode a byte sequence input , run these steps:

Using anything but UTF-8 decode without BOM when the input contains bytes that are not ASCII bytes might be insecure and is not recommended.

  1. Let output be an empty byte sequence .

  2. For each byte byte in input :

    1. If byte is not 0x25 (%), then append byte to output .

    2. Otherwise, if byte is 0x25 (%) and the next two bytes after byte in input are not in the ranges 0x30 (0) to 0x39 (9), 0x41 (A) to 0x46 (F), and 0x61 (a) to 0x66 (f), all inclusive, append byte to output .

    3. Otherwise:

      1. Let bytePoint be the two bytes after byte in input , decoded , and then interpreted as hexadecimal number.

      2. Append a byte whose value is bytePoint to output .

      3. Skip the next two bytes in input .

  3. Return output .

To string percent decode a string input , run these steps:

  1. Let bytes be the UTF-8 encoding of input .

  2. Return the percent decoding of bytes .

The C0 control percent-encode set are the C0 controls and all code points greater than U+007E (~).

The fragment percent-encode set is the C0 control percent-encode set and U+0020 SPACE, U+0022 ("), U+003C (<), U+003E (>), and U+0060 (`).

The path percent-encode set is the fragment percent-encode set and U+0023 (#), U+003F (?), U+007B ({), and U+007D (}).

The userinfo percent-encode set is the path percent-encode set and U+002F (/), U+003A (:), U+003B (;), U+003D (=), U+0040 (@), U+005B ([), U+005C (\), U+005D (]), U+005E (^), and U+007C (|).

To UTF-8 percent encode a codePoint , using a percentEncodeSet , run these steps:

  1. If codePoint is not in percentEncodeSet , then return codePoint .

  2. Let bytes be the result of running UTF-8 encode on codePoint .

  3. Percent encode each byte in bytes , and then return the results concatenated, in the same order.

2. Security considerations

The security of a URL is a function of its environment. Care is to be taken when rendering, interpreting, and passing URLs around.

When rendering and allocating new URLs "spoofing" needs to be considered. An attack whereby one host or URL can be confused for another. For instance, consider how 1/l/I, m/rn/rri, 0/O, and а/a can all appear eerily similar. Or worse, consider how U+202A LEFT-TO-RIGHT EMBEDDING and similar code points are invisible. [UTR36]

When passing a URL from party A to B , both need to carefully consider what is happening. A might end up leaking data it does not want to leak. B might receive input it did not expect and take an action that harms the user. In particular, B should never trust A , as at some point URLs from A can come from untrusted sources.

3. Hosts (domains and IP addresses)

At a high level, a host , valid host string , host parser , and host serializer relate as follows:

3.1. Host representation

A host is a domain , an IPv4 address , an IPv6 address , an opaque host , or an empty host . Typically a host serves as a network address, but it is sometimes used as opaque identifier in URLs where a network address is not necessary.

The RFCs referenced in the paragraphs below are for informative purposes only. They have no influence on host writing, parsing, and serialization. Unless stated otherwise in the sections that follow.

A domain is an ASCII string that identifies a realm within a network. [RFC1034]

The example.com and example.com. domains are not equivalent and typically treated as distinct.

An IPv4 address is a 32-bit unsigned integer that identifies a network address. [RFC791]

An IPv6 address is a 128-bit unsigned integer that identifies a network address. For the purposes of this standard it is represented as a list of eight 16-bit unsigned integers, also known as IPv6 pieces . [RFC4291]

Support for <zone_id> is intentionally omitted .

An opaque host is a non-empty ASCII string holding data that can be used for further processing.

An empty host is the empty string.

3.2. Host miscellaneous

A forbidden host code point is U+0000 NULL, U+0009 TAB, U+000A LF, U+000D CR, U+0020 SPACE, U+0023 (#), U+0025 (%), U+002F (/), U+003A (:), U+003F (?), U+0040 (@), U+005B ([), U+005C (\), or U+005D (]).

A host ’s public suffix is the portion of a host which is included on the Public Suffix List . To obtain host ’s public suffix , run these steps: [PSL]

  1. If host is not a domain , then return null.

  2. Return the public suffix obtained by executing the algorithm defined by the Public Suffix List on host . [PSL] .

A host ’s registrable domain is a domain formed by the most specific public suffix, along with the domain label immediately preceeding it, if any. To obtain host ’s registrable domain , run these steps:

  1. If host ’s public suffix is null or host ’s public suffix equals host , then return null.

  2. Return the registrable domain obtained by executing the algorithm defined by the Public Suffix List on host . [PSL]

Host input Public suffix Registrable domain
com com null
example.com com example.com
www.example.com com example.com
sub.www.example.com com example.com
EXAMPLE.COM com example.com
github.io github.io null
whatwg.github.io github.io whatwg.github.io
إختبار xn--kgbechtv null
example.إختبار xn--kgbechtv example.xn--kgbechtv
sub.example.إختبار xn--kgbechtv example.xn--kgbechtv
[2001:0db8:85a3:0000:0000:8a2e:0370:7334] null null

Specifications should prefer the origin concept for security decisions. The notion of " public suffix " and " registrable domain " cannot be relied-upon to provide a hard security boundary, as the public suffix list will diverge from client to client. Specifications which ignore this advice are encouraged to carefully consider whether URLs' schemes ought to be incorporated into any decisions made, i.e. whether to use the same site or schemelessly same site concepts.

3.3. IDNA

The domain to ASCII algorithm, given a string domain and optionally a boolean beStrict , runs these steps:

  1. If beStrict is not given, set it to false.

  2. Let result be the result of running Unicode ToASCII with domain_name set to domain , UseSTD3ASCIIRules set to beStrict , CheckHyphens set to false, CheckBidi set to true, CheckJoiners set to true, Transitional_Processing set to false, and VerifyDnsLength set to beStrict .

  3. If result is a failure value, validation error , return failure.

  4. Return result .

The domain to Unicode algorithm, given a domain domain , runs these steps:

  1. Let result be the result of running Unicode ToUnicode with domain_name set to domain , CheckHyphens set to false, CheckBidi set to true, CheckJoiners set to true, UseSTD3ASCIIRules set to false, and Transitional_Processing set to false.

  2. Signify validation errors for any returned errors, and then, return result .

3.4. Host writing

A valid host string must be a valid domain string , a valid IPv4-address string , or: U+005B ([), followed by a valid IPv6-address string , followed by U+005D (]).

A domain is a valid domain if these steps return success:

  1. Let result be the result of running domain to ASCII with domain and true.

  2. If result is failure, then return failure.

  3. Set result to the result of running Unicode ToUnicode with domain_name set to result , CheckHyphens set to false, CheckBidi set to true, CheckJoiners set to true, UseSTD3ASCIIRules set to true, and Transitional_Processing set to false.

  4. If result contains any errors, return failure.

  5. Return success.

Ideally we define this in terms of a sequence of code points that make up a valid domain rather than through a whack-a-mole: issue 245 .

A valid domain string must be a string that is a valid domain .

A valid IPv4-address string must be four sequences of up to three ASCII digits per sequence, each representing a decimal number no greater than 255, and separated from each other by U+002E (.).

A valid IPv6-address string is defined in the "Text Representation of Addresses" chapter of IP Version 6 Addressing Architecture . [RFC4291]

A valid opaque-host string must be one or more URL units or: U+005B ([), followed by a valid IPv6-address string , followed by U+005D (]).

This is not part of the definition of valid host string as it requires context to be distinguished.

3.5. Host parsing

The host parser takes a string input with an optional boolean isNotSpecial , and then runs these steps:

  1. If isNotSpecial is not given, then set isNotSpecial to false.

  2. If input starts with U+005B ([), then:

    1. If input does not end with U+005D (]), validation error , return failure.

    2. Return the result of IPv6 parsing input with its leading U+005B ([) and trailing U+005D (]) removed.

  3. If isNotSpecial is true, then return the result of opaque-host parsing input .

  4. Let domain be the result of running UTF-8 decode without BOM on the string percent decoding of input .

    Alternatively UTF-8 decode without BOM or fail can be used, coupled with an early return for failure, as domain to ASCII fails on U+FFFD REPLACEMENT CHARACTER.

  5. Let asciiDomain be the result of running domain to ASCII on domain .

  6. If asciiDomain is failure, validation error , return failure.

  7. If asciiDomain contains a forbidden host code point , validation error , return failure.

  8. Let ipv4Host be the result of IPv4 parsing asciiDomain .

  9. If ipv4Host is an IPv4 address or failure, return ipv4Host .

  10. Return asciiDomain .

The IPv4 number parser takes a string input and a validationErrorFlag pointer, and then runs these steps:

  1. Let R be 10.

  2. If input contains at least two code points and the first two code points are either " 0x " or " 0X ", then:

    1. Set validationErrorFlag .

    2. Remove the first two code points from input .

    3. Set R to 16.

  3. Otherwise, if input contains at least two code points and the first code point is U+0030 (0), then:

    1. Set validationErrorFlag .

    2. Remove the first code point from input .

    3. Set R to 8.

  4. If input is the empty string, then return zero.

  5. If input contains a code point that is not a radix- R digit, then return failure.

  6. Return the mathematical integer value that is represented by input in radix- R notation, using ASCII hex digits for digits with values 0 through 15.

The IPv4 parser takes a string input and then runs these steps:

  1. Let validationErrorFlag be unset.

  2. Let parts be input split on U+002E (.).

  3. If the last item in parts is the empty string, then:

    1. Set validationErrorFlag .

    2. If parts has more than one item, then remove the last item from parts .

  4. If parts has more than four items, return input .

  5. Let numbers be the empty list.

  6. For each part in parts :

    1. If part is the empty string, return input .

      0..0x300 is a domain , not an IPv4 address .

    2. Let n be the result of parsing part using validationErrorFlag .

    3. If n is failure, return input .

    4. Append n to numbers .

  7. If validationErrorFlag is set, validation error .

  8. If any item in numbers is greater than 255, validation error .

  9. If any but the last item in numbers is greater than 255, return failure.

  10. If the last item in numbers is greater than or equal to 256 (5 − the number of items in numbers ) , validation error , return failure.

  11. Let ipv4 be the last item in numbers .

  12. Remove the last item from numbers .

  13. Let counter be zero.

  14. For each n in numbers :

    1. Increment ipv4 by n × 256 (3 − counter ) .

    2. Increment counter by 1.

  15. Return ipv4 .

The IPv6 parser takes a string input and then runs these steps:

  1. Let address be a new IPv6 address whose IPv6 pieces are all 0.

  2. Let pieceIndex be 0.

  3. Let compress be null.

  4. Let pointer be a pointer into input , initially 0 (pointing to the first code point).

  5. If c is U+003A (:), then:

    1. If remaining does not start with U+003A (:), validation error , return failure.

    2. Increase pointer by 2.

    3. Increase pieceIndex by 1 and then set compress to pieceIndex .

  6. While c is not the EOF code point :

    1. If pieceIndex is 8, validation error , return failure.

    2. If c is U+003A (:), then:

      1. If compress is non-null, validation error , return failure.

      2. Increase pointer and pieceIndex by 1, set compress to pieceIndex , and then continue .

    3. Let value and length be 0.

    4. While length is less than 4 and c is an ASCII hex digit , set value to value × 0x10 + c interpreted as hexadecimal number, and increase pointer and length by 1.

    5. If c is U+002E (.), then:

      1. If length is 0, validation error , return failure.

      2. Decrease pointer by length .

      3. If pieceIndex is greater than 6, validation error , return failure.

      4. Let numbersSeen be 0.

      5. While c is not the EOF code point :

        1. Let ipv4Piece be null.

        2. If numbersSeen is greater than 0, then:

          1. If c is a U+002E (.) and numbersSeen is less than 4, then increase pointer by 1.

          2. Otherwise, validation error , return failure.

        3. If c is not an ASCII digit , validation error , return failure.

        4. While c is an ASCII digit :

          1. Let number be c interpreted as decimal number.

          2. If ipv4Piece is null, then set ipv4Piece to number .

            Otherwise, if ipv4Piece is 0, validation error , return failure.

            Otherwise, set ipv4Piece to ipv4Piece × 10 + number .

          3. If ipv4Piece is greater than 255, validation error , return failure.

          4. Increase pointer by 1.

        5. Set address [ pieceIndex ] to address [ pieceIndex ] × 0x100 + ipv4Piece .

        6. Increase numbersSeen by 1.

        7. If numbersSeen is 2 or 4, then increase pieceIndex by 1.

      6. If numbersSeen is not 4, validation error , return failure.

      7. Break .

    6. Otherwise, if c is U+003A (:):

      1. Increase pointer by 1.

      2. If c is the EOF code point , validation error , return failure.

    7. Otherwise, if c is not the EOF code point , validation error , return failure.

    8. Set address [ pieceIndex ] to value .

    9. Increase pieceIndex by 1.

  7. If compress is non-null, then:

    1. Let swaps be pieceIndex − compress .

    2. Set pieceIndex to 7.

    3. While pieceIndex is not 0 and swaps is greater than 0, swap address [ pieceIndex ] with address [ compress + swaps − 1], and then decrease both pieceIndex and swaps by 1.

  8. Otherwise, if compress is null and pieceIndex is not 8, validation error , return failure.

  9. Return address .

The opaque-host parser takes a string input , and then runs these steps:

  1. If input contains a forbidden host code point excluding U+0025 (%), validation error , return failure.

  2. Let output be the empty string.

  3. For each code point in input , UTF-8 percent encode it using the C0 control percent-encode set , and append the result to output .

  4. Return output .

3.6. Host serializing

The host serializer takes a host host and then runs these steps:

  1. If host is an IPv4 address , return the result of running the IPv4 serializer on host .

  2. Otherwise, if host is an IPv6 address , return U+005B ([), followed by the result of running the IPv6 serializer on host , followed by U+005D (]).

  3. Otherwise, host is a domain , opaque host , or empty host , return host .

The IPv4 serializer takes an IPv4 address address and then runs these steps:

  1. Let output be the empty string.

  2. Let n be the value of address .

  3. For each i in the range 1 to 4, inclusive:

    1. Prepend n % 256, serialized , to output .

    2. If i is not 4, then prepend U+002E (.) to output .

    3. Set n to floor( n / 256).

  4. Return output .

The IPv6 serializer takes an IPv6 address address and then runs these steps:

  1. Let output be the empty string.

  2. Let compress be an index to the first IPv6 piece in the first longest sequences of address ’s IPv6 pieces that are 0.

    In 0:f:0:0:f:f:0:0 it would point to the second 0.

  3. If there is no sequence of address ’s IPv6 pieces that are 0 that is longer than 1, then set compress to null.

  4. Let ignore0 be false.

  5. For each pieceIndex in the range 0 to 7, inclusive:

    1. If ignore0 is true and address [ pieceIndex ] is 0, then continue .

    2. Otherwise, if ignore0 is true, set ignore0 to false.

    3. If compress is pieceIndex , then:

      1. Let separator be " :: " if pieceIndex is 0, and U+003A (:) otherwise.

      2. Append separator to output .

      3. Set ignore0 to true and continue .

    4. Append address [ pieceIndex ], represented as the shortest possible lowercase hexadecimal number, to output .

    5. If pieceIndex is not 7, then append U+003A (:) to output .

  6. Return output .

This algorithm requires the recommendation from A Recommendation for IPv6 Address Text Representation. [RFC5952]

3.7. Host equivalence

To determine whether a host A equals B , return true if A is B , and false otherwise.

Certificate comparison requires a host equivalence check that ignores the trailing dot of a domain (if any). However, those hosts have also various other facets enforced, such as DNS length, that are not enforced here, as URLs do not enforce them. If anyone has a good suggestion for how to bring these two closer together, or what a good unified model would be, please file an issue.

4. URLs

At a high level, a URL , valid URL string , URL parser , and URL serializer relate as follows:

Input Base Valid Output
https:example.org ❌ https://example.org/
https://////example.com/// ❌ https://example.com///
https://example.com/././foo ✅ https://example.com/foo
hello:world https://example.com/ ✅ hello:world
https:example.org https://example.com/ ❌ https://example.com/example.org
\example\..\demo/.\ https://example.com/ ❌ https://example.com/demo/
example https://example.com/demo ✅ https://example.com/example
file:///C|/demo ❌ file:///C:/demo
.. file:///C:/demo ✅ file:///C:/
file://loc%61lhost/ ✅ file:///
https://user:password@example.org/ ❌ https://user:password@example.org/
https://example.org/foo bar ❌ https://example.org/foo%20bar
https://EXAMPLE.com/../x ✅ https://example.com/x
https://ex ample.org/ ❌ Failure
example ❌, due to lack of base Failure
https://example.com:demo ❌ Failure
http://[www.example.com]/ ❌ Failure

The base and output URL are represented in serialized form for brevity.

4.1. URL representation

A URL is a universal identifier. To disambiguate from a valid URL string it can also be referred to as a URL record .

A URL ’s scheme is an ASCII string that identifies the type of URL and can be used to dispatch a URL for further processing after parsing . It is initially the empty string.

A URL ’s username is an ASCII string identifying a username. It is initially the empty string.

A URL ’s password is an ASCII string identifying a password. It is initially the empty string.

A URL ’s host is null or a host . It is initially null.

The following table lists allowed URL ’s scheme / host combinations.

scheme host
domain IPv4 address IPv6 address opaque host empty host null
non-" file " special ✅ ✅ ✅ ❌ ❌ ❌
" file " ✅ ✅ ✅ ❌ ✅ ✅
non-special ❌ ❌ ✅ ✅ ✅ ✅

A URL ’s port is either null or a 16-bit unsigned integer that identifies a networking port. It is initially null.

A URL ’s path is a list of zero or more ASCII strings holding data, usually identifying a location in hierarchical form. It is initially empty.

A special URL always has a non-empty path .

A URL ’s query is either null or an ASCII string holding data. It is initially null.

A URL ’s fragment is either null or an ASCII string holding data that can be used for further processing on the resource the URL ’s other components identify. It is initially null.

A URL also has an associated cannot-be-a-base-URL flag . It is initially unset.

A URL also has an associated blob URL entry that is either null or a blob URL entry . It is initially null.

This is used to support caching the object a " blob " URL refers to as well as its origin. It is important that these are cached as the URL might be removed from the blob URL store between parsing and fetching, while fetching will still need to succeed.

4.2. URL miscellaneous

A special scheme is a scheme listed in the first column of the following table. A default port is a special scheme ’s optional corresponding port and is listed in the second column on the same row.

scheme port
" ftp " 21
" file "
" http " 80
" https " 443
" ws " 80
" wss " 443

A URL is special if its scheme is a special scheme . A URL is not special if its scheme is not a special scheme .

A URL includes credentials if its username or password is not the empty string.

A URL cannot have a username/password/port if its host is null or the empty string, its cannot-be-a-base-URL flag is set, or its scheme is " file ".

A URL can be designated as base URL .

A base URL is useful for the URL parser when the input might be a relative-URL string .

A Windows drive letter is two code points, of which the first is an ASCII alpha and the second is either U+003A (:) or U+007C (|).

A normalized Windows drive letter is a Windows drive letter of which the second code point is U+003A (:).

As per the URL writing section, only a normalized Windows drive letter is conforming.

A string starts with a Windows drive letter if all of the following are true:

String Starts with a Windows drive letter
" c: " ✅
" c:/ " ✅
" c:a " ❌

To shorten a url ’s path :

  1. Let path be url ’s path .

  2. If path is empty , then return.

  3. If url ’s scheme is " file ", path ’s size is 1, and path [0] is a normalized Windows drive letter , then return.

  4. Remove path ’s last item.

4.3. URL writing

A valid URL string must be either a relative-URL-with-fragment string or an absolute-URL-with-fragment string .

An absolute-URL-with-fragment string must be an absolute-URL string , optionally followed by U+0023 (#) and a URL-fragment string .

An absolute-URL string must be one of the following

any optionally followed by U+003F (?) and a URL-query string .

A URL-scheme string must be one ASCII alpha , followed by zero or more of ASCII alphanumeric , U+002B (+), U+002D (-), and U+002E (.). Schemes should be registered in the IANA URI [sic] Schemes registry. [IANA-URI-SCHEMES] [RFC7595]

A relative-URL-with-fragment string must be a relative-URL string , optionally followed by U+0023 (#) and a URL-fragment string .

A relative-URL string must be one of the following, switching on base URL ’s scheme :

A special scheme that is not " file "

a scheme-relative-special-URL string

a path-absolute-URL string

a path-relative-scheme-less-URL string

" file "

a scheme-relative-file-URL string

a path-absolute-URL string if base URL ’s host is an empty host

a path-absolute-non-Windows-file-URL string if base URL ’s host is not an empty host

a path-relative-scheme-less-URL string

Otherwise

a scheme-relative-URL string

a path-absolute-URL string

a path-relative-scheme-less-URL string

any optionally followed by U+003F (?) and a URL-query string .

A non-null base URL is necessary when parsing a relative-URL string .

A scheme-relative-special-URL string must be " // ", followed by a valid host string , optionally followed by U+003A (:) and a URL-port string , optionally followed by a path-absolute-URL string .

A URL-port string must be zero or more ASCII digits .

A scheme-relative-URL string must be " // ", followed by an opaque-host-and-port string , optionally followed by a path-absolute-URL string .

An opaque-host-and-port string must be either the empty string or: a valid opaque-host string , optionally followed by U+003A (:) and a URL-port string .

A scheme-relative-file-URL string must be " // ", followed by one of the following

A path-absolute-URL string must be U+002F (/) followed by a path-relative-URL string .

A path-absolute-non-Windows-file-URL string must be a path-absolute-URL string that does not start with: U+002F (/), followed by a Windows drive letter , followed by U+002F (/).

A path-relative-URL string must be zero or more URL-path-segment strings , separated from each other by U+002F (/), and not start with U+002F (/).

A path-relative-scheme-less-URL string must be a path-relative-URL string that does not start with: a URL-scheme string , followed by U+003A (:).

A URL-path-segment string must be one of the following

A single-dot path segment must be " . " or an ASCII case-insensitive match for " %2e ".

A double-dot path segment must be " .. " or an ASCII case-insensitive match for " .%2e ", " %2e. ", or " %2e%2e ".

A URL-query string must be zero or more URL units .

A URL-fragment string must be zero or more URL units .

The URL code points are ASCII alphanumeric , U+0021 (!), U+0024 ($), U+0026 (&), U+0027 ('), U+0028 LEFT PARENTHESIS, U+0029 RIGHT PARENTHESIS, U+002A (*), U+002B (+), U+002C (,), U+002D (-), U+002E (.), U+002F (/), U+003A (:), U+003B (;), U+003D (=), U+003F (?), U+0040 (@), U+005F (_), U+007E (~), and code points in the range U+00A0 to U+10FFFD, inclusive, excluding surrogates and noncharacters .

Code points greater than U+007F DELETE will be converted to percent-encoded bytes by the URL parser .

In HTML, when the document encoding is a legacy encoding, code points in the URL-query string that are higher than U+007F DELETE will be converted to percent-encoded bytes using the document’s encoding . This can cause problems if a URL that works in one document is copied to another document that uses a different document encoding. Using the UTF-8 encoding everywhere solves this problem.

For example, consider this HTML document: 

<!doctype html>
<meta charset="windows-1252">

<


a


href


=


"?sm&ouml;rg&aring;sbord"


>

Test

</


a


>

Since the document encoding is windows-1252, the link’s URL ’s query will be " sm%F6rg%E5sbord ". If the document encoding had been UTF-8, it would instead be " sm%C3%B6rg%C3%A5sbord ".

The URL units are URL code points and percent-encoded bytes .

Percent-encoded bytes can be used to encode code points that are not URL code points or are excluded from being written.

There is no way to express a username or password of a URL record within a valid URL string .

4.4. URL parsing

The URL parser takes a string input , with an optional base URL base and an optional encoding encoding override , and then runs these steps:

Non-web-browser implementations only need to implement the basic URL parser .

  1. Let url be the result of running the basic URL parser on input with base , and encoding override as provided.

  2. If url is failure, return failure.

  3. If url ’s scheme is not " blob ", return url .

  4. Set url ’s blob URL entry to the result of resolving the blob URL url , if that did not return failure, and null otherwise.

  5. Return url .

The basic URL parser takes a string input , optionally with a base URL base , optionally with an encoding encoding override , optionally with a URL url and a state override state override , and then runs these steps:

The encoding override argument is a legacy concept only relevant for HTML. The url and state override arguments are only for use by various APIs. [HTML]

When the url and state override arguments are not passed, the basic URL parser returns either a new URL or failure. If they are passed, the algorithm modifies the passed url and can terminate without returning anything.

  1. If url is not given:

    1. Set url to a new URL .

    2. If input contains any leading or trailing C0 control or space , validation error .

    3. Remove any leading and trailing C0 control or space from input .

  2. If input contains any ASCII tab or newline , validation error .

  3. Remove all ASCII tab or newline from input .

  4. Let state be state override if given, or scheme start state otherwise.

  5. If base is not given, set it to null.

  6. Let encoding be UTF-8 .

  7. If encoding override is given, set encoding to the result of getting an output encoding from encoding override .

  8. Let buffer be the empty string.

  9. Let the @ flag , [] flag , and passwordTokenSeenFlag be unset.

  10. Let pointer be a pointer to first code point in input .

  11. Keep running the following state machine by switching on state . If after a run pointer points to the EOF code point , go to the next step. Otherwise, increase pointer by one and continue with the state machine.

    scheme start state

    1. If c is an ASCII alpha , append c , lowercased , to buffer , and set state to scheme state .

    2. Otherwise, if state override is not given, set state to no scheme state , and decrease pointer by one.

    3. Otherwise, validation error , return failure.

      This indication of failure is used exclusively by Location object’s protocol attribute.

    scheme state

    1. If c is an ASCII alphanumeric , U+002B (+), U+002D (-), or U+002E (.), append c , lowercased , to buffer .

    2. Otherwise, if c is U+003A (:), then:

      1. If state override is given, then:

        1. If url ’s scheme is a special scheme and buffer is not a special scheme , then return.

        2. If url ’s scheme is not a special scheme and buffer is a special scheme , then return.

        3. If url includes credentials or has a non-null port , and buffer is " file ", then return.

        4. If url ’s scheme is " file " and its host is an empty host or null, then return.

      2. Set url ’s scheme to buffer .

      3. If state override is given, then:

        1. If url ’s port is url ’s scheme ’s default port , then set url ’s port to null.

        2. Return.

      4. Set buffer to the empty string.

      5. If url ’s scheme is " file ", then:

        1. If remaining does not start with " // ", validation error .

        2. Set state to file state .

      6. Otherwise, if url is special , base is non-null, and base ’s scheme is equal to url ’s scheme , set state to special relative or authority state .

        This means that base ’s cannot-be-a-base-URL flag is unset.

      7. Otherwise, if url is special , set state to special authority slashes state .

      8. Otherwise, if remaining starts with an U+002F (/), set state to path or authority state and increase pointer by one.

      9. Otherwise, set url ’s cannot-be-a-base-URL flag , append an empty string to url ’s path , and set state to cannot-be-a-base-URL path state .

    3. Otherwise, if state override is not given, set buffer to the empty string, state to no scheme state , and start over (from the first code point in input ).

    4. Otherwise, validation error , return failure.

      This indication of failure is used exclusively by Location object’s protocol attribute. Furthermore, the non-failure termination earlier in this state is an intentional difference for defining that attribute.

    no scheme state

    1. If base is null, or base ’s cannot-be-a-base-URL flag is set and c is not U+0023 (#), validation error , return failure.

    2. Otherwise, if base ’s cannot-be-a-base-URL flag is set and c is U+0023 (#), set url ’s scheme to base ’s scheme , url ’s path to a copy of base ’s path , url ’s query to base ’s query , url ’s fragment to the empty string, set url ’s cannot-be-a-base-URL flag , and set state to fragment state .

    3. Otherwise, if base ’s scheme is not " file ", set state to relative state and decrease pointer by one.

    4. Otherwise, set state to file state and decrease pointer by one.

    special relative or authority state

    If c is U+002F (/) and remaining starts with U+002F (/), then set state to special authority ignore slashes state and increase pointer by one.

    Otherwise, validation error , set state to relative state and decrease pointer by one.

    path or authority state

    If c is U+002F (/), then set state to authority state .

    Otherwise, set state to path state , and decrease pointer by one.

    relative state

    Set url ’s scheme to base ’s scheme , and then, switching on c :

    The EOF code point

    Set url ’s username to base ’s username , url ’s password to base ’s password , url ’s host to base ’s host , url ’s port to base ’s port , url ’s path to a copy of base ’s path , and url ’s query to base ’s query .

    U+002F (/)

    Set state to relative slash state .

    U+003F (?)

    Set url ’s username to base ’s username , url ’s password to base ’s password , url ’s host to base ’s host , url ’s port to base ’s port , url ’s path to a copy of base ’s path , url ’s query to the empty string, and state to query state .

    U+0023 (#)

    Set url ’s username to base ’s username , url ’s password to base ’s password , url ’s host to base ’s host , url ’s port to base ’s port , url ’s path to a copy of base ’s path , url ’s query to base ’s query , url ’s fragment to the empty string, and state to fragment state .

    Otherwise

    If url is special and c is U+005C (\), validation error , set state to relative slash state .

    Otherwise, run these steps:

    1. Set url ’s username to base ’s username , url ’s password to base ’s password , url ’s host to base ’s host , url ’s port to base ’s port , url ’s path to a copy of base ’s path , and then remove url ’s path ’s last item, if any.

    2. Set state to path state , and decrease pointer by one.

    relative slash state

    1. If url is special and c is U+002F (/) or U+005C (\), then:

      1. If c is U+005C (\), validation error .

      2. Set state to special authority ignore slashes state .

    2. Otherwise, if c is U+002F (/), then set state to authority state .

    3. Otherwise, set url ’s username to base ’s username , url ’s password to base ’s password , url ’s host to base ’s host , url ’s port to base ’s port , state to path state , and then, decrease pointer by one.

    special authority slashes state

    If c is U+002F (/) and remaining starts with U+002F (/), then set state to special authority ignore slashes state and increase pointer by one.

    Otherwise, validation error , set state to special authority ignore slashes state , and decrease pointer by one.

    special authority ignore slashes state

    If c is neither U+002F (/) nor U+005C (\), then set state to authority state and decrease pointer by one.

    Otherwise, validation error .

    authority state

    1. If c is U+0040 (@), then:

      1. Validation error .

      2. If the @ flag is set, prepend " %40 " to buffer .

      3. Set the @ flag .

      4. For each codePoint in buffer :

        1. If codePoint is U+003A (:) and passwordTokenSeenFlag is unset, then set passwordTokenSeenFlag and continue .

        2. Let encodedCodePoints be the result of running UTF-8 percent encode codePoint using the userinfo percent-encode set .

        3. If passwordTokenSeenFlag is set, then append encodedCodePoints to url ’s password .

        4. Otherwise, append encodedCodePoints to url ’s username .

      5. Set buffer to the empty string.

    2. Otherwise, if one of the following is true

      then:

      1. If @ flag is set and buffer is the empty string, validation error , return failure.

      2. Decrease pointer by the number of code points in buffer plus one, set buffer to the empty string, and set state to host state .

    3. Otherwise, append c to buffer .

    host state
    hostname state

    1. If state override is given and url ’s scheme is " file ", then decrease pointer by one and set state to file host state .

    2. Otherwise, if c is U+003A (:) and the [] flag is unset, then:

      1. If buffer is the empty string, validation error , return failure.

      2. Let host be the result of host parsing buffer with url is not special .

      3. If host is failure, then return failure.

      4. Set url ’s host to host , buffer to the empty string, and state to port state .

      5. If state override is given and state override is hostname state , then return.

    3. Otherwise, if one of the following is true

      then decrease pointer by one, and then:

      1. If url is special and buffer is the empty string, validation error , return failure.

      2. Otherwise, if state override is given, buffer is the empty string, and either url includes credentials or url ’s port is non-null, validation error , return.

      3. Let host be the result of host parsing buffer with url is not special .

      4. If host is failure, then return failure.

      5. Set url ’s host to host , buffer to the empty string, and state to path start state .

      6. If state override is given, then return.

    4. Otherwise:

      1. If c is U+005B ([), then set the [] flag .

      2. If c is U+005D (]), then unset the [] flag .

      3. Append c to buffer .

    port state

    1. If c is an ASCII digit , append c to buffer .

    2. Otherwise, if one of the following is true

      then:

      1. If buffer is not the empty string, then:

        1. Let port be the mathematical integer value that is represented by buffer in radix-10 using ASCII digits for digits with values 0 through 9.

        2. If port is greater than 2 16  − 1, validation error , return failure.

        3. Set url ’s port to null, if port is url ’s scheme ’s default port , and to port otherwise.

        4. Set buffer to the empty string.

      2. If state override is given, then return.

      3. Set state to path start state , and decrease pointer by one.

    3. Otherwise, validation error , return failure.

    file state

    1. Set url ’s scheme to " file ".

    2. If c is U+002F (/) or U+005C (\), then:

      1. If c is U+005C (\), validation error .

      2. Set state to file slash state .

    3. Otherwise, if base is non-null and base ’s scheme is " file ", switch on c :

      The EOF code point

      Set url ’s host to base ’s host , url ’s path to a copy of base ’s path , and url ’s query to base ’s query .

      U+003F (?)

      Set url ’s host to base ’s host , url ’s path to a copy of base ’s path , url ’s query to the empty string, and state to query state .

      U+0023 (#)

      Set url ’s host to base ’s host , url ’s path to a copy of base ’s path , url ’s query to base ’s query , url ’s fragment to the empty string, and state to fragment state .

      Otherwise

      1. If the substring from pointer in input does not start with a Windows drive letter , then set url ’s host to base ’s host , url ’s path to a copy of base ’s path , and then shorten url ’s path .

        This is a (platform-independent) Windows drive letter quirk.

      2. Otherwise, validation error .

      3. Set state to path state , and decrease pointer by one.

    4. Otherwise, set state to path state , and decrease pointer by one.

    file slash state

    1. If c is U+002F (/) or U+005C (\), then:

      1. If c is U+005C (\), validation error .

      2. Set state to file host state .

    2. Otherwise:

      1. If base is non-null, base ’s scheme is " file ", and the substring from pointer in input does not start with a Windows drive letter , then:

        1. If base ’s path [0] is a normalized Windows drive letter , then append base ’s path [0] to url ’s path .

          This is a (platform-independent) Windows drive letter quirk. Both url ’s and base ’s host are null under these conditions and therefore not copied.

        2. Otherwise, set url ’s host to base ’s host .

      2. Set state to path state , and decrease pointer by one.

    file host state

    1. If c is the EOF code point , U+002F (/), U+005C (\), U+003F (?), or U+0023 (#), then decrease pointer by one and then:

      1. If state override is not given and buffer is a Windows drive letter , validation error , set state to path state .

        This is a (platform-independent) Windows drive letter quirk. buffer is not reset here and instead used in the path state .

      2. Otherwise, if buffer is the empty string, then:

        1. Set url ’s host to the empty string.

        2. If state override is given, then return.

        3. Set state to path start state .

      3. Otherwise, run these steps:

        1. Let host be the result of host parsing buffer with url is not special .

        2. If host is failure, then return failure.

        3. If host is " localhost ", then set host to the empty string.

        4. Set url ’s host to host .

        5. If state override is given, then return.

        6. Set buffer to the empty string and state to path start state .

    2. Otherwise, append c to buffer .

    path start state

    1. If url is special , then:

      1. If c is U+005C (\), validation error .

      2. Set state to path state .

      3. If c is neither U+002F (/) nor U+005C (\), then decrease pointer by one.

    2. Otherwise, if state override is not given and c is U+003F (?), set url ’s query to the empty string and state to query state .

    3. Otherwise, if state override is not given and c is U+0023 (#), set url ’s fragment to the empty string and state to fragment state .

    4. Otherwise, if c is not the EOF code point :

      1. Set state to path state .

      2. If c is not U+002F (/), then decrease pointer by one.

    path state

    1. If one of the following is true

      then:

      1. If url is special and c is U+005C (\), validation error .

      2. If buffer is a double-dot path segment , shorten url ’s path , and then if neither c is U+002F (/), nor url is special and c is U+005C (\), append the empty string to url ’s path .

      3. Otherwise, if buffer is a single-dot path segment and if neither c is U+002F (/), nor url is special and c is U+005C (\), append the empty string to url ’s path .

      4. Otherwise, if buffer is not a single-dot path segment , then:

        1. If url ’s scheme is " file ", url ’s path is empty , and buffer is a Windows drive letter , then:

          1. If url ’s host is neither the empty string nor null, validation error , set url ’s host to the empty string.

          2. Replace the second code point in buffer with U+003A (:).

          This is a (platform-independent) Windows drive letter quirk.

        2. Append buffer to url ’s path .

      5. Set buffer to the empty string.

      6. If url ’s scheme is " file " and c is the EOF code point , U+003F (?), or U+0023 (#), then while url ’s path ’s size is greater than 1 and url ’s path [0] is the empty string, validation error , remove the first item from url ’s path .

      7. If c is U+003F (?), then set url ’s query to the empty string and state to query state .

      8. If c is U+0023 (#), then set url ’s fragment to the empty string and state to fragment state .

    2. Otherwise, run these steps:

      1. If c is not a URL code point and not U+0025 (%), validation error .

      2. If c is U+0025 (%) and remaining does not start with two ASCII hex digits , validation error .

      3. UTF-8 percent encode c using the path percent-encode set , and append the result to buffer .

    cannot-be-a-base-URL path state

    1. If c is U+003F (?), then set url ’s query to the empty string and state to query state .

    2. Otherwise, if c is U+0023 (#), then set url ’s fragment to the empty string and state to fragment state .

    3. Otherwise:

      1. If c is not the EOF code point , not a URL code point , and not U+0025 (%), validation error .

      2. If c is U+0025 (%) and remaining does not start with two ASCII hex digits , validation error .

      3. If c is not the EOF code point , UTF-8 percent encode c using the C0 control percent-encode set , and append the result to url ’s path [0].

    query state

    1. If encoding is not UTF-8 and one of the following is true

      then set encoding to UTF-8 .

    2. If state override is not given and c is U+0023 (#), then set url ’s fragment to the empty string and state to fragment state .

    3. Otherwise, if c is not the EOF code point :

      1. If c is not a URL code point and not U+0025 (%), validation error .

      2. If c is U+0025 (%) and remaining does not start with two ASCII hex digits , validation error .

      3. Let bytes be the result of encoding c using encoding .

      4. If bytes starts with ` &# ` and ends with 0x3B (;), then:

        1. Replace ` &# ` at the start of bytes with ` %26%23 `.

        2. Replace 0x3B (;) at the end of bytes with ` %3B `.

        3. Append bytes , isomorphic decoded , to url ’s query .

        This can happen when encoding code points using a non- UTF-8 encoding .

      5. Otherwise, for each byte in bytes :

        1. If one of the following is true

          • byte is less than 0x21 (!)

          • byte is greater than 0x7E (~)

          • byte is 0x22 ("), 0x23 (#), 0x3C (<), or 0x3E (>)

          • byte is 0x27 (') and url is special

          then append byte , percent encoded , to url ’s query .

        2. Otherwise, append a code point whose value is byte to url ’s query .

    fragment state

    1. Switching on If c : The is not the EOF code point Do nothing. U+0000 NULL Validation error . , then:

      Otherwise

      1. If c is not a URL code point and not U+0025 (%), validation error .

      2. If c is U+0025 (%) and remaining does not start with two ASCII hex digits , validation error .

      3. UTF-8 percent encode c using the fragment percent-encode set and append the result to url ’s fragment .

  12. Return url .

To set the username given a url and username , run these steps:

  1. Set url ’s username to the empty string.

  2. For each code point in username , UTF-8 percent encode it using the userinfo percent-encode set , and append the result to url ’s username .

To set the password given a url and password , run these steps:

  1. Set url ’s password to the empty string.

  2. For each code point in password , UTF-8 percent encode it using the userinfo percent-encode set , and append the result to url ’s password .

4.5. URL serializing

The URL serializer takes a URL url , an optional exclude fragment flag , and then runs these steps, returning an ASCII string :

  1. Let output be url ’s scheme and U+003A (:) concatenated.

  2. If url ’s host is non-null:

    1. Append " // " to output .

    2. If url includes credentials , then:

      1. Append url ’s username to output .

      2. If url ’s password is not the empty string, then append U+003A (:), followed by url ’s password , to output .

      3. Append U+0040 (@) to output .

    3. Append url ’s host , serialized , to output .

    4. If url ’s port is non-null, append U+003A (:) followed by url ’s port , serialized , to output .

  3. Otherwise, if url ’s host is null and url ’s scheme is " file ", append " // " to output .

  4. If url ’s cannot-be-a-base-URL flag is set, append url ’s path [0] to output .

  5. Otherwise, then for each string in url ’s path , append U+002F (/) followed by the string to output .

  6. If url ’s query is non-null, append U+003F (?), followed by url ’s query , to output .

  7. If the exclude fragment flag is unset and url ’s fragment is non-null, append U+0023 (#), followed by url ’s fragment , to output .

  8. Return output .

4.6. URL equivalence

To determine whether a URL A equals B , optionally with an exclude fragments flag , run these steps:

  1. Let serializedA be the result of serializing A , with the exclude fragment flag set if the exclude fragments flag is set.

  2. Let serializedB be the result of serializing B , with the exclude fragment flag set if the exclude fragments flag is set.

  3. Return true if serializedA is serializedB , and false otherwise.

4.7. Origin

See origin ’s definition in HTML for the necessary background information. [HTML]

A URL ’s origin is the origin returned by running these steps, switching on URL ’s scheme :

" blob "

  1. If URL ’s blob URL entry is non-null, then return URL ’s blob URL entry ’s environment ’s origin .

  2. Let url be the result of parsing URL ’s path [0].

  3. Return a new opaque origin , if url is failure, and url ’s origin otherwise.

The origin of blob:https://whatwg.org/d0360e2f-caee-469f-9a2f-87d5b0456f6f is the tuple ( https , whatwg.org , null, null).

" ftp "
" http "
" https "
" ws "
" wss "

Return a tuple consisting of URL ’s scheme , URL ’s host , URL ’s port , and null.

" file "

Unfortunate as it is, this is left as an exercise to the reader. When in doubt, return a new opaque origin .

Otherwise

Return a new opaque origin .

This does indeed mean that these URLs cannot be same-origin with themselves.

4.8. URL rendering

A URL should be rendered in its serialized form, with modifications described below, when the primary purpose of displaying a URL is to have the user make a security or trust decision. For example, users are expected to make trust decisions based on a URL rendered in the browser address bar.

4.8.1. Simplify non-human-readable or irrelevant components

Remove components that can provide opportunities for spoofing or distract from security-relevant information:

4.8.2. Elision

In a space-constrained display, URLs should be elided carefully to avoid misleading the user when making a security decision:

4.8.3. Internationalization and special characters

Internationalized domain names (IDNs), special characters, and bidirectional text should be handled with care to prevent spoofing:

5. application/x-www-form-urlencoded

The application/x-www-form-urlencoded format provides a way to encode name-value pairs.

The application/x-www-form-urlencoded format is in many ways an aberrant monstrosity, the result of many years of implementation accidents and compromises leading to a set of requirements necessary for interoperability, but in no way representing good design practices. In particular, readers are cautioned to pay close attention to the twisted details involving repeated (and in some cases nested) conversions between character encodings and byte sequences. Unfortunately the format is in widespread use due to the prevalence of HTML forms. [HTML]

5.1. application/x-www-form-urlencoded parsing

A legacy server-oriented implementation might have to support encodings other than UTF-8 as well as have special logic for tuples of which the name is ` _charset `. Such logic is not described here as only UTF-8 is conforming.

The application/x-www-form-urlencoded parser takes a byte sequence input , and then runs these steps:

  1. Let sequences be the result of splitting input on 0x26 (&).

  2. Let output be an initially empty list of name-value tuples where both name and value hold a string.

  3. For each byte sequence bytes in sequences :

    1. If bytes is the empty byte sequence, then continue .

    2. If bytes contains a 0x3D (=), then let name be the bytes from the start of bytes up to but excluding its first 0x3D (=), and let value be the bytes, if any, after the first 0x3D (=) up to the end of bytes . If 0x3D (=) is the first byte, then name will be the empty byte sequence. If it is the last, then value will be the empty byte sequence.

    3. Otherwise, let name have the value of bytes and let value be the empty byte sequence.

    4. Replace any 0x2B (+) in name and value with 0x20 (SP).

    5. Let nameString and valueString be the result of running UTF-8 decode without BOM on the percent decoding of name and value , respectively.

    6. Append ( nameString , valueString ) to output .

  4. Return output .

5.2. application/x-www-form-urlencoded serializing

The application/x-www-form-urlencoded byte serializer takes a byte sequence input and then runs these steps:

  1. Let output be the empty string.

  2. For each byte in input , depending on byte :

    0x20 (SP)

    Append U+002B (+) to output .

    0x2A (*)
    0x2D (-)
    0x2E (.)
    0x30 (0) to 0x39 (9)
    0x41 (A) to 0x5A (Z)
    0x5F (_)
    0x61 (a) to 0x7A (z)

    Append a code point whose value is byte to output .

    Otherwise

    Append byte , percent encoded , to output .

  3. Return output .

The application/x-www-form-urlencoded serializer takes a list of name-value tuples tuples , optionally with an encoding encoding override , and then runs these steps:

  1. Let encoding be UTF-8 .

  2. If encoding override is given, set encoding to the result of getting an output encoding from encoding override .

  3. Let output be the empty string.

  4. For each tuple in tuples :

    1. Let name be the result of serializing the result of encoding tuple ’s name, using encoding .

    2. Let value be tuple ’s value.

    3. If value is a file, then set value to value ’s filename.

    4. Set value to the result of serializing the result of encoding value , using encoding .

    5. If tuple is not the first pair in tuples , then append U+0026 (&) to output .

    6. Append name , followed by U+003D (=), followed by value , to output .
  5. Return output .

The HTML standard invokes this algorithm with values that are files. [HTML]

5.3. Hooks

The application/x-www-form-urlencoded string parser takes a string input , UTF-8 encodes it, and then returns the result of application/x-www-form-urlencoded parsing it.

6. API

6.1. URL class 

[Exposed=(Window,Worker),
 LegacyWindowAlias=webkitURL]
interface URL {
  constructor(USVString url, optional USVString base);
  stringifier attribute USVString href;
  readonly attribute USVString origin;
           attribute USVString protocol;
           attribute USVString username;
           attribute USVString password;
           attribute USVString host;
           attribute USVString hostname;
           attribute USVString port;
           attribute USVString pathname;
           attribute USVString search;
  [SameObject] readonly attribute URLSearchParams searchParams;
           attribute USVString hash;
  USVString toJSON();
};

A URL object has an associated url (a URL ) and query object (a URLSearchParams object).

The URL( url , base ) constructor, when invoked, must run these steps:

  1. Let parsedBase be null.

  2. If base is given, then:

    1. Let parsedBase be the result of running the basic URL parser on base .

    2. If parsedBase is failure, then throw a TypeError .

  3. Let parsedURL be the result of running the basic URL parser on url with parsedBase .

  4. If parsedURL is failure, then throw a TypeError .

  5. Let query be parsedURL ’s query , if that is non-null, and the empty string otherwise.

  6. Let result be a new URL object.

  7. Set result ’s url to parsedURL .

  8. Set result ’s query object to a new URLSearchParams object using query , and then set that query object ’s url object to result .

  9. Return result .

To parse a string into a URL without using a base URL , invoke the URL constructor with a single argument: 

var input = "https://example.org/💩",
    url = new URL(input)
url
.

pathname

//
"/%F0%9F%92%A9"

This throws an exception if the input is not an absolute-URL-with-fragment string : 

try {
  var url = new URL("/🍣🍺")
} catch(e) {
  // that happened

}

A base URL is necessary if the input is a relative-URL string : 

var input = "/🍣🍺",
    url = new URL(input, document.baseURI)
url
.

href

//
"https://url.spec.whatwg.org/%F0%9F%8D%A3%F0%9F%8D%BA"

A URL object can be used as base URL (while IDL requires a string as argument, a URL object stringifies to its href attribute value): 

var url = new URL("🏳️‍🌈", new URL("https://pride.example/hello-world"))
url
.

pathname

//
"/%F0%9F%8F%B3%EF%B8%8F%E2%80%8D%F0%9F%8C%88"

The href attribute’s getter and the toJSON() method, when invoked, must return the serialization of context object ’s url .

The href attribute’s setter must run these steps:

  1. Let parsedURL be the result of running the basic URL parser on the given value.

  2. If parsedURL is failure, then throw a TypeError .

  3. Set context object ’s url to parsedURL .

  4. Empty context object ’s query object ’s list .

  5. Let query be context object ’s url ’s query .

  6. If query is non-null, then set context object ’s query object ’s list to the result of parsing query .

The origin attribute’s getter must return the serialization of context object ’s url ’s origin . [HTML]

The protocol attribute’s getter must return context object url ’s scheme , followed by U+003A (:).

The protocol attribute’s setter must basic URL parse the given value, followed by U+003A (:), with context object ’s url as url and scheme start state as state override .

The username attribute’s getter must return context object ’s url ’s username .

The username attribute’s setter must run these steps:

  1. If context object ’s url cannot have a username/password/port , then return.

  2. Set the username given context object ’s url and the given value.

The password attribute’s getter must return context object ’s url ’s password .

The password attribute’s setter must run these steps:

  1. If context object ’s url cannot have a username/password/port , then return.

  2. Set the password given context object ’s url and the given value.

The host attribute’s getter must run these steps:

  1. Let url be context object ’s url .

  2. If url ’s host is null, return the empty string.

  3. If url ’s port is null, return url ’s host , serialized .

  4. Return url ’s host , serialized , followed by U+003A (:) and url ’s port , serialized .

The host attribute’s setter must run these steps:

  1. If context object ’s url ’s cannot-be-a-base-URL flag is set, then return.

  2. Basic URL parse the given value with context object ’s url as url and host state as state override .

If the given value for the host attribute’s setter lacks a port , context object ’s url ’s port will not change. This can be unexpected as host attribute’s getter does return a URL-port string so one might have assumed the setter to always "reset" both.

The hostname attribute’s getter must run these steps:

  1. If context object ’s url ’s host is null, return the empty string.

  2. Return context object ’s url ’s host , serialized .

The hostname attribute’s setter must run these steps:

  1. If context object ’s url ’s cannot-be-a-base-URL flag is set, then return.

  2. Basic URL parse the given value with context object ’s url as url and hostname state as state override .

The port attribute’s getter must run these steps:

  1. If context object ’s url ’s port is null, return the empty string.

  2. Return context object ’s url ’s port , serialized .

The port attribute’s setter must run these steps:

  1. If context object ’s url cannot have a username/password/port , then return.

  2. If the given value is the empty string, then set context object ’s url ’s port to null.

  3. Otherwise, basic URL parse the given value with context object ’s url as url and port state as state override .

The pathname attribute’s getter must run these steps:

  1. If context object ’s url ’s cannot-be-a-base-URL flag is set, then return context object ’s url ’s path [0].

  2. If context object ’s url ’s path is empty , then return the empty string.

  3. Return U+002F (/), followed by the strings in context object ’s url ’s path (including empty strings), if any, separated from each other by U+002F (/).

The pathname attribute’s setter must run these steps:

  1. If context object ’s url ’s cannot-be-a-base-URL flag is set, then return.

  2. Empty context object ’s url ’s path .

  3. Basic URL parse the given value with context object ’s url as url and path start state as state override .

The search attribute’s getter must run these steps:

  1. If context object ’s url ’s query is either null or the empty string, return the empty string.

  2. Return U+003F (?), followed by context object ’s url ’s query .

The search attribute’s setter must run these steps:

  1. Let url be context object ’s url .

  2. If the given value is the empty string, set url ’s query to null, empty context object ’s query object ’s list , and then return.

  3. Let input be the given value with a single leading U+003F (?) removed, if any.

  4. Set url ’s query to the empty string.

  5. Basic URL parse input with url as url and query state as state override .

  6. Set context object ’s query object ’s list to the result of parsing input .

The searchParams attribute’s getter must return context object ’s query object .

The hash attribute’s getter must run these steps:

  1. If context object ’s url ’s fragment is either null or the empty string, return the empty string.

  2. Return U+0023 (#), followed by context object ’s url ’s fragment .

The hash attribute’s setter must run these steps:

  1. If the given value is the empty string, then set context object ’s url ’s fragment to null and return.

  2. Let input be the given value with a single leading U+0023 (#) removed, if any.

  3. Set context object ’s url ’s fragment to the empty string.

  4. Basic URL parse input with context object ’s url as url and fragment state as state override .

6.2. URLSearchParams class 

[Exposed=(Window,Worker)]
interface URLSearchParams {
  constructor(optional (sequence<sequence<USVString>> or record<USVString, USVString> or USVString) init = "");
  void append(USVString name, USVString value);
  void delete(USVString name);
  USVString? get(USVString name);
  sequence<USVString> getAll(USVString name);
  boolean has(USVString name);
  void set(USVString name, USVString value);
  void sort();
  iterable<USVString, USVString>;
  stringifier;
};

Constructing and stringifying a URLSearchParams object is fairly straightforward: 

let params = new URLSearchParams({key: "730d67"})
params
.

toString

()


//
"key=730d67"

A URLSearchParams object has an associated list of name-value pairs, which is initially empty.

A URLSearchParams object has an associated url object , which is initially null.

To create a new URLSearchParams object using init , run these steps:

  1. Let query be a new URLSearchParams object.

  2. If init is a sequence , then for each pair in init :

    1. If pair does not contain exactly two items, then throw a TypeError .

    2. Append a new name-value pair whose name is pair ’s first item, and value is pair ’s second item, to query ’s list .

  3. Otherwise, if init is a record , then for each name → value in init , append a new name-value pair whose name is name and value is value , to query ’s list .

  4. Otherwise, init is a string, then set query ’s list to the result of parsing init .

  5. Return query .

A URLSearchParams object’s update steps are to run these steps:

  1. Let query be the serialization of URLSearchParams object’s list .

  2. If query is the empty string, then set query to null.

  3. Set url object ’s url ’s query to query .

The URLSearchParams( init ) constructor, when invoked, must run these steps:

  1. If init is a string and starts with U+003F (?), remove the first code point from init .

  2. Return a new URLSearchParams object using init .

The append( name , value ) method, when invoked, must run these steps:

  1. Append a new name-value pair whose name is name and value is value , to list .

  2. Run the update steps .

The delete( name ) method, when invoked, must run these steps:

  1. Remove all name-value pairs whose name is name from list .

  2. Run the update steps .

The get( name ) method, when invoked, must return the value of the first name-value pair whose name is name in list , if there is such a pair, and null otherwise.

The getAll( name ) method, when invoked, must return the values of all name-value pairs whose name is name , in list , in list order, and the empty sequence otherwise.

The has( name ) method, when invoked, must return true if there is a name-value pair whose name is name in list , and false otherwise.

The set( name , value ) method, when invoked, must run these steps:

  1. If there are any name-value pairs whose name is name , in list , set the value of the first such name-value pair to value and remove the others.

  2. Otherwise, append a new name-value pair whose name is name and value is value , to list .

  3. Run the update steps .

It can be useful to sort the name-value pairs in a URLSearchParams object, in particular to increase cache hits. This can be accomplished through invoking the sort() method: 

const url = new URL("https://example.org/?q=🏳️‍🌈&key=e1f7bc78");
url.searchParams.sort();
url
.

search

;


//
"?key=e1f7bc78&q=%F0%9F%8F%B3%EF%B8%8F%E2%80%8D%F0%9F%8C%88"

To avoid altering the original input, e.g., for comparison purposes, construct a new URLSearchParams object: 

const sorted = new URLSearchParams(url.search)
sorted
.

sort

()

The sort() method, when invoked, must run these steps:

  1. Sort all name-value pairs, if any, by their names. Sorting must be done by comparison of code units. The relative order between name-value pairs with equal names must be preserved.

  2. Run the update steps .

The value pairs to iterate over are the list name-value pairs with the key being the name and the value being the value.

The stringification behavior must return the serialization of the URLSearchParams object’s list .

6.3. URL APIs elsewhere

A standard that exposes URLs , should expose the URL as a string (by serializing an internal URL ). A standard should not expose a URL using a URL object. URL objects are meant for URL manipulation. In IDL the USVString type should be used.

The higher-level notion here is that values are to be exposed as immutable data structures.

If a standard decides to use a variant of the name "URL" for a feature it defines, it should name such a feature "url" (i.e., lowercase and with an "l" at the end). Names such as "URL", "URI", and "IRI" should not be used. However, if the name is a compound, "URL" (i.e., uppercase) is preferred, e.g., "newURL" and "oldURL".

The EventSource and HashChangeEvent interfaces in HTML are examples of proper naming. [HTML]

Acknowledgments

There have been a lot of people that have helped make URLs more interoperable over the years and thereby furthered the goals of this standard. Likewise many people have helped making this standard what it is today.

With that, many thanks to 100の人, Adam Barth, Addison Phillips, Albert Wiersch, Alex Christensen, Alexandre Morgaut, Andrew Sullivan, Arkadiusz Michalski, Behnam Esfahbod, Bobby Holley, Boris Zbarsky, Brad Hill, Brandon Ross, Chris Dumez, Chris Rebert, Corey Farwell, Dan Appelquist, Daniel Bratell, Daniel Stenberg, David Burns, David Håsäther, David Sheets, David Singer, David Walp, Domenic Denicola, Emily Schechter, Emily Stark, Eric Lawrence, Erik Arvidsson, Gavin Carothers, Geoff Richards, Glenn Maynard, Gordon P. Hemsley, Henri Sivonen, Ian Hickson, Ilya Grigorik, Italo A. Casas, Jakub Gieryluk, James Graham, James Manger, James Ross, Jeffrey Posnick, Jeffrey Yasskin, Joe Duarte, Joshua Bell, Jxck, 田村健人 (Kent TAMURA), Kevin Grandon, Kornel Lesiński, Larry Masinter, Leif Halvard Silli, Mark Davis, Marcos Cáceres, Marijn Kruisselbrink, Martin Dürst, Mathias Bynens, Michael Peick, Michael™ Smith, Michal Bukovský, Michel Suignard, Noah Levitt, Peter Occil, Philip Jägenstedt, Philippe Ombredanne, Prayag Verma, Rimas Misevičius, Robert Kieffer, Rodney Rehm, Roy Fielding, Ryan Sleevi, Sam Ruby, Santiago M. Mola, Sebastian Mayr, Simon Pieters, Simon Sapin, Steven Vachon, Stuart Cook, Sven Uhlig, Tab Atkins, 吉野剛史 (Takeshi Yoshino), Tantek Çelik, Tiancheng "Timothy" Gu, Tim Berners-Lee, 簡冠庭 (Tim Guan-tin Chien), Titi_Alone, Tomek Wytrębowicz, Trevor Rowbotham, Valentin Gosu, Vyacheslav Matva, Wei Wang, 山岸和利 (Yamagishi Kazutoshi), and 成瀬ゆい (Yui Naruse) for being awesome!

This standard is written by Anne van Kesteren ( Mozilla , annevk@annevk.nl ).

Copyright © 2020 WHATWG (Apple, Google, Mozilla, Microsoft). This work is licensed under a Creative Commons Attribution 4.0 International License .

Index

Terms defined by this specification

Terms defined by reference

References

Normative References

[BIDI]
Mark Davis; Aharon Lanin; Andrew Glass. Unicode Bidirectional Algorithm . 12 February 2020. Unicode Standard Annex #9. URL: https://www.unicode.org/reports/tr9/tr9-42.html
[DOM]
Anne van Kesteren. DOM Standard . Living Standard. URL: https://dom.spec.whatwg.org/
[ENCODING]
Anne van Kesteren. Encoding Standard . Living Standard. URL: https://encoding.spec.whatwg.org/
[FILEAPI]
Marijn Kruisselbrink; Arun Ranganathan. File API . URL: https://w3c.github.io/FileAPI/
[HTML]
Anne van Kesteren; et al. HTML Standard . Living Standard. URL: https://html.spec.whatwg.org/multipage/
[IANA-URI-SCHEMES]
Uniform Resource Identifier (URI) Schemes . URL: https://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml
[INFRA]
Anne van Kesteren; Domenic Denicola. Infra Standard . Living Standard. URL: https://infra.spec.whatwg.org/
[MEDIA-SOURCE]
Matthew Wolenetz; et al. Media Source Extensions™ . URL: https://w3c.github.io/media-source/
[PSL]
Public Suffix List . Mozilla Foundation.
[RFC4291]
R. Hinden; S. Deering. IP Version 6 Addressing Architecture . February 2006. Draft Standard. URL: https://tools.ietf.org/html/rfc4291
[UTS46]
Mark Davis; Michel Suignard. Unicode IDNA Compatibility Processing . 6 February 2020. Unicode Technical Standard #46. URL: https://www.unicode.org/reports/tr46/tr46-25.html
[WEBIDL]
Boris Zbarsky. Web IDL . URL: https://heycam.github.io/webidl/

Informative References

[IDNFAQ]
Internationalized Domain Names (IDN) FAQ . URL: https://unicode.org/faq/idn.html
[RFC1034]
P.V. Mockapetris. Domain names - concepts and facilities . November 1987. Internet Standard. URL: https://tools.ietf.org/html/rfc1034
[RFC3986]
T. Berners-Lee; R. Fielding; L. Masinter. Uniform Resource Identifier (URI): Generic Syntax . January 2005. Internet Standard. URL: https://tools.ietf.org/html/rfc3986
[RFC3987]
M. Duerst; M. Suignard. Internationalized Resource Identifiers (IRIs) . January 2005. Proposed Standard. URL: https://tools.ietf.org/html/rfc3987
[RFC5952]
S. Kawamura; M. Kawashima. A Recommendation for IPv6 Address Text Representation . August 2010. Proposed Standard. URL: https://tools.ietf.org/html/rfc5952
[RFC6454]
A. Barth. The Web Origin Concept . December 2011. Proposed Standard. URL: https://tools.ietf.org/html/rfc6454
[RFC7595]
D. Thaler, Ed.; T. Hansen; T. Hardie. Guidelines and Registration Procedures for URI Schemes . June 2015. Best Current Practice. URL: https://tools.ietf.org/html/rfc7595
[RFC791]
J. Postel. Internet Protocol . September 1981. Internet Standard. URL: https://tools.ietf.org/html/rfc791
[UTR36]
Mark Davis; Michel Suignard. Unicode Security Considerations . 19 September 2014. Unicode Technical Report #36. URL: https://www.unicode.org/reports/tr36/tr36-15.html
[UTS39]
Mark Davis; Michel Suignard. Unicode Security Mechanisms . 6 February 2020. Unicode Technical Standard #39. URL: https://www.unicode.org/reports/tr39/tr39-22.html

IDL Index 

[Exposed=(Window,Worker),
 LegacyWindowAlias=webkitURL]
interface URL {
  constructor(USVString url, optional USVString base);
  stringifier attribute USVString href;
  readonly attribute USVString origin;
           attribute USVString protocol;
           attribute USVString username;
           attribute USVString password;
           attribute USVString host;
           attribute USVString hostname;
           attribute USVString port;
           attribute USVString pathname;
           attribute USVString search;
  [SameObject] readonly attribute URLSearchParams searchParams;
           attribute USVString hash;
  USVString toJSON();
};
[Exposed=(Window,Worker)]
interface URLSearchParams {
  constructor(optional (sequence<sequence<USVString>> or record<USVString, USVString> or USVString) init = "");
  void append(USVString name, USVString value);
  void delete(USVString name);
  USVString? get(USVString name);
  sequence<USVString> getAll(USVString name);
  boolean has(USVString name);
  void set(USVString name, USVString value);
  void sort();
  iterable<USVString, USVString>;
  stringifier;
};