This lib is written with ES6, so it doesn't get parsed correctly with browserify:

Parse error at -:3082,12
SyntaxError: Unexpected token: name (x)
Error

It'd be great if there were a browser version available to serve as a polyfill for IE.

Has URLUtils.searchParams been removed?

What is the best way to do this?

Unrelated: Why doesn't URL use the same terminology as RFC3986? (protocol -> scheme, hash -> fragment, search -> query, etc.)

Relative from: https://github.com/Automattic/mongoose/issues/10713

In https://github.com/jsdom/whatwg-url/blob/master/src/encoding.js line 2 says:

"use strict";
const utf8Encoder = new TextEncoder();
const utf8Decoder = new TextDecoder("utf-8", { ignoreBOM: true });

But node show an error message:

/app/node_modules/whatwg-url/dist/encoding.js:2
const utf8Encoder = new TextEncoder();
                    ^

ReferenceError: TextEncoder is not defined
    at Object.<anonymous> (/app/node_modules/whatwg-url/dist/encoding.js:2:21)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (/app/node_modules/whatwg-url/dist/url-state-machine.js:5:34)
    at Module._compile (internal/modules/cjs/loader.js:778:30)

My node version:

$ node -v
v14.17.5

$ npm -v
6.14.14

The version in package.json of dependency is:

{
  "_from": "whatwg-url@^9.1.0",
  "_id": "[email protected]",

I installed from mongoose package but have last version of whatwg-url package.

There's a problem with our URLSearchParams constructor. We explicitly check the objects prototype there, which is not correct, I think, which then fails one of the w3c tests.

In URL.js:

function URL(url) {
  if (!new.target) {
    throw new TypeError(
      "Failed to construct 'URL'. Please use the 'new' operator; this constructor cannot be called as a function."
    );
  }

Where does this variable new even come from?

I'm trying to work with this library. When I try to use it in a new URL(url) way, it says: Type Error: URL is not a constructor

When I try to use it as function, it says: Type Error: Failed to construct 'URL'. Please use the 'new' operator; this constructor cannot be called as a function.

I'm using node v6.10.3

I'm not at all a JS expert but after getting silently upgraded to 8.2.0 I'm seeing the following error in my build:

TypeError: Cannot read property 'some' of undefined
    at Object.exports.install (/Users/mhughe008/development/vise/node_modules/whatwg-url/dist/URL.js:84:20)
    at exports.installInterfaces (/Users/mhughe008/development/vise/node_modules/jsdom/lib/jsdom/living/interfaces.js:202:24)
    at setupWindow (/Users/mhughe008/development/vise/node_modules/jsdom/lib/jsdom/browser/Window.js:63:3)
    at new Window (/Users/mhughe008/development/vise/node_modules/jsdom/lib/jsdom/browser/Window.js:107:3)
    at exports.createWindow (/Users/mhughe008/development/vise/node_modules/jsdom/lib/jsdom/browser/Window.js:38:10)

Failing at this section of code:

exports.install = (globalObject, globalNames) => {
  if (!globalNames.some(globalName => exposed.has(globalName))) {
    return;
  }
  class URL {
    constructor(url) {
      if (arguments.length < 1) {

This is using jsdom 16.1.2. Best I can tell this was probably caused by the upgrade to webidl-conversions:

-    "webidl-conversions": "^5.0.0"
+    "webidl-conversions": "^6.1.0"

Safari in macOS High Sierra and iOS 11.02 13604.1.38.1.6 fails on URL.js with:

new.target can't come after a prefix operator

Issue is webkit, but workaround needs to be applied to webidl2js (and a new version of whatwg-url built). See https://github.com/jsdom/webidl2js/issues/78.

A validation error indicates a mismatch between input and valid input. User agents, especially conformance checkers, are encouraged to report them somewhere.

NOTE:

A validation error does not mean that the parser terminates. Termination of a parser is always stated explicitly, e.g., through a return statement.

It is useful to signal validation errors as error-handling can be non-intuitive, legacy user agents might not implement correct error-handling, and the intent of what is written might be unclear to other developers.

It seems that this script throws 'TypeError: Invalid URL' upon a validation error, but that terminates the parser, preventing the availability of the parsed URL parts. This confuses, for example, Live URL Viewer.

Got this error on Docker Ubuntu, whatwg-url got from npm i whatwg-url:

const utf8Encoder = new TextEncoder();
                    ^

ReferenceError: TextEncoder is not defined

[email protected]:/app/ApiServer# npm start

> [email protected] start /app/ApiServer
> node index.js

/app/ApiServer/node_modules/whatwg-url/dist/encoding.js:2
const utf8Encoder = new TextEncoder();
                    ^

ReferenceError: TextEncoder is not defined
    at Object.<anonymous> (/app/ApiServer/node_modules/whatwg-url/dist/encoding.js:2:21)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
    at Function.Module._load (internal/modules/cjs/loader.js:585:3)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (/app/ApiServer/node_modules/whatwg-url/dist/url-state-machine.js:5:34)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] start: `node index.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2021-09-07T11_30_49_084Z-debug.log

Solution helped for me:

nano /app/MyProj/node_modules/whatwg-url/dist/encoding.js

and just add new line:

const { TextEncoder, TextDecoder } = require("util");

When importing and using URL in an app created with create-react-app, building a production app fails with:

> react-scripts build

Creating an optimized production build...
Failed to compile.

static/js/main.38ef0137.js from UglifyJs
SyntaxError: Name expected [./~/whatwg-url/lib/url-state-machine.js:6,0]

Start with the input:

http://xn--ls8h/

It works, as it should, and the result is the same. It's the poop emoji, in case you were wondering. It's in the WPT tests.

Anyway, we can just start adding text to the end of this. For example:

(Input) -> (JSDOM output)

"http://xn--ls8h=/" -> "http://xn--js8hea/"
"http://xn--ls8h==/" -> "http://xn--hs8hdh/"
"http://xn--ls8h===/" -> "http://xn--gs8hcfj/"

"http://xn--ls8h===helloworldhowareyoutoday/" -> "http://xn--gs8hcaceekclworldhowareyoutoday/"
                   ^^^^^^^^^^^^^^^^^^^^^^^^                              ^^^^^^^^^^^^^^^^^^^

All of these inputs are invalid. Safari refuses to parse them, and my own IDNA implementation agrees. The tail part of the Punycode domain (after the xn--) should only consist of ASCII alphanumerics; having "=" signs in there (in the input) is clearly invalid.

This removes the file host state and parses file URL authorities using the generic authority state.

The outward behaviour is unchanged and all tests are passing.

One thing that might warrant another look is that the host parser can now return an empty host. Empty hosts are allowed in file URLs but not in other special URLs. This is now enforced elsewhere.

There are checks in place that return failure if the URL is a file URL that has credentials or a port. There is some discussion about whether that should be changed in https://github.com/whatwg/url/issues/548 and https://github.com/whatwg/url/issues/97. This should help with whatever is decided there.

The URL standard is getting more sophisticated about validation errors. See e.g. https://github.com/whatwg/url/issues/406.

There is also the issue where the URL standard contains two different ways of checking URL validity, which are probably not equivalent; see some discussion in https://github.com/whatwg/url/issues/437#issuecomment-488022818. I'll call these "parser validity" and "writing validity", since the latter is derived from the URL writing section.

It would be very neat if this package could provide:

• Whether a URL is writing-valid. (At least for "absolute-URL-with-fragment string"; maybe relative URLs are out of scope.)
• Whether a URL is parser-valid, including the name of any validation error(s) triggered during parsing.
• Surface this information in the live URL viewer.
• A test corpus of valid/invalid URLs, maybe drawn from urltestdata.json. We should at least have one parser-invalid URL for every named parser validation error, and it'd also be good to have writing-invalid URLs derived from systematically violating every rule in the writing URLs section.
• Bonus: a fuzzer that generates strings and tests if they are parser-valid but writing-invalid, or parser-invalid but writing-valid.

Right now we have some not-very-well-maintained code to track validation errors when the spec says to do so. It does this by setting urlStateMachine.parseError to true. ("parse error" is the old name for "validation error".) We'd probably want to update this to an array of strings (representing validation error names) that get pushed. And we should probably expose it on the URL record; that's #61. Or maybe elsewhere, since if you get a parsing failure, there's no URL record, so any fatal validation errors would be invisible this way?

We have some known places where we miss validation errors that the spec asks about; I think most of them are due to not implementing "is a URL code point", which #59 tried to address, but got stalled.

One reason why I was ambivalent about #59 is that it's a good bit of code, and potentially performance cost, for validity checking. At least one major consumer of this project, jsdom, prefers speed, and has no use for validity checking. It would also add memory consumption for every URL record, especially if as per the above we go beyond the simple boolean to instead be an array of strings.

On the other hand, there aren't many other consumers of this project for "production" use. Everyone else should probably be using the built-in URL constructor in their environment (either Node or the browser). And we're not exactly tuned for speed anyway; there is probably lower-hanging fruit we're ignoring.

We could address this in a number of ways, e.g.:

• Take the code size hit, and benchmark the speed hit. (Probably after #155 gets us some basic benchmarks.) Maybe it's fine for jsdom's purposes.

• Use source code rewriting to prune out validity-related parts of the parser code, so that we end up with two output .js files, e.g. url-state-machine-with-validy-checking.js and url-state-machine.js. (Related: #153, and maybe #155 because if we remove the p() source code rewriting then this would just be adding some back.) This would result in a larger npm package size but a smaller loaded-into-memory size.

As far as I can tell there's no low-level access to parseError. This makes it impossible to implement a simple validator on top of this code. (It would be even better for that to have detailed messages of course, but a boolean is a start.)

Perhaps you were waiting since it's not a pretty thing to have to implement. I'm doing it for my own purposes, so here's a version in what I think is your style if you want it.