I use furl(v1.1) in http proxy app, and receive url in server side(the http proxy received from client):

>>> url_received = 'http://example.com/?redirect=http%3A%2F%2Fwww.example.com%2F'
>>> parsed = furl.furl(url_received)

After do something with the parsed url, and generate url in client side(to be sent to the real server):

>>> url_to_be_sent = parsed.url 
>>> print(url_to_be_sent)
'http://example.com/?redirect=http://www.example.com/'
>>> print(url_to_be_sent.query)
'redirect=http://www.example.com/'

Should the "redirect=http://www.example.com/" part be percent encoded as original query "redirect=http%3A%2F%2Fwww.example.com%2F"?

Hey folks. Thanks for your work on this project!

I wanted to package furl for NixOS, and the build throws this error for me:

.........................F..................................................
======================================================================
FAIL: test_join (test_furl.TestFurl)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/build/furl-2.1.0/tests/test_furl.py", line 2042, in test_join
    assert f.url == 'wss://slrp.com/foo:1'
AssertionError

----------------------------------------------------------------------
Ran 76 tests in 9.423s

It seems only this single test fails, all others are fine? That leads me to believe it's not a peculiarity of my packaging that's responsible.

Anyways, for context here's the build nix expression I used:

{ lib, python3, radicale2 }:
with python3.pkgs;

buildPythonPackage rec {
    pname = "furl";
    version = "2.1.0";

    src = fetchPypi {
      inherit pname version;
      sha256 = "08dnw3bs1mk0f1ccn466a5a7fi1ivwrp0jspav9arqpf3wd27q60";
    };

    propagatedBuildInputs = [
      orderedmultidict six
    ];

    checkInputs = [
      flake8
    ];
}

this module is great and easy when manipulating urls

adding "encoding" parameter to ".tostr" function would be nice rather than encoding everything in "utf8" since "utf8" is not only option in real life

thanks

Hi,

I'm using your module directly and indirectly via the SQLAlchemy-Utils module and its URLType column type which coerces a URL into a furl object.

I've found an issue when I try and use the URLType type on Python 3.x, I get the following error when I try and query an object using such a column:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../venv-3.5/lib/python3.5/site-packages/sqlalchemy/orm/query.py", line 2588, in all
    return list(self)
  File ".../venv-3.5/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 86, in instances
    util.raise_from_cause(err)
  File ".../venv-3.5/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 189, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=exc_value)
  File ".../venv-3.5/lib/python3.5/site-packages/sqlalchemy/util/compat.py", line 183, in reraise
    raise value
  File ".../venv-3.5/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 71, in instances
    rows = [proc(row) for row in fetch]
  File ".../venv-3.5/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 71, in <listcomp>
    rows = [proc(row) for row in fetch]
  File ".../venv-3.5/lib/python3.5/site-packages/sqlalchemy/orm/loading.py", line 379, in _instance
    instance = session_identity_map.get(identitykey)
  File ".../venv-3.5/lib/python3.5/site-packages/sqlalchemy/orm/identity.py", line 146, in get
    if key not in self._dict:
TypeError: unhashable type: 'furl'

I've managed to fix this by monkeypatching the furl class with a __hash__ method like so:

from sqlalchemy_utils import URLType
from furl import furl

def furl_hash(self):
    return hash(self.url)

furl.__hash__ = furl_hash

class Foo(Model):
    url = Column(URLType)
    ...

Would it be possible to add such a method? I'm not sure if I'm 100% correct just using the value of self.url here but it looked close enough.

This implements the test from the comments of https://github.com/gruns/furl/issues/32 and demonstrates a regression (bug was reported in 0.3.mumble, fixed in 0.3.8, and is now broken in 0.4.1 and 0.4.2)

Python 3.9.1 #121

======================================================================
FAIL: test_join (test_furl.TestFurl)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shadchin/github/furl/tests/test_furl.py", line 2060, in test_join
    assert f.url == 'wss://slrp.com/foo:1'
AssertionError

Python 3.9.2

======================================================================
FAIL: test_join (test_furl.TestFurl)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shadchin/github/furl/tests/test_furl.py", line 2060, in test_join
    assert f.url == 'wss://slrp.com/foo:1'
AssertionError

======================================================================
FAIL: test_add (test_furl.TestQuery)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shadchin/github/furl/tests/test_furl.py", line 678, in test_add
    assert q.params.allitems() == runningsum
AssertionError

======================================================================
FAIL: test_params (test_furl.TestQuery)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shadchin/github/furl/tests/test_furl.py", line 806, in test_params
    assert item1 == item2
AssertionError

======================================================================
FAIL: test_remove (test_furl.TestQuery)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shadchin/github/furl/tests/test_furl.py", line 720, in test_remove
    assert len(q.params) == 0
AssertionError

======================================================================
FAIL: test_set (test_furl.TestQuery)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shadchin/github/furl/tests/test_furl.py", line 687, in test_set
    assert q.params.allitems() == items_omd.allitems()
AssertionError

======================================================================
FAIL: test_various (test_furl.TestQuery)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/shadchin/github/furl/tests/test_furl.py", line 650, in test_various
    assert q.params.allitems() == items.allitems()
AssertionError

Description

Hi :wave: I ran the DeepSource static analyzer on the forked copy of this repo and found some interesting code quality issues. This PR fixes a few of them.

Summary of changes

• fixed ambiguous variable names
• Used literal syntax instead of function calls to create data structure
• Refactored if expression
• Refactored unnecessary else / elif when if block has a return statement
• added .deepsource.toml config

In [1]: furl('http://www.google.com/?one=1&two=2').set({'one': 'one'})
Out[1]: furl('http://www.google.com/?one=one')

In [2]: furl('http://www.google.com/?one=1&two=2').remove('one').add({'one': 'one'})
Out[2]: furl('http://www.google.com/?two=2&one=one')

How to realize In [2] more simply

I do not get this error with furl-0.3.7. I get this error with furl-0.4.2.

In [3]: furl.furl()
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-3-a2bd23daefba> in <module>()
----> 1 furl.furl()

/Library/Python/2.7/site-packages/furl/furl.pyc in __init__(self, url, strict)
    847         self.strict = strict
    848 
--> 849         self.load(url)  # Raises ValueError on invalid url.
    850 
    851     def load(self, url):

/Library/Python/2.7/site-packages/furl/furl.pyc in load(self, url)
    865         # Python 2.7+. In Python <= 2.6, urlsplit() doesn't raise a
    866         # ValueError on malformed IPv6 addresses.
--> 867         tokens = urlsplit(url)
    868 
    869         self.netloc = tokens.netloc  # Raises ValueError in Python 2.7+.

/Library/Python/2.7/site-packages/furl/furl.pyc in urlsplit(url)
   1308         url = _set_scheme(url, 'http')
   1309     toks = urllib.parse.urlsplit(url)
-> 1310     return urllib.parse.SplitResult(*_change_urltoks_scheme(toks, original_scheme))
   1311 
   1312 

AttributeError: 'Module_six_moves_urllib_parse' object has no attribute 'SplitResult'

It appears that the 2 keyword arguments described in the API documentation aren't there yet:

>>> furl.__version__
'2.0.0'
>>> f.query.encode(dont_quote='@')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: encode() got an unexpected keyword argument 'don't_quote'
>>> f.tostr(query_dont_quote=True)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: tostr() got an unexpected keyword argument 'query_don't_quote'

As Below

In [1]: from furl import urlsplit

In [2]: urlsplit('http://x.com/api/test?url=http://a.com')
Out[2]: SplitResult(scheme='http', netloc='x.com', path='/api/test', query='url=http://a.com', fragment='')

In [3]: urlsplit('/api/test?url=http://a.com')
Out[3]: SplitResult(scheme='/api/test?url=http', netloc='a.com', path='', query='', fragment='')

In [4]: from furl import furl

In [5]: furl('http://x.com/api/test?url=http://a.com').query
Out[5]: Query('url=http://a.com')

In [6]: furl('/api/test?url=http://a.com').query
Out[6]: Query('')

About Scheme

I not find any description about scheme's format, just a list of registered schemes.

• http://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml

So I just add one line code to check whether ? in beforeColon to solve above problem.

Preferred Solution

What's your opinion?

Hi, ClearML uses furl to remove a user from a git remote URL, so furl removes username when the remote URL is like "ssh://[email protected]:1234/path/to/repo.git" (with "ssh://git@"):

• Ok: furl.furl("[email protected]:1234/path/to/repo.git").remove(username=True, password=True).tostr() expected: [email protected]:1234/path/to/repo.git received: [email protected]:1234/path/to/repo.git

• Not as expected: furl.furl("ssh://[email protected]:1234/path/to/repo.git").remove(username=True, password=True).tostr() expected: ssh://[email protected]:1234/path/to/repo.git received: ssh://0.0.0.0:1234/path/to/repo.git

should I use more specific lib/tool to parse git-like URLs?

from furl import furl


def url() -> furl:
    return furl("http://foo.com/?a=1&b=2")


print(url().set({"a": 3}))  # http://foo.com/?a=3
print(url().add({"a": 3}))  # http://foo.com/?a=1&b=2&a=3

# i would like to be able do this in one method:
print(url().remove(["a"]).add({"a": 3}))  # http://foo.com/?b=2&a=3

the current behavior of add where it supports multiple query params with the same value seems like an edge case imo

This PR is based on #165 and fixes #124. If #124 is too controversial, I can rebase this PR on master, just tell me :)

Depending on the contexts, people does not always agree on brackets usage with IPv6. For instance, the ipaddress core module handles IPv6 without brackets as do some other low-level python modules. Another example: DNS AAAA records usually handles IPv6 without brackets. However in a URL context, with a port number being optionally associated with an IP, the brackets become mandatory to distinguish the columns inside the IP from the column that separate the port number from the IP.

As suggested in #124, I think this would be nice if a convenience utility like furl would tolerate and correct any usage. This would allow users to not need to check and correct manually the IPv6 they get from other libraries that may generate IPv6 without brackets because this is relevant in their development context. For instance a DNS SRV record being read with aiodns.

This is what this PR allows. There is not much code here, everything lies here:

    def host(self, host):
        ...
        if (
            is_valid_ipv6(host)
            and callable_attr(host, 'startswith')
            and callable_attr(host, 'endswith')
            and not host.startswith("[")
            and not host.endswith("]")
        ):
            host = "[" + host + "]"
        ...

This allows this usage:

        f.set(host="::1")
        assert f.host == "[::1]"
        assert f.url == "http://[::1]/"

        f.set(host="[::]")
        assert f.host == "[::]"
        assert f.url == "http://[::]/"

Fixes #164

This leads to a breaking change: well-formed but invalid IPv6 such as [0:0:0:0:0:0:0:1:1:1:1:1:1:1:1:9999999999999] now raise a ValueError. However one can argue that this is not exactly a behavior change but more a bugfix.

If you think the breaking change is too hash, I can do another PR that would raise a deprecation warning when a well-formed invalid IPv6 is encountered, to be merged and released before this PR is merged and released.

What do you think?

I was surprised to see that furl would allow invalid but well-formed IPV6:

https://github.com/gruns/furl/blob/774846234ff803606fdd289a7549f9b50b2b3677/tests/test_furl.py#L1658-L1661

I would suggest to use the IPV6Address object from the python3 ipaddress module to validate IPV6. For python2 I would suggest using the ipaddress backport on pypi.

If it is OK I would volunteer for a PR. What do you think?