Using json.dumps() gets slightly complicated when you try to serialize more complex objects, like datetime.datetime or uuid.UUID objects, as simplejson and the built-in json module do not know how to serialize these objects:
import simplejson as json
import uuid
from datetime import datetime
json.dumps({'now': datetime.now()})
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# File ".../simplejson/__init__.py", line 382, in dumps
# return _default_encoder.encode(obj)
# File ".../simplejson/encoder.py", line 385, in encode
# chunks = list(chunks)
# File ".../simplejson/encoder.py", line 770, in _iterencode
# for chunk in _iterencode_dict(o, _current_indent_level):
# File ".../simplejson/encoder.py", line 727, in _iterencode_dict
# for chunk in chunks:
# File ".../simplejson/encoder.py", line 790, in _iterencode
# o = _default(o)
# File ".../simplejson/encoder.py", line 88, in _method
# return method.__get__(instance, owner)(*args, **kwargs)
# File ".../simplejson/encoder.py", line 360, in default
# o.__class__.__name__)
# TypeError: Object of type datetime is not JSON serializable
json.dumps({'uuid': uuid.uuid4()})
# Traceback (most recent call last):
# File "<stdin>", line 1, in <module>
# File ".../simplejson/__init__.py", line 382, in dumps
# return _default_encoder.encode(obj)
# File ".../simplejson/encoder.py", line 385, in encode
# chunks = list(chunks)
# File ".../simplejson/encoder.py", line 770, in _iterencode
# for chunk in _iterencode_dict(o, _current_indent_level):
# File ".../simplejson/encoder.py", line 727, in _iterencode_dict
# for chunk in chunks:
# File ".../simplejson/encoder.py", line 790, in _iterencode
# o = _default(o)
# File ".../simplejson/encoder.py", line 88, in _method
# return method.__get__(instance, owner)(*args, **kwargs)
# File ".../simplejson/encoder.py", line 360, in default
# o.__class__.__name__)
# TypeError: Object of type UUID is not JSON serializable
There are two workarounds for this:
-
Subclass simplejson.encoder.JSONEncoder and override its default method, then explicitly call JSONEncoder.encode() everywhere (this is exactly what Django does).
-
Define a function that accepts a single argument and serializes it appropriately, then call json.dumps() with the default argument everywhere:
def my_encoder(obj):
if isinstance(obj, datetime):
return obj.isoformat()
elif isinstance(obj, uuid.UUID):
return str(obj)
else:
raise TypeError("Cannot serialize object of type '%s'" % (type(obj)))
json.dumps({'now': datetime.now(), 'uuid': uuid.uuid4()}, default=my_encoder)
# '{"now": "2018-01-21T12:34:42.993445", "uuid": "5f9a8bbe-d3b2-4a32-8bc0-83ac87cb195c"}'
Both of these workarounds require users to modify/override the custom serializer function rather than extending it, and then to use the modified code everywhere instead of directly using the built-in json module or simplejson. This generally means frameworks like Django force all users to use their own serializers with their own serializer functions. It's also rather confusing for new Django users, who ask questions like this on Stack Overflow "How do I encode a UUID to make it JSON serializable?".
Furthermore, for a non-trivial project, the custom default function can become unwieldy and begins to look exactly like the usecase for singledispatch.
This PR simply uses a modified singledispatch (called singledispatchmethod) decorator to extend the default function for specific object types. This makes the simplejson.encoder.JSONEncoder.default method better adhere to the Open/Closed Principle - open to extension but closed to modification (or override). This decorator should also be completely backwards compatible - subclasses of JSONEncoder can still override it, and calls to simplejson.dumps() with a default= argument will still use the given custom encoder function.
This change allows frameworks to define and register those functions anywhere that is automatically imported (like, for instance, django/__init__.py):
import simplejson as json
import uuid
from datetime import datetime
@json.JSONEncoder.default.register(datetime)
def jsonify_datetime(encoder, dt):
return dt.isoformat()
@json.JSONEncoder.default.register(uuid.UUID)
def jsonify_uuid(encoder, uuid_):
return str(uuid_)
That's it - now every user of Django can simply call simplejson.dumps() directly and it will be able to properly serialize datetimes and UUIDs:
import simplejson as json
import uuid
from datetime import datetime
json.dumps({'now': datetime.now(), 'uuid': uuid.uuid4()})
# '{"now": "2018-01-22T03:00:49.847345", "uuid": "629a7f1e-3eb6-4bf3-a6cf-6717910e4a92"}'
Now, I realize that Django is a huge framework, and it has DRF, which implements its own framework for registering serializers. But I think this PR has the potential to greatly simplify framework code.
The singledispatch decorator does not work on instance methods, but there is a PR to fix this. I have included the two implementations (one naïve, one that is more complex but handles corner cases well) in this PR FOR REFERENCE ONLY. I will be removing them and replacing them the proper imports from functools once PR python/cpython#4987 or something like it is merged.
This PR should also fix or obviate the needs to fix the following issues and PRs:
I'm happy to respond to any criticisms or answer any questions. I would eventually like to push this API into Python's built-in json module, but it's written in C and I wanted to get feedback on this Python implementation before I do that.
Thanks!
delete the simplejson/tests directory could reduce 28kb.
1.7k Dec 29, 2022
13 Nov 10, 2022
6.4k Jan 02, 2023
1.1k Jan 02, 2023
57.6k Jan 03, 2023
1.8k Jan 07, 2023
562 Jan 08, 2023
3.9k Jan 02, 2023
19.6k Jan 01, 2023
223 Dec 11, 2022
1.3k Jan 05, 2023
5 May 14, 2021
329 Dec 28, 2022
4.1k Dec 30, 2022
469 Jan 04, 2023