• typical version: 2.6.3
• Python version: 3.8.5
• Operating System: CentOS 8

Description

In this version(2.6.3), it can handle Union if the given argument is just an instance of one of the types in the Union, but not support subtypes, checking type by issubclass or isinstance can sovle this. But if Union contains constrained types, the transmute function did not behave like we hope.

import typic, typing

PositiveFloat = typic.constrained(float, gt=0)
PositiveInt = typic.constrained(int, gt=0)


proto = typic.protocol(typing.Union[PositiveInt, PositiveFloat])

proto.transmute(PositiveFloat(2.1))  # 2.1
proto.transmute(2.1)  # 2
isinstance(2.1, PositiveFloat)  # False
issubclass(PositiveFloat, float)  # True

It's stated in the README:

The following annotations are special forms which cannot be resolved: Union Any Because these signal an unclear resolution, Typical will ignore this flavor of annotation ...

However, when I try to coerce some values to Unions, I didn't expect Typical to silently ignore Union annotations: (e.g. it returns a str when a str is passed in, when none of the Union members are str). Since it is ignored, I can't rely on passing my value through coercion to actually get the correct type returned to me. I think this is a major limitation and a source of errors.

from typing import Union
import typic


# Case A. Incorrect. Expected: ValueError
print(typic.coerce("foo", Union[float, int]))  # foo

from enum import Enum

class MyEnum(Enum):
  a = "foo"
  b = "bar"

# Case C1. Correct: Union with None works.
print(typic.coerce("foo", Union[MyEnum, None]))  # MyEnum.a
print(typic.coerce(None, Union[MyEnum, None]))  # None

# Case C1. Incorrect. Expected: MyEnum.a
print(typic.coerce("foo", Union[MyEnum, int]))  # foo
print(typic.coerce("foo", Union[MyEnum, bool]))  # foo

class Sentinel:
  pass

# Case B. Incorrect. Expected: MyEnum.a
print(typic.coerce("foo", Union[MyEnum, Sentinel]))  # foo

# Case C1: Correct: The value is an instance of one of the types
print(typic.coerce(1, Union[bool, int]))  # 1
print(typic.coerce(False, Union[bool, int]))  # False

# Case B. Incorrect. Expected: True (since it can be coerced to bool, but not to int)
print(typic.coerce("asdf", Union[bool, int]))  # asdf

# Case C2. Expected: TypeError, since it could be coerced to both
print(typic.coerce("1", Union[bool, int]))  # 1

Would it really be problematic to handle coercing to Union in some cases?

I can imagine scenarios where it's unambiguous, and one scenario where it is and an error could be raised:

• A. If the value cannot be coerced to any of the types: raise one of the coercion errors (e.g. the last)
• B. If the value can be coerced to only one of the types (the others may raise ValueError/TypeError): return that coerced value
• C. If the value can be coerced to multiple of the types:
  • C1: If the value is an instance of one of the union's types, then "coerce" it to that type -- coerce in this context means just perform any constraint checks that there might be (This coercion is already handled for Union[T, None])
  • C2: Otherwise: Raise a TypeError (this is the ambiguous scenario)

What do you think?

In my mind, it might look something like this:

from typing import Any, Iterable, Type
from typic import coerce

# Sentinel value
class Empty:
    pass


_empty = Empty()


def coerce_union(value: Any, union: Iterable[Type]) -> Any:
    coerced_value = _empty

    for typ in union:
        # If the value is already an instance of one of the types in the Union, coerce using that type
        # Presumably, if the type was constrained, coercing could raise errors here, but that would be expected.
        if isinstance(value, typ):
            return coerce(value, typ)
    error = None

    for typ in union:
        try:
            new_coerced_value = coerce(value, typ)
        except (ValueError, TypeError) as e:
            # Store the error and continue
            error = e
            continue
        if coerced_value is not _empty:
            raise TypeError("Ambiguous coercion: the value could be coerced to multiple types in the Union")
        coerced_value = new_coerced_value
    if coerced_value is not _empty:
        return coerced_value
    # The value couldn't be coerced to any of the types in the Union
    raise error

…e where they were referenced

This helps to allow definitions to be reused. Previously, the behavior would be that there would be 2 definitions (named First and Second) rather than one.

It's also the behavior I originally expected, as I wondered why my definitions got a different name than the name of the e.g. dataclass referencced.

However this change is backwards-incompatible. Not sure if there should be an option somewhere to configure the schema builder's behavior (it could be that you could provide callable which returns the name for an object)? If so, I'm not sure where the appropriate place, API-wise, to place such an option would be. Another option would be to make defname not a staticmethod so you could easily subclass SchemaBuilder and override defname.

There is also the issue that the name could conflict with another object (which can also happen with the previous naming behavior) from a different module with the same name. In that case, either an error should be thrown or perhaps the module name should be included and prepended to the type's __name__?

What do you think?

• typical version: 2.0.0
• Python version: 3.8.2
• Operating System: macOS Catalina

Expected that when using strict with datetime, that it only accept datetime object. And do not do any type coercer. But the following example do not behave as I expected. Expected to get an exception in both cases.

>>> import typic
>>> typic.transmute(typic.Strict[pendulum.DateTime], '2020-04-09') 
DateTime(2020, 4, 9, 0, 0, 0, tzinfo=Timezone('UTC'))

or similary

>>> @typic.klass(strict=True)
... class Test:
... date: pendulum.DateTime
>>> Test('2020-04-09') 
Test(date=DateTime(2020, 4, 9, 0, 0, 0, tzinfo=Timezone('UTC')))

• typical version: 2.4.0
• Python version: 3.10.b2
• Operating System: Linux

import typic

@typic.al
def foo(bar: str | int):
    pass

Would raise these errors:

Traceback (most recent call last):
  File "/home/lsongzhi/code/python/pydemo/main.py", line 4, in <module>
    def foo(bar: str | int):
  File "/home/lsongzhi/.pyenv/versions/3.10.0b2/lib/python3.10/site-packages/typic/api.py", line 384, in typed
    return _typed(_cls_or_callable) if _cls_or_callable is not None else _typed
  File "/home/lsongzhi/.pyenv/versions/3.10.0b2/lib/python3.10/site-packages/typic/api.py", line 378, in _typed
    return wrap(obj, delay=delay, strict=strict)  # type: ignore
  File "/home/lsongzhi/.pyenv/versions/3.10.0b2/lib/python3.10/site-packages/typic/api.py", line 172, in wrap
    protocols(func)
  File "/home/lsongzhi/.pyenv/versions/3.10.0b2/lib/python3.10/site-packages/typic/serde/resolver.py", line 782, in protocols
    resolved = self.resolve(
  File "/home/lsongzhi/.pyenv/versions/3.10.0b2/lib/python3.10/site-packages/typic/serde/resolver.py", line 709, in resolve
    resolved = self._resolve_from_annotation(anno, namespace=namespace)
  File "/home/lsongzhi/.pyenv/versions/3.10.0b2/lib/python3.10/site-packages/typic/serde/resolver.py", line 524, in _resolve_from_annotation
    deserializer, validator = self.des.factory(
  File "/home/lsongzhi/.pyenv/versions/3.10.0b2/lib/python3.10/site-packages/typic/serde/des.py", line 866, in factory
    deserializer = self._build_des(annotation, key, namespace)
  File "/home/lsongzhi/.pyenv/versions/3.10.0b2/lib/python3.10/site-packages/typic/serde/des.py", line 709, in _build_des
    anno_name = get_unique_name(origin)
  File "/home/lsongzhi/.pyenv/versions/3.10.0b2/lib/python3.10/site-packages/typic/util.py", line 255, in get_unique_name
    return f"{get_name(obj)}_{id(obj)}".replace("-", "_")
  File "/home/lsongzhi/.pyenv/versions/3.10.0b2/lib/python3.10/site-packages/typic/util.py", line 236, in get_name
    return obj.__name__
AttributeError: 'types.Union' object has no attribute '__name__'. Did you mean: '__ne__'?

• typical version: 2.0.19
• Python version: 3.7.4
• Operating System:

Description

import typic

class FooMeta(type):
    def __new__(mcs, name, bases, namespace):
        cls = super().__new__(mcs, name, bases, namespace)
        ...
        cls = typic.klass(cls, slots=True)
        ...
        return cls

class Foo(metaclass=FooMeta):   # RecursionError
    a: int

What I Did

Paste the command(s) you ran and the output.
If there was a crash, please include the traceback here.

It seems like typical doesn't convert the StringFormat.DATE enum field into a primitive (e.g. str), which fastjsonschema doesn't like.

>>> import typic
>>> from datetime import date
>>> from dataclasses import dataclass
>>> @typic.al
... @dataclass
... class Foo:
...   foo: date
...
>>> typic.schema(Foo)
ObjectSchemaField(title='Foo', description='Foo(foo: datetime.date)', properties={'foo': StrSchemaField(format=<StringFormat.DATE: 'date'>)}, additionalProperties=False, required=('foo',))
>>> typic.schema(Foo).validate({})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../venv/lib/python3.7/site-packages/typic/schema/field.py", line 194, in validate
    return self.validator(obj)
  File ".../venv/lib/python3.7/site-packages/typic/util.py", line 226, in __get__
    cache[attrname] = self.func(instance)
  File ".../venv/lib/python3.7/site-packages/typic/schema/field.py", line 189, in validator
    return fastjsonschema.compile(self.asdict())
  File ".../venv/lib/python3.7/site-packages/fastjsonschema/__init__.py", line 167, in compile
    exec(code_generator.func_code, global_state)
  File "<string>", line 5
    raise JsonSchemaException("data must be object", value=data, name="data", definition={'type': 'object', 'title': 'Foo', 'description': 'Foo(foo: datetime.date)', 'properties': {'foo': {'type': 'string', 'format': <StringFormat.DATE: 'date'>}}, 'additionalProperties': False, 'required': ['foo'], 'definitions': {}}, rule='type')
                                                                                                                                                                                                                         ^
SyntaxError: invalid syntax

Heyo! I very like your library! But when i saw the docs, i found that typical uses python dataclasses from standart library which, of course, will decrease performance because they are slow. What do you think about reinventing it for more performance? :)

I didn't expect that None would be coerced to a string for a field marked as str. Can that behavior be disabled?

>>> @typic.al
... @dataclass
... class Y:
...   a: str
... 
>>> Y(None)
Y(a='None')

The same behavior happens if you try to coerce other types:

>>> Y(dict)
Y(a="<class 'dict'>")

• typical version: 2.6.2
• Python version: 3.8.5
• Operating System: CentOS 8

Description

import typic, typing


@typic.klass
class Foo:
    a: typing.Union[float, int] = 5

        
Foo(0.3)  # Foo(a=0)

• typical version: 2.0.29
• Python version: 3.8.3
• Operating System: Linux

Description

I have a class with a custom from_dict class method that does some stuff and then calls transmute. However I note that the deserializer used in transmute creates and then uses a from_dict class method internally. This means that because I have defined my own method of the same name that in turn calls transmute, I obviously hit the recursion limit.

Simple example:

@typic.al
@dataclass
class Person:
    name: str
    age: int

    @classmethod
    def from_dict(cls, dict):
        return cls.transmute(dict)


person = Person.from_dict({"name": "Jack", "age": 10})

Gives (full stacktrace ommited due to obvious size):

  ...
  File "<typical generated deserializer__283563495926319966>", line 7, in deserializer__283563495926319966
  File "/home/arranhs/Desktop/typical/typical.py, line 20, in from_dict
    return cls.transmute(dict)
  File "<typical generated deserializer__283563495926319966>", line 3, in deserializer__283563495926319966
RecursionError: maximum recursion depth exceeded in __instancecheck__

And the generated deserializer:

def deserializer__283563495926319966(val):
    _, val = __eval(val) if isinstance(val, (str, bytes)) else (False, val)
    vtype = val.__class__
    if vtype is Person_38601264:
        return val
    val = Person_38601264.from_dict(val)
    return val

What I Did

Naturally the simple solution (and I suppose the good enough one) is to rename my from_dict method. Unfortunately for me this is a breaking change and I would prefer to avoid it.

Is there a way to get around this? The only path i could see thus far would be to fork the repo and make from_dict, _from_dict. Would this break anything? Could this feasibly and easily be done in this repo for avoid issues for others since this is a pretty common function name. If not, maybe I can help add a documentation warning as this took me quite a while to understand the root cause of.

On a side note, I also have a to_dict method, and I note you also implement this in typical with from_dict. So far I have not noticed any issues having this method and overriding yours, but maybe there is something I am missing. Could this cause issues?

Thank you so much for the work that you've done with this library. I haven't gotten to actually use it yet (so I may be missing important details, and would appreciate knowing that), and I'm extremely impressed. It is taking the right approach to design in ways that have really caused me trouble with other typing and serialization libraries, or that have caused me to just not be happy enough to want to use them.

The principles you lay out as guiding principles are spot on. By leaning into the standard tooling to provide extensibility over those tools rather than working a different parallel path, or even in opposition to them, we're better able to gradually improve our code. Bit by bit, making it better, as we learn and grow, without having to risk an entire rewrite to get the benefits.

When I think about working with Python instead of against it, which is a key feature that I'm very impressed with your work in typical for data classes and other built-in types, I find the approach of the default non-strict mode to be a very notable departure from that principle. Python doesn't do implicit type coercion, and that has been an intentional and deliberate design choice of the language for as long as I've known it, and I think probably since its inception.

The non-strict mode default turns that design principle absolutely on its head. You've provided strict mode, and that helps us work around the concern, but the philosophy is still reversed from the Pythonic first principles. If I care to preserve this principle, I'm left with a variety of (easy to use) options, where I have to constantly re-affirm that I agree with this Pythonic first principle.

The global strict mode to solve this is a non-starter for non-trivial applications, because it breaks assumptions other libraries might very reasonably be making about the state of that. I want everyone to use typical, so I want that concern to be a common thing to encounter. IMO, it is a misfeature to even offer that API, it's far too powerful of a foot gun.

All that said, easy and loose type coercion is extremely valuable. You had an eye toward this with your initial definition of @typic.al. That is a great tool, and I don't wish for you to take it out of the toolbox. However, I think it should be a different tool than the typing and serialization layer's defaults.

Personally, I find this core philosophical difference between typical as it currently exists and the way I think of the principles that have guided your design of typical to be so significant that its worth having an entirely separate API if needed that defaults to strict mode. There are a few approaches I can see to doing this, depending on what you want typical to be.

1. Do nothing, because the current design is what you intend typical to be.
2. Cut a major, breaking release of typical, fixing and reversing this default.
3. Release new APIs in this package that are reliably strict by default.
4. Create a new package in this distribution with strict by default APIs.

Gut reactions to which of these is best might be further informed by these additional considerations:

• If the current design is the right choice for typical, I can see myself considering releasing another distribution package to PyPI, that perhaps works with typical under the hood, but exposes the strict mode by default.
• A new package in this distribution could be typical, matching the distribution package name. This could leave the cute typic.al shortcut names defaulting to the non-strict mode, which may be preferable to many people, and allowing others to choose to use the strict APIs that might have more no-frills business-mode names like attrs ended up adding.
• I'm sure we can find a nice keyword for non-strict mode, less boring than "non-strict". magic=True, maybe? Magic is cool, as long as you've asked for it. friendly? autoconvert?

If you got through all that, I hope that it came through in the intended spirit of gratitude and deference. I greatly appreciate that you've released this to the world, and that I get to see it. Still, this is your project, and it is and should be what you say it is, and I respect that.

What would you like to be the future of strict mode in typical? Do you agree with me that it's critical enough to warrant one of these significant options to allow for really changing the default mode? Or is that just not what you want typical to be? Have I perhaps missed something important?

A lot of Python libraries/frameworks support the use of choices as collection of <value>/<label> tuples (or even an enum, for example http.HTTPStatus. Is there a specific way to have the labels for such a collection to the generated schema?

I'm considering using typical as a replacement for pydantic, and I already asked a similar question for pydantic here.

Thanks in advance

• typical version: 2.8.0
• Python version: 3.8.13
• Operating System: Linux - Redhat EL7

Description

Starting to use typical, and recently installed my package which also has a dependency on "rich". "rich" requires "typing-extensions" > 4 and "typical" has "typing-extensions" pinned to 3.10.0.2.

What I Did

Searched "typing-extensions" in the "typical" github repository and it isn't used in any code - so there must be a deeper reason why "typical" needs the pinned version.

Paste the command(s) you ran and the output.
If there was a crash, please include the traceback here.

Bumps numpy from 1.21.4 to 1.22.0.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Bumps django from 2.2.25 to 2.2.28.

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Current benchmark is not fair for libraries like pydantic or marshmallow because they use standard json library while typical use orjson, which is a lot faster.