EDIT: For the current status of the issue, skip to https://github.com/encode/django-rest-framework/issues/4231#issuecomment-332935943

===

I'm having an issue implementing an Optimistic Concurrency library on an application that uses DRF to interact with the database. I'm trying to:

• Confirm that the behavior I'm seeing is attributable to DRF
• Confirm that this is the intended behavior
• Determine if there's any practical way to overcome this behavior

I recently added optimistic concurrency to my Django application. To save you the Wiki lookup:

• Every model has a version field
• When an editor edits an object, they get the version of the object they're editing
• When the editor saves the object, the included version number is compared to the database
• If the versions match, the editor updated the latest document and the save goes through
• If the versions don't match, we assume a "conflicting" edit was submitted between the time the editor loaded and saved so we reject the edit
• If the version is missing, we cannot do testing and should reject the edit

I had a legacy UI talking through DRF. The legacy UI did not handle version numbers. I expected this to cause concurrency errors, but it did not. If I understand the discussion in #3648 correctly:

• DRF merges the PUT with the existing record. This causes a missing version ID to be filled with the current database ID
• Since this always provides a match, omitting this variable will always break an optimistic concurrency system that communicates across DRF
• ~There are no easy options (like making the field "required") to ensure the data is submitted every time.~ (edit: you can workaround the issue by making it required as demonstrated in this comment)

Steps to reproduce

1. Setup an Optimistic Concurrency field on a model
2. Create a new instance and update several times (to ensure you no longer have a default version number)
3. Submit an update (PUT) through DRF excluding the version ID

Expected behavior

The missing version ID should not match the database and cause a concurrency issue.

Actual behavior

The missing version ID is filled by DRF with the current ID so the concurrency check passes.

This patch is meant to fix #3111, regarding comments made to #3137 and #3169.

The ValidationError will now contain a code attribute.

Before this patch, ValidationError.detail only contained a dict with values equal to a list of string error messages or directly a list containing string error messages.

Now, the string error messages are replaced with ValidationError.

This means that, depending on the case, you will not only get a string back but a full object containing both the error message and the error code, respectively ValidationError.detail and ValidationError.code.

It is important to note that the code attribute is not relevant when the ValidationError represents a combination of errors and hence is None in such cases.

The main benefit of this change is that the error message and error code are now accessible in the custom exception handler and can be used to format the error response.

A custom exception handler example is available in the TestValidationErrorWithCode test.

Regarding the other discussion:

• This PR doesn't require any new settings
• The build_error method was dropped
• The build_error_from_django_validation_error has been kept because this is the only way to "convert" a regular DjangoValidationError to a djrf ValidationError.

@tomchristie @jpadilla @qsorix: if you don't mind giving this a look ;-)

Here _closable_objects was added to rendering_attrs https://github.com/tomchristie/django-rest-framework/commit/59d0a0387d907260ef8f91bbbca618831abd75a3#diff-21a03cd19b8422c334c33ec8da66e9c8R21

Every rendering attribute is deleted in __getstate__ method https://github.com/django/django/blob/1.7/django/template/response.py#L45

But _closable_objects is used in base response handler: https://github.com/django/django/blob/1.7/django/core/handlers/base.py#L210

That's why after processing response from cache i get error like this:

Traceback (most recent call last):
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 85, in run
    self.result = application(self.environ, self.start_response)
  File "/Users/web-chib/drf-ussie/lib/python2.7/site-packages/django/contrib/staticfiles/handlers.py", line 64, in __call__
    return self.application(environ, start_response)
  File "/Users/web-chib/drf-ussie/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 187, in __call__
    response = self.get_response(request)
  File "/Users/web-chib/drf-ussie/lib/python2.7/site-packages/django/core/handlers/base.py", line 211, in get_response
    response._closable_objects.append(request)
AttributeError: 'Response' object has no attribute '_closable_objects'

I don't know, should i recreate this attribute by hand, or that's DRF error?

This is a useful feature, at least for swagger, and the way things are right now it cannot be implemented from outside of DRF, since the Link object doesn't contain the schema for the response.

While the fields can be implied from the input fields, for read only endpoints this is not an option.

Django==1.9.2 djangorestframework==3.3.2

This is essentially a repost of #3814. There is already some research and discussion attached to that issue that I will not rehash, but the author closed out the item once a workaround was found.

Is that workaround the long term solution? Is the answer to never touch the Django Request before DRF has a chance to do its thing? This can be difficult to control in projects with third party middleware.

The issue seems to be that, when POSTing data, Django is exhausting the underlying request stream if the WSGIRequest is accessed prior to initialize_request in the DRF View dispatcher. PUTs seem to be unaffected.

A totally contrived example that demonstrates the issue. First example, presuming no middleware Request meddling...

class TestView(APIView):

    def post(self, request, *args, **kwargs):
        # request.data, .FILES, and .POST are populated
        # request._request.FILES and .POST are empty
        return Response()

    def put(self, request, *args, **kwargs):
        # request.data, .FILES, and .POST are populated
        # request._request.FILES and .POST are empty
        return Response()

Second example, forcibly evaluating the Django request before dispatching.

class TestView(APIView):

    def dispatch(self, request, *args, **kwargs):
        p = request.POST  # Force evaluation of the Django request
        return super().dispatch(request, *args, **kwargs)

    def post(self, request, *args, **kwargs):
        # request.data, .FILES, and .POST are empty
        # request._request.FILES and .POST are populated
        return Response()

    def put(self, request, *args, **kwargs):
        # request.data, .FILES, and .POST are populated
        # request._request.FILES and .POST are empty
        return Response()

Cheers!

Add support for creating HTML forms for the browsable API for "extra actions" for routing.

For example using the example from the linked page:

    @detail_route(methods=['post'])
    def set_password(self, request, pk=None):

creating a form for calling set_password. Likely this would require additional arguments passed to the detail_route decorator (such as the serializer used, PasswordSerializer).

See: https://groups.google.com/forum/#!topic/django-rest-framework/e75osh1Wcyg

I wrote a PoC NestedSimpleRouter, that nests its URLs into some other router routes. Usage:

# urls.py
(...)
router = routers.SimpleRouter()
router.register(r'carts', CartViewSet)

carts_router = routers.NestedSimpleRouter(router, r'carts')
carts_router.register(r'itens', ItensViewSet)

url_patterns = patterns('',
    url(r'^', include(router.urls)),
    url(r'^', include(carts_router.urls)),
)

This creates the following URLs:

^carts/$ [name='cart-list']
^carts/(?P<pk>[^/]+)/$ [name='cart-detail']
^carts/(?P<nested_1_pk>[^/]+)/itens/$ [name='item-list']
^carts/(?P<nested_1_pk>[^/]+)/itens/(?P<pk>[^/]+)/$ [name='item-detail']

And is intended to be used like:

# views.py
class ItemViewSet(viewsets.ViewSet):
    model = Item

    def list(self, request, nested_1_pk=None):
        cart = Cart.objects.get(pk=nested_1_pk)
        return Response([])

    def retrieve(self, request, pk=None, nested_1_pk=None):
        (...)

The nested_1_pk is automatic, and should grow to nested_{n}_{lookup_field} as you nest Nested routers, but this will need more work into SimpleRouter.get_lookup_regex() to do it ( and I need some sleep now :P )

It works very well, but have no tests yet.

What you think?

If you use a viewset with a queryset using prefetch_related, any update to those prefetched many to many fields is not reflected in the response to the client, although the changes have been saved to the database.

See below code for very simple example:

from django.conf.urls import patterns, include, url
from django.contrib import admin
from django.contrib.auth.models import User
from rest_framework import serializers, viewsets, routers


class UserSerializer(serializers.ModelSerializer):
    class Meta:
        model = User
        fields = ('id', 'username', 'email', 'groups')


class UserViewSet(viewsets.ModelViewSet):
    queryset = User.objects.all().prefetch_related('groups')
    serializer_class = UserSerializer


router = routers.DefaultRouter()
router.register(r'users', UserViewSet)

urlpatterns = patterns('',
    url(r'^', include(router.urls)),
    url(r'^api-auth/', include('rest_framework.urls',
                               namespace='rest_framework')),
    url(r'^admin/', include(admin.site.urls)),
)

With the above code, assume there is an existing user with 2 groups assigned:

{
    "id": 1, 
    "username": "chris", 
    "email": "[email protected]", 
    "groups": [
        1,
        2
    ]
}

Submitting a PUT to /users/1/ changing groups to [1], the response will still show groups as [1, 2], but if you then did a GET request for /users/1/ you would see the reflected changes.

You can see this behaviour using the browseable api also.

Removing prefetch_related from the queryset restores normal behaviour and edits are again reflected in the response.

This example is very trivial, but if people are using prefetch_related to try and optimise their database queries they may encounter this issue.

Given Django 3.1's upcoming support for async views, it's worth us talking though if there are useful points of async support in REST framework, and what they would be if so.

I'm going to prefix this by starting with a bit of expectation setting... Django 3.1's async view support is a really impressive bit of foundational work, but there's currently limitations to where it's actually valuable, given that the ORM isn't yet async-capable.

One thing that'd be really helpful to this discussion would be concrete use-cases where folks demonstrate an actual use-case where they've used or would use an async view in Django, together with the motivation, and demonstrable improvements vs. sticking with a regular sync view.

We'd also want to scoping this down to the most minimal possible starting point.

In particular, what would we need to change in order to support this?...

@api_view(['GET'])
async def my_view(request):
    ...

The @api_view decorator is implemented on top of the APIView class based view, so an even more minimal question is: what would we need to change in order to support something like this?...

class MyView(AsyncAPIView):
    async def get(self, request):
        ...

There's a whole bunch of different things to consider there, eg...

• The authentication/permissions is likely to be a sync ORM operation at the moment.
• The throttling is likely to be a sync cache operation at the moment.
• Does Django support reading the request body as an async operation? How would we need to tie this in with the parsing.

But let's just put those aside for the moment.

Django's upcoming docs for 3.1 async views mentions...

For a class-based view, this means making its __call__() method an async def (not its __init__() or as_view()).

So here's some even simpler questions:

• What does a Django 3.1 async CBV look like?
• What would using REST framework's Request explicitly within a Django async view look like (rather than wrapping with @api_view). What provisos are there on operations on the request instance that are currently sync?
• What would using REST framework's Response explicitly within a Django async view look like (rather than wrapping with @api_view). Are there any provisos on sync operations there?

I have TIME_ZONE = 'Asia/Kolkata' and USE_TZ = True in my settings.

When I create a new object with the browsable api, the serializer displays the newly created object with datetimes that have a trailing +5:30, indicating the timezone. The database is storing the times in UTC.

The unexpected behavior is that when the object is serialized again later, the datetimes are all in UTC format with a trailing Z. According to the Django docs on current and default time zone, I would expect the serializer to convert the datetimes to the current time zone, which defaults to the default time zone, which is set by TIME_ZONE = 'Asia/Kolkata'.

Am I missing something?

Implementation for #560

• ListField, ListOrObjectField, DictField compound field types
• Also, for BaseSerializer.from_native, give files parameter default of None

Description

Allow Request, Response, Field, and GenericAPIView to be subscriptable. This is a follow up to encode/django-rest-framework#7385 and allows the classes to be made generic for type checking.

By making these classes generic, type checkers can more easily do their job when working with DRF classes. Today, django-stubs and djangorestframework-stubs must monkey patch types to support type checking, but the approach is problematic with DRF as seen in this issue.

I have enabled AcceptHeaderVersioning as the DEFAULT_VERSIONING_CLASS. I have added a custom middleware(added at the last of the MIDDLEWARE list) which looks at request.version value and take actions based on the custom logic. However, I see that request.version is not populated in the middleware and returns the error.

  File "/Users/test/opt/miniconda3/envs/test/lib/python3.9/site-packages/django/core/handlers/base.py", line 171, in _get_response
    response = middleware_method(request, callback, callback_args, callback_kwargs)
  File "/Users/test/work/test1/versioncheck/middleware.py", line 35, in process_view
    print(request.version)
AttributeError: 'WSGIRequest' object has no attribute 'version'

Checklist

• [ ] Raised initially as discussion #...
• [x] This cannot be dealt with as a third party library. (We prefer new functionality to be in the form of third party libraries where possible.)
• [x] I have reduced the issue to the simplest possible case.

The documentation used to state that the detail argument was mandatory, but in fact it currently is not.

Here is the line that defines the default argument: https://github.com/encode/django-rest-framework/blob/master/rest_framework/exceptions.py#L148

Checklist

• [x] Raised initially as discussion #...
• [x] This cannot be dealt with as a third party library. (We prefer new functionality to be in the form of third party libraries where possible.)
• [x] I have reduced the issue to the simplest possible case.

• Add docs/community/3.15-announcement.md
• Update docs/community/release-notes.md

As mentioned in https://github.com/encode/django-rest-framework/pull/8614#issuecomment-1338997904

Updates ModelSerializer’s field generation to set pk_field=UUIDField() if the related field is PrimaryKeyRelatedField and the foreign key field on the model is a UUIDField.

See this discussion for an explanation of why I think this change should be made

Compatibility

This could potentially break existing code if that code was relying on getting a UUID instance in serializer.data. This doesn’t apply once the data has been passed to the renderer, since AFAIK all built-in renderers will stringify UUIDs.

Alternatives

A potential alternative to this implementation would be for the behavior to be moved into PrimaryKeyRelatedField itself. In that case, the pk_field setting would apply even when used outside of ModelSerializer (unless overridden).