Hi,

i don't know if is the right place to ask, because it's not an issue, just a question.

I'm using django-filter with Django-REST-Framework to build APIs for my app (great job by the way to both of you!)

I would like to know if there's a way to get this filter working

http://example.com/api/products?category=clothing,shoes

to get all products with category "clothing" OR category "shoes".

I've tried with

http://example.com/api/products?category=clothing&category=shoes

but as expected it does an AND between the filters and retrieve only the shoes products (i suppose because it's the last filter).

When filtering on a foreign key relationship, if a non-existent foreign key is specifed, the qs method in BaseFilterSet receives a ValidationError on line 259.

https://github.com/alex/django-filter/blob/master/django_filters/filterset.py#L259

This results in the queryset not getting filtered. Shouldn't one expect this case to return 0 results? The reasoning is that if no object matches the foreign key specified, then nothing in the queryset could match, so perhaps return an empty list? Maybe as simple as the following:

@property
def qs(self):
    if not hasattr(self, '_qs'):
        qs = self.queryset.all()
        for name, filter_ in self.filters.iteritems():
            if self.is_bound:
                data = self.form[name].data
            else:
                data = self.form.initial.get(name, self.form[name].field.initial)
            try:
                val = self.form.fields[name].clean(data)
            except forms.ValidationError:
                qs = []
            else:
                qs = filter_.filter(qs, val)
        if self._meta.order_by:
            try:
                value = self.form.fields[self.order_by_field].clean(self.form[self.order_by_field].data)
                if value:
                    qs = qs.order_by(value)
            except forms.ValidationError:
                pass
        self._qs = qs
    return self._qs

Django doesn't let me pass two query sets for filtering.

In my app, join wasn't working so my final result is generated using two query sets. How can I use two query sets?

I supplied this third option so that Django Rest Framework could return a 400 w/ an appropriate error message.

If this is to your liking, Alex, I'd be glad to add some unit tests before you merge.

Note the bizarre constants True, False, "RAISE" -- I did this to support backward compatibility. If you recommend a cleaner way to manage the strict options, let me know. Someone here recommended another boolean option, but then there'd be 4 possible values of the two booleans, only three of which make sense. I think this is more robust.

Based on discussion from https://github.com/alex/django-filter/issues/137

Using some of @kmwenja's work from his comment, and some from the original gist I had.

I considered using similar terms to Django for cleaning/validating the values, but I left it with the function names used in the comment where the cleaning was suggested.

There is an issue at get_schema_fields which prevents nested views to be processed. The problem is that url named argument contains a data that is consumed by queryset is always None since you manually setup a view.

Example

    # some view
    def get_project(self):
       return Project.objects.get(id=self.kwargs.get('project_id'))

    def get_queryset(self):
        project = self.get_project()
        return self.queryset.filter(project=project)

Root of a problem lays here:

# django_filters.rest_framework.backends.DjangoFilterBackend#get_schema_fields
filter_class = self.get_filter_class(view, view.get_queryset()) # <---- :(

As related in the issue #537, the Django ORM generate duplicated INNER JOINS for each .filter() .

For example: Model.objecs.filter(table1__attr1=value1).filter(table1__attr2=value2)

The built query is:

SELECT *
FROM model_table
INNER JOIN table1 ON (model_table.id = table1.model_table_id)
INNER JOIN table1 T5 ON (model_table.id = T5.model_table_id)
WHERE (table1.attr1=value1 AND T5.attr2=value2)

But the correct query should be:

SELECT *
FROM model_table
INNER JOIN table1 ON (model_table.id = table1.model_table_id)
WHERE (table1.attr1=value1 AND table1.attr2=value2)

The first query may return unwanted or unexpected results.

Reading the code I found the method def qs(self) in class BaseFilterSet from the file filterset.py and I realized the following workflow:

1. Get all the model objects using qs = self.queryset.all()
2. Iterates in the list of declared filters (fields)
3. In other words, for each field is done the same as qs = self.queryset.filter(field_n=value_n), what causes the described problem.
4. When the iteration finishes, the queryset (qs) is returned

The solution that I've developed for the chained filters problem is:

    def qs(self):
        kwargs = {}
        if not hasattr(self, '_qs'):
            if not self.is_bound:
                self._qs = self.queryset.all()
                return self._qs

            if not self.form.is_valid():
                if self.strict == STRICTNESS.RAISE_VALIDATION_ERROR:
                    raise forms.ValidationError(self.form.errors)
                elif self.strict == STRICTNESS.RETURN_NO_RESULTS:
                    self._qs = self.queryset.none()
                    return self._qs
                # else STRICTNESS.IGNORE...  ignoring

            # start with all the results and filter from there
            qs = self.queryset.all()
            for name, filter_ in six.iteritems(self.filters):
                value = self.form.cleaned_data.get(name)

                if value is not None and value:  # valid & clean data
                    # If all o these conditions are true, it means that
                    # field (filter_) is filtering the queryset
                    # So, it must be added to the filtering dict
                    kwargs[filter_.name] = value

            # The same as qs.filter(filter_.name1=value1, filter_.name2=value2, ...)
            self._qs = qs.filter(**kwargs)

        return self._qs

Instead of making queries for each field, all of them are stored in a dictionary and the queryset is filtered only one time using this dict.

What do you think about this solution? Would it break other parts of the code?

This is an evolution of #783. In short, I think the validation/strictness behavior can be improved by moving it out of the FilterSet and into the view code. The current .qs implementation is somewhat difficult to extend, and is a little confusing given that it performs both form validation and queryset filtering.

Overall changes:

• Expands the FilterSet API to include the following:
  • .is_valid() - proxies form.is_valid()
  • .errors - proxies form.errors
  • .filter_queryset(qs) - allows users to easily override queryset handling. Cached by .qs
  • .get_form_class() - easier overriding of the form class on a per-instance basis, cached by .form
• FilterSet-level strictness handling is removed in favor of shifting it to the view layer
  • Generic View has a strict/non-strict attribute (previously, RETURN_NO_RESULTS/IGNORE)
  • DjangoFilterBackend has a raise/non-raise attribute (previously, RAISE_VALIDATION_ERROR/IGNORE)
• Reworked raw_validation. Error details now include codes. However, these codes are only reachable via exc.get_full_details(), and do not automatically show up in the rendered error response.
• Other small testing/docs changes

Additional thoughts:

• #136 is definitely targeting API use. Raising a forms.ValidationError in the context of a Django view does not make any sense, especially given that none of the provided view code actually handles this exception. Raising an exception has been moved to the backend, where it uses the same utility code to raise the filterset.errors.
• The 'no results'/'ignore' handling is capable of being handled at the view layer, dependent on a FilterMixin.strict option.
• Given that views are easily customizable, the FILTERS_STRICTNESS setting is no longer necessary.
• This slightly changes the Meta.together handling by
  • Allowing multiple 'together' set errors to be displayed (instead of just one).
  • Removes the fields from the form.cleaned_data

TODO:

• [x] ~~Determine how much deprecation warning needs to occur (if at all)~~ no deprecation for strictness
• [x] Add tests for FilterBackend/FilterMixin validation behavior
• [x] Deprecate strictness in docs (remove relevant docs, add migration)
• [x] ~~Document new public FilterSet API~~ postponing to docs overhaul

Additionally, this resolves #740 by extending #738. Inlining the original PR comment:

I was thinking about #740 a while back, and realized that overriding the FilterSet.qs property is slightly awkward, due to the _qs caching. In the case of #740, Prefetch calls can only occur once, as duplicate calls can raise an exception. If you want to cache the prefetch, a correct solution looks like the following:

@property
def qs(self):
    # ensure that we haven't already cached, as this also implies prefetching
    already_prefetched = hasattr(self, '_qs')
    super().qs
    if self.is_bound and not already_prefetched:
        self._qs = self._qs.prefetch_related(...)
    return self._qs

Overriding filter_queyset() is a little more intuitive, as it would only be executed once.

def filter_queryset(self, queryset):
    qs = super().filter_queryset(queryset)
    return qs.prefetch_related(...)

This allows to use an annotated field easily, without overriding filter altogether:

class MyMultipleChoiceFilter(django_filters.ModelMultipleChoiceFilter):
    def get_filter_predicate(self, v):
        return {'annotated_field': v.annotated_field}

foo = MyMultipleChoiceFilter(
    to_field_name='annotated_field',
    queryset=Model.objects.qs_with_annotated_field(),
)

They both work well together, it took me a while to fiddle it out. Based on the example in the docs:

{% block content %}
    <form action="" method="get">
        {{ f.form.as_p }}
        <input type="submit" />
    </form>

    {% autopaginate f.qs 40 as filter_list %}

    {% for obj in filter_list %}
        {{ obj.name }} - ${{ obj.price }}<br />
    {% endfor %}

    {% paginate %}
{% endblock %}

The key is that you have to use pagination's as argument.

I have the a view, AccountList, which is trying to render a django_table2 table. The view's source code:

class AccountList(SingleTableMixin, FilterView):
    model = Account
    table_class = AccountTable
    template_name = 'accounts/account_list.html'
    context_table_name = 'object_list'
    ordering = ['vps']

    filterset_class = AccountFilter

This view is currently using this filterset (from django_filters):

import django_filters
from accounts.models import Account

class AccountFilter(django_filters.FilterSet):
    class Meta:
        model = Account
        fields = ['is_suspended', 'is_abandoned']

    is_suspended = django_filters.BooleanFilter(name='is_suspended', initial='False')
    is_abandoned = django_filters.BooleanFilter(name='is_abandoned', initial='False')

    def __init__(self, data=None, *args, **kwargs):
        # if filterset is bound, use initial values as defaults
        if data is not None:
            # get a mutable copy of the QueryDict
            data = data.copy()

            for name, f in self.base_filters.items():
                initial = f.extra.get('initial')

                # filter param is either missing or empty, use initial as default
                if not data.get(name) and initial:
                    data[name] = initial

        super().__init__(data, *args, **kwargs)

Using this template:

{% if filter %}
    <form action="" method="get" class="form form-inline">
        {{ filter.form.as_p }}
        <input type="submit" />
    </form>
{% endif %}

{% render_table object_list %}

{% endblock %}

This is my from my urls.py

path('', login_required(AccountList.as_view())),

When I visit my page, 127.0.0.1:8000, I see that the filters are not set: []1

But then if i do 127.0.0.1:8000?page=1, I see the filters are initialized properly:

[]2

What is causing my filters to not have default value when I don't have page=1 appended to my url?

FilterSet in django_filters.rest_framework.filterset.py defines the template pack to use. It looks like bootstrap3 is hardcoded here:

        if compat.is_crispy():
            from crispy_forms.helper import FormHelper
            from crispy_forms.layout import Layout, Submit

            layout_components = list(form.fields.keys()) + [
                Submit('', _('Submit'), css_class='btn-default'),
            ]
            helper = FormHelper()
            helper.form_method = 'GET'
            helper.template_pack = 'bootstrap3'
            helper.layout = Layout(*layout_components)

This breaks crispy_bootstrap5. Would it not be nicer to do?:

from django.conf import settings

        if hasattr(settings, 'CRISPY_TEMPLATE_PACK'):
            helper.template_pack = settings.CRISPY_TEMPLATE_PACK
        else:
            helper.template_pack = 'bootstrap3'

Or is there an other place to set the template pack?

Hey I was using an old version of django-filter and I added some non-model fields for filtering. for example for user I added a name filter with custom method which concats first_name and last_name and filter both of them but there is no name field in the User model. After I updated to the latest version I faced this error:

TypeError: 'Meta.fields' must not contain non-model field names: name

I checked the codes and I found out this error happen here. in the comment it say warn if the field doesn't exist. but it actually raise a TypeError in line 349

I think it's better to make it a REAL warning instead a ERROR

Ref #1525

Just adding a PR so you can see it failing in CI for convenience.

Add test to show memoization of form fields may "break" ModelChoiceFilter's queryset kwarg

Given the following filterset (details omitted for brevity):

def get_bar(request):
    return Bar.objects.filter(…) if request else Bar.objects.none()
    
class FooFilterSet(FilterSet):
    bar = ModelChoiceFilter(queryset=get_bar)

If you happen to access base_filters at module level and (or anywhere without a request) like so:

FooFilterSet.base_filters['bar'].field

Then the bar queryset is evaluated and returns an empty queryset. Subsequently when you process the filtering during an actual request the filtered queryset never gets attached to the form field.