A django integration for huey task queue that supports multi queue management

Related tags

Djangodjango-huey
Overview

Version

django-huey

This package is an extension of huey contrib djhuey package that allows users to manage multiple queues.

Installation

Using pip package manager run:

# pip install Django  if not installed
# pip install huey    if not installed
pip install django-huey

Note: use a virtualenv to isolate your dependencies. Note 2: django and huey must be installed.

Then, in your settings.py file add django_huey to the INSTALLED_APPS:

INSTALLED_APPS = [
	...
    'django_huey',
]

Configuration

In settings.py you must add the HUEYS setting:

HUEYS = {
    'first': {#this name will be used in decorators below
        'huey_class': 'huey.RedisHuey',  
        'name': 'first_tasks',  
        'consumer': {
            'workers': 2,
            'worker_type': 'thread',
        },
    },
    'emails': {#this name will be used in decorators below
        'huey_class': 'huey.RedisHuey',  
        'name': 'emails_tasks',  
        'consumer': {
            'workers': 5,
            'worker_type': 'thread',
        },
    }
}

Note: This setting is incompatible with HUEY setting.

Usage

Now you will be able to run multiple queues using:

python manage.py run_djangohuey --queue first
python manage.py run_djangohuey --queue emails

Each queue must be run in a different terminal.

Configuring tasks

You can use usual huey decorators to register tasks, but they must be imported from django_huey as shown below:

from django_huey import db_task, task

@db_task(queue='first')
	# perform some db task

@task(queue='emails')
	# send some emails

All the args and kwargs defined in huey decorators should work in the same way, if not, let us know.

You might also like...
Bootstrap 3 integration with Django.

django-bootstrap3 Bootstrap 3 integration for Django. Goal The goal of this project is to seamlessly blend Django and Bootstrap 3. Want to use Bootstr

Bootstrap 4 integration with Django.

django-bootstrap 4 Bootstrap 4 integration for Django. Goal The goal of this project is to seamlessly blend Django and Bootstrap 4. Requirements Pytho

Plug and play continuous integration with django and jenkins

django-jenkins Plug and play continuous integration with Django and Jenkins Installation From PyPI: $ pip install django-jenkins Or by downloading th

Django admin CKEditor integration.

Django CKEditor NOTICE: django-ckeditor 5 has backward incompatible code moves against 4.5.1. File upload support has been moved to ckeditor_uploader.

TinyMCE integration for Django

django-tinymce django-tinymce is a Django application that contains a widget to render a form field as a TinyMCE editor. Quickstart Install django-tin

Bootstrap 3 integration with Django.

django-bootstrap3 Bootstrap 3 integration for Django. Goal The goal of this project is to seamlessly blend Django and Bootstrap 3. Want to use Bootstr

Django + Next.js integration

Django Next.js Django + Next.js integration From a comment on StackOverflow: Run 2 ports on the same server. One for django (public facing) and one fo

This is a repository for collecting global custom management extensions for the Django Framework.

Django Extensions Django Extensions is a collection of custom extensions for the Django Framework. Getting Started The easiest way to figure out what

File and Image Management Application for django
File and Image Management Application for django

Django Filer django Filer is a file management application for django that makes handling of files and images a breeze. Contributing This is a an open

Comments
  • HueyException <task> not found in TaskRegistry

    HueyException not found in TaskRegistry

    When using django-huey, the consumer is able to correctly find the registered tasks, however when running the Django server and trying to enqueue a task, I get an "HueyException not found in TaskRegistry" error. From doing wayy too much debugging, I finally tracked this down to the fact that the huey package initializes itself with RegistryA, and then django_huey initializes itself with a separate RegistryB. So when trying to enqueue a task, it attempts to find the task in RegistryA which is empty, because all of the tasks are in RegistryB.

    I use import django_huey as huey everywhere, and then use @huey.task(queue="test"), but I'm still getting this error. I'm not sure what I'm doing wrong 😞

    opened by wgordon17 7
  • django-huey-monitor not working with djangohuey

    django-huey-monitor not working with djangohuey

    I use djangohuey --queue [name of queue] to run consumer, this runs my task, but the django-huey-monitor does not track the tasks, or are not created in the huey_monitor database tables. Which configuration am I missing?

    opened by EvelynArinaitwe 2
  • Unable to use `django-huey` with `SqliteHuey`

    Unable to use `django-huey` with `SqliteHuey`

    Settings:

    DJANGO_HUEY = {
        'default': 'xxx',
        'queues': {
            'xxx': {
                'huey_class': 'huey.SqliteHuey',
                'name': 'xxx_tasks',
                'consumer': {
                    'workers': 4,
                    'worker_type': 'thread',
                },
            },
            'yyy': { 
                'huey_class': 'huey.SqliteHuey',
                'name': 'yyy_tasks',
                'consumer': {
                    'workers': 2,
                    'worker_type': 'thread',
                },
            }
        }
    }
    

    Exception raised:

    huey.exceptions.ConfigurationError: "redis" python module not found, cannot use Redis storage backend. Run "pip install redis" to install.
    

    redis should be optional.

    opened by fabiocaccamo 2
  • Cannot configure djangohuey with different workers.

    Cannot configure djangohuey with different workers.

    Is there a way to configure the djangohuey settings, so that a specific worker is assigned to a specific queue? At the moment, I implement different queues with different number of workers in each consumer, but anytime a worker is free, it is taken on by any of the queue tasks that are waiting. what I want is for example: worker1 listen to only queue_a and worker2&worker3 listen to queue_b, regardless of which worker is free. Is this possible? Here are my settings:

    DJANGO_HUEY = {
        'default': 'first', #this name must match with any of the queues defined below.
        'queues': {
            'queue_a': {
                'huey_class': 'huey.RedisHuey',
                'name': 'tasks_a',
                "immediate": False,
                'results': True,
                'blocking': True,
                'connection': {
                        'host': 'localhost',
                        'port': 6379,
                        'db': 0,
                        'connection_pool': None,
                        'read_timeout': 1,
                        'url': None,  
                    },
                'consumer': {
                    'workers': 1,
                    'worker_type': 'thread',
                },
            },
            'queue_b': {
                'huey_class': 'huey.RedisHuey',
                'name': 'tasks_b',
                "immediate": False,
                'blocking': True,
                
                'consumer': {
                    'workers': 2,
                    'worker_type': 'thread',
                },
            },
        }
    }
    
    opened by EvelynArinaitwe 1
Releases(v1.1.1)
  • v1.1.1(Feb 7, 2022)

  • v1.1.0(Jan 19, 2022)

  • v1.0.1(Jan 14, 2022)

    1.0.1 - 2022-01-14

    Added

    • Close db connections before task body. https://github.com/coleifer/huey/commit/e77acf307bfdade914ab7f91c65dbbc183af5d8f
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(May 19, 2021)

    1.0.0 - 2021-05-19

    Note: This release contains breaking changes, see them below with the migration instructions.

    Added

    • Allow definition of a default queue.

    Changed

    • HUEYS django setting renamed to DJANGO_HUEY.
    • Change command run_djangohuey to djangohuey.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Apr 18, 2021)

    0.2.0 - 2021-04-18

    Added

    Nothing added this release

    Changed

    Nothing changed this release

    Fixed

    • When a huey name was not provided, default django db name was used. Now it's defaulted to queue name.

    Removed

    • Removed incompatibility with HUEY setting used by huey project.
    Source code(tar.gz)
    Source code(zip)
  • v0.1.1(Apr 4, 2021)

Owner
GAIA Software
Cooperativa de Desarrollo de Software | Quilmes, Buenos Aires, Argentina
GAIA Software
Set the draft security HTTP header Permissions-Policy (previously Feature-Policy) on your Django app.

django-permissions-policy Set the draft security HTTP header Permissions-Policy (previously Feature-Policy) on your Django app. Requirements Python 3.

Adam Johnson 78 Jan 02, 2023
Duckiter will Automatically dockerize your Django projects.

Duckiter Duckiter will Automatically dockerize your Django projects. Requirements : - python version : python version 3.6 or upper version - OS :

soroush safari 23 Sep 16, 2021
Bleach is an allowed-list-based HTML sanitizing library that escapes or strips markup and attributes

Bleach Bleach is an allowed-list-based HTML sanitizing library that escapes or strips markup and attributes. Bleach can also linkify text safely, appl

Mozilla 2.5k Dec 29, 2022
Use heroicons in your Django and Jinja templates.

heroicons Use heroicons in your Django and Jinja templates. Requirements Python 3.6 to 3.9 supported. Django 2.2 to 3.2 supported. Are your tests slow

Adam Johnson 52 Dec 14, 2022
Vehicle registration using Python, Django and SQlite3

PythonCrud Cadastro de veículos utilizando Python, Django e SQlite3 Para acessar o deploy no Heroku:

Jorge Thiago 4 May 20, 2022
Hello world written in Django.

Learning Django 💡 create a virtual environment create python -m venv ./venv. this virtualenv file will be excluded by .gitignore activate the virtual

Dipak giri 4 Nov 26, 2021
Django + Next.js integration

Django Next.js Django + Next.js integration From a comment on StackOverflow: Run 2 ports on the same server. One for django (public facing) and one fo

Quera 162 Jan 03, 2023
English dictionary using Django based on freecodecamp

English Dictionary Hi there, i made this english dictionary using Django based on freecodecamp.org tutorial :) Table of Contents Preview Technologies

Aline Alencar 3 May 09, 2022
Mobile Detect is a lightweight Python package for detecting mobile devices (including tablets).

Django Mobile Detector Mobile Detect is a lightweight Python package for detecting mobile devices (including tablets). It uses the User-Agent string c

Botir 6 Aug 31, 2022
This is a repository for a web application developed with Django, built with Crowdbotics

assignment_32558 This is a repository for a web application developed with Django, built with Crowdbotics Table of Contents Project Structure Features

Crowdbotics 1 Dec 29, 2021
I managed to attach the Django Framework to my Telegram Bot and set a webhook

I managed to attach the Django Framework to my Telegram Bot and set a webhook. I've been developing it from 10th of November 2021 and I want to have a basic working prototype.

Valentyn Vovchak 2 Sep 08, 2022
Automated image processing for Django. Currently v4.0

ImageKit is a Django app for processing images. Need a thumbnail? A black-and-white version of a user-uploaded image? ImageKit will make them for you.

Matthew Dapena-Tretter 2.1k Jan 04, 2023
Django Login Api With Python

How to run this project Download and extract this project Create an environment and install all the libraries from requiements.txt pip freeze -r requi

Vikash Kisku 1 Dec 10, 2021
The uncompromising Python code formatter

The Uncompromising Code Formatter “Any color you like.” Black is the uncompromising Python code formatter. By using it, you agree to cede control over

Python Software Foundation 30.7k Jan 03, 2023
Custom Django field for using enumerations of named constants

django-enumfield Provides an enumeration Django model field (using IntegerField) with reusable enums and transition validation. Installation Currently

5 Monkeys 195 Dec 20, 2022
Simple yet powerful and really extendable application for managing a blog within your Django Web site.

Django Blog Zinnia Simple yet powerful and really extendable application for managing a blog within your Django Web site. Zinnia has been made for pub

Julien Fache 2.1k Dec 24, 2022
An automatic django's update checker and MS teams notifier

Django Update Checker This is small script for checking any new updates/bugfixes/security fixes released in django News & Events and sending correspon

prinzpiuz 4 Sep 26, 2022
GameStop clone with Django

GameStop clone with Django This is my side project with GameStop clone Author: HackerApe GitHub Profile: View Profile LinkedIn Profile: View Profile

Dmitriy Shin 2 Dec 26, 2021
Simple tagging for django

django-taggit This is a Jazzband project. By contributing you agree to abide by the Contributor Code of Conduct and follow the guidelines. django-tagg

Jazzband 3k Jan 02, 2023
Packs a bunch of smaller CSS files together from 1 folder.

Packs a bunch of smaller CSS files together from 1 folder.

1 Dec 09, 2021