Send e-mails asyncronously using cron

Overview

django-yubin

https://codecov.io/github/APSL/django-yubin/coverage.svg?branch=master Documentation Status

Django Yubin allows the programmer to control when he wants to send the e-mail in this application, making the web application to answer fast as it not has to wait for the mail server.

As in our projects we use always two django packages for dealing with emails: django-mailer2 (our own fork in APSL) and django-mailviews to compose the emails we decided to create this package to fit our own needs and share with the community.

As you can see it seems django-mailer2 is not accepting patches, so in order to put a new version on pypi a new name was mandatory. So django-yubin was born (yubin is postal mail in japanese). The name attribution is for @morenosan.

How it works

Yubin replaces the standard Django Email Backend with its own. Instead of sending the e-mail trough the SMTP server Yubin stores the e-mails on the database and allows you to sent them using a cron command.

Advantages

  • Your application can answer faster, as other process is going to take care of connecting with the SMTP server and sending the e-mail.
  • Yubin stores the e-mail and allows you to retrieve using the admin. Even with the attachments.
  • Yubin allows to define prioritary queues, resend e-mails
  • Yubin helps in your development. It's a good way to work when you're developping the application and you don't want to flood your test users with e-mails. With Django Yubin, and without running the cron commands, you can see how your e-mails are, retrieve them and even delete them with out sending it.

On production mode you'll just nedd to add a cron entry in your server to send the e-mails, someting like

          • (cd $PROJECT; python manage.py send_mail >> $PROJECT/cron_mail.log 2>&1)

This will send the queued e-mail every minute.

Django Yubin is a fork from django-mailer2 with some addtions from django-mailviews and additional improvements made from apsl.net that we need for our daly basis workd. It has also contributions from other people, so don't heasitate to read the humans.txt.

django-mailer-2 by is a Chris Beaven fork from a fork of James Tauber's django-mailer and is a reusable Django app for queuing the sending of email.

django-mailviews from Disqus, allows you to compose e-mails using templates in the same way you compose your html templates, and allows you to preview the e-mails.

If you want to run the test you'll need a test smtpd server, you can find one in

./bin/fake-server

run it in a different console and execute runtests.py

You can read the package documentation at http://django-yubin.readthedocs.org/en/latest/

Changelog

  • 1.7.0 Add optional MAILER_MESSAGE_SEARCH_FIELDS setting. It's a tuple of strings with the fields to use in admin.Message.search_fields attribute.
  • 1.6.0 Support for Django 3.0
  • 1.5.0 New TemplatedMultipleAttachmentsEmailMessageView to allow to send emails with more than 1 attachment.
  • 1.4.1 Detecting if messages are encoding using different encoding headers to be able to preview them (now base64, quoted-printable).
  • 1.4.0 Option added in status_mail command to return the output in json format.
  • 1.3.1 Fix unicode and encode errors: sending queued and non queued emails and in admin detail view.
  • 1.3.0 Allow to send emails inmediatly without being saved in database (priority «now-not-queued»). Add support for Python 3.7 and Django 2.1. Remove old code for Django < 1.3.
  • 1.2.0 Fix is_base64 detection. Add a «send_test_email» command to check connection parameters. New health check view. Don't open a connection if there are no messages in queue to send. Add a "date_sent" field to detect when the mail was sent.
  • 1.1.0 Fix attachment headers in TemplateAttachmentEmailMessagView making both "attachment" and "filename" args mandatory.
  • 1.0.5 Add missing paths in MANIFEST.in.
  • 1.0.4 Fix attachment visualization in the admin. Attach pdf in create_mail command. Solved Content-Transfer-Encoding issue.
  • 1.0.3 Fixed issue decoding the message payload, added support for django 1.9, updated changelog and added support to deploy the package from travis.
  • 1.0.0 Add support for Django 2.0 and remove django 1.8.
  • 0.8.2 Fix date created column in QueuedMessages admin.
  • 0.8.1 Ensure that LOCK_WAIT_TIMEOUT is never negative to avoid a bug in lockfile in systems which use a LinkFileLock.
  • 0.8.0 Use settings.MAILER_PAUSE_SEND to skip smtp connections. Fix UTF-8 encoding in messages. Fix encoding errors in email visualization in the admin.
  • 0.7.0 Fix template context bug for Django 1.11. Add Python 3.6 to CI and drop Python 3.3 and Django 1.9.
  • 0.6.0 Support for Python 3.6.
  • 0.5.0 Limit nº of emails sent by send_mail command. Update the debug handlers options for verbosity to accept v3.
  • 0.4.0 Support Django 1.11: subject and body are no longer unscaped, you need to add {% autoescape off %} to your non HTML templates.
  • 0.3.1 Delete unused template that caused an error with django-compressor offline. testmail command now generates HTML emails.
  • 0.3.0 Support Django >= 1.8 and <=1.10, Python 2.7, 3.3, 3.4 and 3.5. Re-send mails admin action. Fix bug in status_mail command. Demo project configured to send mails with the same mail fake-server used for tests.
  • 0.2.3 Removed {% load url from future %} to support Django 1.9. Now Django < 1.5 is not supported.
  • 0.2.2 Include migrations directory in .tar.gz in PyPi.
  • 0.2.1 Updated links to CI and Code Coverage Services
  • 0.2.0 Merged sergei-maertens contribution.
  • 0.1.8 Added migrations for Django 1.9 compatibility. See http://django-yubin.readthedocs.org/en/latest/install.html#upgrading-from-previous-versions
  • 0.1.7 Support for Django 1.8.
  • 0.1.6 Bugfixes.
  • 0.1.5 Bugfixes.
  • 0.1.4 Updated README.
  • 0.1.3 Fixed Python3 compatibility, thanks Marc, Cesc & Dani.
  • 0.1.2 Fixed Templates.
  • 0.1.1 Updated documentation and unit tests.
Comments
  • Limit Emails per Call

    Limit Emails per Call

    To avoid mail throttles/spamming this adds a message_limit option to limit how many emails can get sent in a single call. I tested this with a number of settings and it's working for me.

    opened by the-mace 15
  • Introduce additional permission checks

    Introduce additional permission checks

    In the custom views in the Message admin check check if the 'change' permission for the 'Message' model is set in addition to the existing check if for is_active and is_staff which the django's admin_site.admin_view decorator introduces.

    opened by flupzor 8
  • Only warn if Yubin deferred or skipped mail

    Only warn if Yubin deferred or skipped mail

    It might be a matter of preference, however using loglevel warning for regular behaviour (1 or more emails have been sent successfully) seems incorrect in my eyes.

    As such a slight tweak to only use warning when Yubin defers or skips 1 or more emails.

    opened by alextreme 7
  • (fix) UnicodeEncodeError when priority is 'now-not-queued'

    (fix) UnicodeEncodeError when priority is 'now-not-queued'

    When trying to send email with priority 'now-not-queued' I can see in the errors files messages as :

    failed due to: 'ascii' codec can't encode character '\xf3' in position 481: ordinal not in range(128)
    

    This is the summarize:

    • Add testUnicodePriorityNowNotQueuedMessage
    • Bug fix for UnicodeEncodeError when priority is 'now-not-queued'
    • Bug fix for counting num_sent emailsI
    • Change models.QueuedMessage.objects.all() -> models.QueuedMessage.objects.filter(deferred__isnull=True) to check there is not deferred
    opened by xusy2k 6
  • Unicode encode errors

    Unicode encode errors

    Fix UnicodeEncodeErrors:

    • Sending queued emails
    • Sending non queued emails
    • In admin detail view

    Thanks to @xusy2k to find an trying to fix the bugs in PR #48.

    opened by sastred 5
  • Lock already in place. Exiting.

    Lock already in place. Exiting.

    Im having a significant issue with the lock system.

    Usually yubin works great for everything. However, sometimes the lock file does not get deleted properly when the program exits. I dont know if its because of abnormal termination or due to some other reason.

    Whatever the cause - after that, yubin cant run anymore because it sees the old lock file. Since Im not getting emails, sometimes it takes me days or weeks to realize that there is a problem. This happens rarely but often enough - roughly every few months.

    Obviously this is a really serious issue on a production system.

    I see there is a finally: block where the lock is supposed to get released. I have no idea why its not getting there sometimes.

    Im not so familiar with the system, so the only solution I can think of is to create a management command that can be run say once or twice a day that checks for a really old lock file and deletes it. For instance a lock file thats 4 hours old is probably not really supposed to be there.

    Maybe there is a more elegant and correct solution but thats all I can come up with given my limited knowledge

    Thanks

    Daniel

    opened by drownload 4
  • UnicodeDecodeError

    UnicodeDecodeError

    Traceback (most recent call last): File "./manage.py", line 10, in execute_from_command_line(sys.argv) File "/home/healersource/lib/python2.7/django/core/management/init.py", line 354, in execute_from_command_line utility.execute() File "/home/healersource/lib/python2.7/django/core/management/init.py", line 346, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/healersource/lib/python2.7/django/core/management/base.py", line 394, in run_from_argv self.execute(*args, **cmd_options) File "/home/healersource/lib/python2.7/django/core/management/base.py", line 445, in execute output = self.handle(*args, **options) File "/home/healersource/webapps/hs/env/lib/python2.7/site-packages/django_yubin/management/commands/send_mail.py", line 56, in handle message_limit=options['message_limit']) File "/home/healersource/webapps/hs/env/lib/python2.7/site-packages/django_yubin/engine.py", line 113, in send_all blacklist=blacklist) File "/home/healersource/webapps/hs/env/lib/python2.7/site-packages/django_yubin/engine.py", line 207, in send_queued_message smart_str(message.encoded_message).encode('utf-8')) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 559: ordinal not in range(128)

    opened by drownload 4
  • No CI for pushes/pull requests (Travis)

    No CI for pushes/pull requests (Travis)

    I see that there is no Travis integration. I could add this (set up the whole tox stack) so you can at least see if the build is failing for certain Python/Django versions or not.

    What's the support policy? Which Python/Django versions? I'd suggest Py27, Py34 and Py35, Django 1.8 and Django 1.9. For the rest I would point to django-mailer-2, which is unsupported but so are the Python/Django versions lower than those specified.

    opened by sergei-maertens 4
  • TemplateAttachmentEmailMessagView: filename required if attachment is provided

    TemplateAttachmentEmailMessagView: filename required if attachment is provided

    Before this fix, when an attachment was added using the render_to_message the filename and the content of the attachment wasn't provided to the attach method at the same time. This provoke that the Content-Disposition header wasn't added at the attachment multipart.

    The proposed solution force to assign a filename when an attachment is added (and vicecersa), in order to provide all the parameters to the attach method in order that the Content-Disposition header is written inside the attachment multipart.

    opened by csalom 3
  • Update the debug handlers options for verbosity

    Update the debug handlers options for verbosity

    Since the commands (send_mail etc.) are using the standard -v verbosity in the arg parser, the help for the command shows that there are four verbosity levels (0-3). At the moment if you run

    python manage.py send_mail -v3

    for instance, it fails with a cryptic message about the key "3" not being present in the LOGGING_LEVEL dictionary. This change duplicates the "2" key so that both 2 and 3 verbosity have the DEBUG message level.

    An alternative would be to specify the -v option in each of the commands to only allow -v[0/1/2].

    opened by davepeake 3
  • Rate limiting

    Rate limiting

    I've been using django-yubin for a while and it's a great addition. My challenge is sometimes my queue gets too many items in it and then I have a large amount of emails to send (all opt-in, no spamming). My email provider doesn't like it when the volume spikes and starts aggressively throttling and can even lock the account for a while. It would be really helpful if django-yubin had some way of limiting the number of emails per time period. So feed from the queue but if we've sent X emails in this cron cycle then stop sending emails until next cycle.

    Is anything like this possible now or is it possible to add something like this?

    I found an option, block_size, in the code which I didn't see in the docs. It's related but still sends all mail in the queue.

    opened by the-mace 3
  • To support creating a single Email object with several addresses

    To support creating a single Email object with several addresses

    It would be nice to support multiple addresses (either with to / cc / bcc fields) for the same Email. Currently, a new Message object is created per address.

    By allowing the same email to be sent to multiple addresses, we could reduce table space significantly.

    enhancement 
    opened by celdrake 0
Releases(1.7.1)
Owner
APSL
Web, mobile apps and DevOps mostly in Python, Django, JavaScript and Go.
APSL
Kanmail - An email client that functions like a kanban board, for Mac/Windows/Docker

Kanmail - An email client that functions like a kanban board, for Mac/Windows/Docker

Oxygem 1.2k Dec 31, 2022
A Django app that allows you to send email asynchronously in Django. Supports HTML email, database backed templates and logging.

Django Post Office Django Post Office is a simple app to send and manage your emails in Django. Some awesome features are: Allows you to send email as

User Inspired 856 Dec 25, 2022
Send email in Python conveniently for gmail using yagmail

yagmail -- Yet Another GMAIL/SMTP client For the asynchronous asyncio version, look here: https://github.com/kootenpv/aioyagmail The goal here is to m

Pascal van Kooten 2.4k Dec 31, 2022
Python IMAP for Human beings

Imbox - Python IMAP for Humans Python library for reading IMAP mailboxes and converting email content to machine readable data Requirements Python (3.

Martin Rusev 1.1k Dec 30, 2022
Email-osint - Email OSINT tool written in python3

Email-osint - Email OSINT tool written in python3

Surtains 7 Nov 28, 2022
send email & telegram message whenever an analog in is recieved

send email & telegram message whenever an analog in is recieved (so when attached to an alarm siren out it will alert via mail)

Naor Livne 2 Feb 11, 2022
This is a bot that interacts with you over voice and sends mail.Uses speech_recognition,pyttsx3 and smtplib

AutoMail This is a bot that interacts with you over voice and sends mail Before you run the bot , go to mail.py and put your respective email address

Aditya Subrahmanya Bhat 2 Nov 04, 2022
A Discord Mod Mail bot made in python

Fish-Mail The mod mail bot for Fish Hosting Note: You are not allowed to remove the names in the credit command Note: If you want any ideas/commands a

28 Aug 30, 2022
Bulk send personalized emails using a .csv file and Gmail API (via EZGmail)

GSender Bulk send personalized emails using a .csv file and Gmail API (via EZGmail). Installation Install requirements.txt. Follow the EZGmail Install

1 Nov 23, 2021
EmailAll - a powerful Email Collect tool

EmailAll A powerful Email Collect tool 0x1 介绍 😲 EmailAll is a powerful Email Co

473 Dec 22, 2022
A SMTP server for use as a pytest fixture that implements encryption and authentication.

SMTPDFix: Test email, locally A simple SMTP server based on aiosmtpd for use as a fixture with pytest that supports encryption and authentication. All

James Warne 11 Sep 03, 2022
automatic mails sender with attachments

أزعجني لين تدربني Automatic mails sender with attachments. Note: You need to have gmail account & and you need to turn on "Less secure app access" set

6 Dec 30, 2022
Djrill is an email backend and new message class for Django users that want to take advantage of the Mandrill transactional email service from MailChimp.

Djrill: Mandrill Transactional Email for Django Djrill integrates the Mandrill transactional email service into Django. PROJECT STATUS: INACTIVE As of

Brack3t 327 Oct 01, 2022
A Django email backend for Amazon's Simple Email Service

Django-SES Info: A Django email backend for Amazon's Simple Email Service Author: Harry Marr (http://github.com/hmarr, http://twitter.com/harrymarr) C

882 Dec 29, 2022
Envia-emails - A Python Program that creates emails

Envia-emails Os emails é algo muito importante e usado. Pensando nisso, eu criei

José Rodolfo 2 Mar 05, 2022
SMTP In some vulnerable configurations, email servers can also be aggregated Use information that gives us information about the host or network Give

SMTP In some vulnerable configurations, email servers can also be aggregated Use information that gives us information about the host or network Give. The SMTP protocol supports some basic commands s

m3hr44n 1 Jan 16, 2022
Send e-mails asyncronously using cron

django-yubin Django Yubin allows the programmer to control when he wants to send the e-mail in this application, making the web application to answer

APSL 44 Sep 24, 2022
An email generator code in python language

An email generator code in python language. I have done it in the simplest way possible and with no link to an SMTP server. Generating infinite emails until CTRL+C . It is a code that can be used in

Kelvin Ndungu Wanja 1 Jan 17, 2022
this is django project through this project you can easily sends message to any email

SEND-EMAIL this is django project through this project you can easily sends message to any email home when you run the server then you will see this t

Ankit jadhav 1 Oct 17, 2021
Will iterate through a list of emails on an attached csv file and email all of them a message of your choice

Email_Bot Will iterate through a list of emails on an attached csv file and email all of them a message of your choice. Before using, make sure you al

J. Brandon Walker 1 Nov 30, 2021