Send e-mails asyncronously using cron

Overview

django-yubin

https://codecov.io/github/APSL/django-yubin/coverage.svg?branch=master Documentation Status

Django Yubin allows the programmer to control when he wants to send the e-mail in this application, making the web application to answer fast as it not has to wait for the mail server.

As in our projects we use always two django packages for dealing with emails: django-mailer2 (our own fork in APSL) and django-mailviews to compose the emails we decided to create this package to fit our own needs and share with the community.

As you can see it seems django-mailer2 is not accepting patches, so in order to put a new version on pypi a new name was mandatory. So django-yubin was born (yubin is postal mail in japanese). The name attribution is for @morenosan.

How it works

Yubin replaces the standard Django Email Backend with its own. Instead of sending the e-mail trough the SMTP server Yubin stores the e-mails on the database and allows you to sent them using a cron command.

Advantages

  • Your application can answer faster, as other process is going to take care of connecting with the SMTP server and sending the e-mail.
  • Yubin stores the e-mail and allows you to retrieve using the admin. Even with the attachments.
  • Yubin allows to define prioritary queues, resend e-mails
  • Yubin helps in your development. It's a good way to work when you're developping the application and you don't want to flood your test users with e-mails. With Django Yubin, and without running the cron commands, you can see how your e-mails are, retrieve them and even delete them with out sending it.

On production mode you'll just nedd to add a cron entry in your server to send the e-mails, someting like

          • (cd $PROJECT; python manage.py send_mail >> $PROJECT/cron_mail.log 2>&1)

This will send the queued e-mail every minute.

Django Yubin is a fork from django-mailer2 with some addtions from django-mailviews and additional improvements made from apsl.net that we need for our daly basis workd. It has also contributions from other people, so don't heasitate to read the humans.txt.

django-mailer-2 by is a Chris Beaven fork from a fork of James Tauber's django-mailer and is a reusable Django app for queuing the sending of email.

django-mailviews from Disqus, allows you to compose e-mails using templates in the same way you compose your html templates, and allows you to preview the e-mails.

If you want to run the test you'll need a test smtpd server, you can find one in

./bin/fake-server

run it in a different console and execute runtests.py

You can read the package documentation at http://django-yubin.readthedocs.org/en/latest/

Changelog

  • 1.7.0 Add optional MAILER_MESSAGE_SEARCH_FIELDS setting. It's a tuple of strings with the fields to use in admin.Message.search_fields attribute.
  • 1.6.0 Support for Django 3.0
  • 1.5.0 New TemplatedMultipleAttachmentsEmailMessageView to allow to send emails with more than 1 attachment.
  • 1.4.1 Detecting if messages are encoding using different encoding headers to be able to preview them (now base64, quoted-printable).
  • 1.4.0 Option added in status_mail command to return the output in json format.
  • 1.3.1 Fix unicode and encode errors: sending queued and non queued emails and in admin detail view.
  • 1.3.0 Allow to send emails inmediatly without being saved in database (priority «now-not-queued»). Add support for Python 3.7 and Django 2.1. Remove old code for Django < 1.3.
  • 1.2.0 Fix is_base64 detection. Add a «send_test_email» command to check connection parameters. New health check view. Don't open a connection if there are no messages in queue to send. Add a "date_sent" field to detect when the mail was sent.
  • 1.1.0 Fix attachment headers in TemplateAttachmentEmailMessagView making both "attachment" and "filename" args mandatory.
  • 1.0.5 Add missing paths in MANIFEST.in.
  • 1.0.4 Fix attachment visualization in the admin. Attach pdf in create_mail command. Solved Content-Transfer-Encoding issue.
  • 1.0.3 Fixed issue decoding the message payload, added support for django 1.9, updated changelog and added support to deploy the package from travis.
  • 1.0.0 Add support for Django 2.0 and remove django 1.8.
  • 0.8.2 Fix date created column in QueuedMessages admin.
  • 0.8.1 Ensure that LOCK_WAIT_TIMEOUT is never negative to avoid a bug in lockfile in systems which use a LinkFileLock.
  • 0.8.0 Use settings.MAILER_PAUSE_SEND to skip smtp connections. Fix UTF-8 encoding in messages. Fix encoding errors in email visualization in the admin.
  • 0.7.0 Fix template context bug for Django 1.11. Add Python 3.6 to CI and drop Python 3.3 and Django 1.9.
  • 0.6.0 Support for Python 3.6.
  • 0.5.0 Limit nº of emails sent by send_mail command. Update the debug handlers options for verbosity to accept v3.
  • 0.4.0 Support Django 1.11: subject and body are no longer unscaped, you need to add {% autoescape off %} to your non HTML templates.
  • 0.3.1 Delete unused template that caused an error with django-compressor offline. testmail command now generates HTML emails.
  • 0.3.0 Support Django >= 1.8 and <=1.10, Python 2.7, 3.3, 3.4 and 3.5. Re-send mails admin action. Fix bug in status_mail command. Demo project configured to send mails with the same mail fake-server used for tests.
  • 0.2.3 Removed {% load url from future %} to support Django 1.9. Now Django < 1.5 is not supported.
  • 0.2.2 Include migrations directory in .tar.gz in PyPi.
  • 0.2.1 Updated links to CI and Code Coverage Services
  • 0.2.0 Merged sergei-maertens contribution.
  • 0.1.8 Added migrations for Django 1.9 compatibility. See http://django-yubin.readthedocs.org/en/latest/install.html#upgrading-from-previous-versions
  • 0.1.7 Support for Django 1.8.
  • 0.1.6 Bugfixes.
  • 0.1.5 Bugfixes.
  • 0.1.4 Updated README.
  • 0.1.3 Fixed Python3 compatibility, thanks Marc, Cesc & Dani.
  • 0.1.2 Fixed Templates.
  • 0.1.1 Updated documentation and unit tests.
Comments
  • Limit Emails per Call

    Limit Emails per Call

    To avoid mail throttles/spamming this adds a message_limit option to limit how many emails can get sent in a single call. I tested this with a number of settings and it's working for me.

    opened by the-mace 15
  • Introduce additional permission checks

    Introduce additional permission checks

    In the custom views in the Message admin check check if the 'change' permission for the 'Message' model is set in addition to the existing check if for is_active and is_staff which the django's admin_site.admin_view decorator introduces.

    opened by flupzor 8
  • Only warn if Yubin deferred or skipped mail

    Only warn if Yubin deferred or skipped mail

    It might be a matter of preference, however using loglevel warning for regular behaviour (1 or more emails have been sent successfully) seems incorrect in my eyes.

    As such a slight tweak to only use warning when Yubin defers or skips 1 or more emails.

    opened by alextreme 7
  • (fix) UnicodeEncodeError when priority is 'now-not-queued'

    (fix) UnicodeEncodeError when priority is 'now-not-queued'

    When trying to send email with priority 'now-not-queued' I can see in the errors files messages as :

    failed due to: 'ascii' codec can't encode character '\xf3' in position 481: ordinal not in range(128)
    

    This is the summarize:

    • Add testUnicodePriorityNowNotQueuedMessage
    • Bug fix for UnicodeEncodeError when priority is 'now-not-queued'
    • Bug fix for counting num_sent emailsI
    • Change models.QueuedMessage.objects.all() -> models.QueuedMessage.objects.filter(deferred__isnull=True) to check there is not deferred
    opened by xusy2k 6
  • Unicode encode errors

    Unicode encode errors

    Fix UnicodeEncodeErrors:

    • Sending queued emails
    • Sending non queued emails
    • In admin detail view

    Thanks to @xusy2k to find an trying to fix the bugs in PR #48.

    opened by sastred 5
  • Lock already in place. Exiting.

    Lock already in place. Exiting.

    Im having a significant issue with the lock system.

    Usually yubin works great for everything. However, sometimes the lock file does not get deleted properly when the program exits. I dont know if its because of abnormal termination or due to some other reason.

    Whatever the cause - after that, yubin cant run anymore because it sees the old lock file. Since Im not getting emails, sometimes it takes me days or weeks to realize that there is a problem. This happens rarely but often enough - roughly every few months.

    Obviously this is a really serious issue on a production system.

    I see there is a finally: block where the lock is supposed to get released. I have no idea why its not getting there sometimes.

    Im not so familiar with the system, so the only solution I can think of is to create a management command that can be run say once or twice a day that checks for a really old lock file and deletes it. For instance a lock file thats 4 hours old is probably not really supposed to be there.

    Maybe there is a more elegant and correct solution but thats all I can come up with given my limited knowledge

    Thanks

    Daniel

    opened by drownload 4
  • UnicodeDecodeError

    UnicodeDecodeError

    Traceback (most recent call last): File "./manage.py", line 10, in execute_from_command_line(sys.argv) File "/home/healersource/lib/python2.7/django/core/management/init.py", line 354, in execute_from_command_line utility.execute() File "/home/healersource/lib/python2.7/django/core/management/init.py", line 346, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/healersource/lib/python2.7/django/core/management/base.py", line 394, in run_from_argv self.execute(*args, **cmd_options) File "/home/healersource/lib/python2.7/django/core/management/base.py", line 445, in execute output = self.handle(*args, **options) File "/home/healersource/webapps/hs/env/lib/python2.7/site-packages/django_yubin/management/commands/send_mail.py", line 56, in handle message_limit=options['message_limit']) File "/home/healersource/webapps/hs/env/lib/python2.7/site-packages/django_yubin/engine.py", line 113, in send_all blacklist=blacklist) File "/home/healersource/webapps/hs/env/lib/python2.7/site-packages/django_yubin/engine.py", line 207, in send_queued_message smart_str(message.encoded_message).encode('utf-8')) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 559: ordinal not in range(128)

    opened by drownload 4
  • No CI for pushes/pull requests (Travis)

    No CI for pushes/pull requests (Travis)

    I see that there is no Travis integration. I could add this (set up the whole tox stack) so you can at least see if the build is failing for certain Python/Django versions or not.

    What's the support policy? Which Python/Django versions? I'd suggest Py27, Py34 and Py35, Django 1.8 and Django 1.9. For the rest I would point to django-mailer-2, which is unsupported but so are the Python/Django versions lower than those specified.

    opened by sergei-maertens 4
  • TemplateAttachmentEmailMessagView: filename required if attachment is provided

    TemplateAttachmentEmailMessagView: filename required if attachment is provided

    Before this fix, when an attachment was added using the render_to_message the filename and the content of the attachment wasn't provided to the attach method at the same time. This provoke that the Content-Disposition header wasn't added at the attachment multipart.

    The proposed solution force to assign a filename when an attachment is added (and vicecersa), in order to provide all the parameters to the attach method in order that the Content-Disposition header is written inside the attachment multipart.

    opened by csalom 3
  • Update the debug handlers options for verbosity

    Update the debug handlers options for verbosity

    Since the commands (send_mail etc.) are using the standard -v verbosity in the arg parser, the help for the command shows that there are four verbosity levels (0-3). At the moment if you run

    python manage.py send_mail -v3

    for instance, it fails with a cryptic message about the key "3" not being present in the LOGGING_LEVEL dictionary. This change duplicates the "2" key so that both 2 and 3 verbosity have the DEBUG message level.

    An alternative would be to specify the -v option in each of the commands to only allow -v[0/1/2].

    opened by davepeake 3
  • Rate limiting

    Rate limiting

    I've been using django-yubin for a while and it's a great addition. My challenge is sometimes my queue gets too many items in it and then I have a large amount of emails to send (all opt-in, no spamming). My email provider doesn't like it when the volume spikes and starts aggressively throttling and can even lock the account for a while. It would be really helpful if django-yubin had some way of limiting the number of emails per time period. So feed from the queue but if we've sent X emails in this cron cycle then stop sending emails until next cycle.

    Is anything like this possible now or is it possible to add something like this?

    I found an option, block_size, in the code which I didn't see in the docs. It's related but still sends all mail in the queue.

    opened by the-mace 3
  • To support creating a single Email object with several addresses

    To support creating a single Email object with several addresses

    It would be nice to support multiple addresses (either with to / cc / bcc fields) for the same Email. Currently, a new Message object is created per address.

    By allowing the same email to be sent to multiple addresses, we could reduce table space significantly.

    enhancement 
    opened by celdrake 0
Releases(1.7.1)
Owner
APSL
Web, mobile apps and DevOps mostly in Python, Django, JavaScript and Go.
APSL
Email-bomber - Email bomber unlike other email bombers u don't need your gmail email id to use this

Email-bomber - Email bomber unlike other email bombers u don't need your gmail email id to use this

rfeferfefe 82 Dec 17, 2022
Certificate generating and mailing system

skylab-certificate-system Through the this system, you can generate personalized certificates for people with name-surname-mail information in an exce

Oğuzhan Ercan 9 Sep 27, 2022
An automation program that checks whether email addresses are real, whether they exist and whether they are a validated mail

Email Validator It is an automation program that checks whether email addresses are real, whether they exist and whether they are a validated mail. Re

Ender MIRIZ 4 Dec 22, 2021
Send Multiple Mail From List With Python

Send Multiple Mail From List With Python You can send multiple e-mail using HTML themes with Python. Here is the e-mail information to be sent. #The m

Mücahid Eker 1 Dec 23, 2021
A SMTP server for use as a pytest fixture that implements encryption and authentication.

SMTPDFix: Test email, locally A simple SMTP server based on aiosmtpd for use as a fixture with pytest that supports encryption and authentication. All

James Warne 11 Sep 03, 2022
PGP encrypted / multipart templated emails for Django

Created by Stephen McDonald Introduction django-email-extras is a Django reusable app providing the ability to send PGP encrypted and multipart emails

stephenmcd 75 May 14, 2022
Mailrise is an SMTP server that converts the emails it receives into Apprise notifications

Mailrise is an SMTP server that converts the emails it receives into Apprise notifications. The intended use case is as an email relay for a home lab or network. By accepting ordinary email, Mailrise

Ryan Young 293 Jan 07, 2023
Python Email Sender (PES) is a program made with Python using smtplib, socket and tkinter.

Python Email Sender (PES) is a program made with Python using smtplib, socket and tkinter. This program was made for sender email to be a gmail account because that's what I used when testing it out,

Zacky2613 1 Aug 26, 2022
A news curator and newsletter subscription package for Django

django-newsfeed What is django-newsfeed? django-newsfeed is a news curator and newsletter subscription package for django. It can be used to create a

Maksudul Haque 179 Nov 14, 2022
Suplantar mails de empresas como google, facebook, github, etc...

Suplantar mails de empresas como google, facebook, github, etc...

piter 3 Feb 05, 2022
SMTP In some vulnerable configurations, email servers can also be aggregated Use information that gives us information about the host or network Give

SMTP In some vulnerable configurations, email servers can also be aggregated Use information that gives us information about the host or network Give. The SMTP protocol supports some basic commands s

m3hr44n 1 Jan 16, 2022
A simple email sender

Email-Sender Un semplice Email-Sender che utilizza il modulo smtplib con aggiunta di interfaccia grafica realizzata con il modulo tkinter Per il corre

Vincenzo Caliendo 0 Jan 14, 2022
spam_box is a self hosted temp mail service by hacksec

spam_box spam_box is a self hosted temp mail service by hacksec Requirement python3 open port 25 and 6660 root access in a vps How to install in linux

ScRiPt1337 25 Dec 14, 2022
A simple free API that allows you to extract abuse emails from IPs.

Abuse-Email-API A simple free API that allows you to extract abuse emails from IPs. also isnt worth 500 dollars :) Requirements A Debian based OS The

Keratin 1 Dec 20, 2021
Bulk Email and certificate sending application

demir.ai E-mail services This application allows you to send automatic mass mail and automatic mass certificates to the people in your mailing list, m

Ahmet Furkan DEMIR 16 Nov 01, 2022
A functional demo of the O365 Module to send an email on an authenticated, tokenized account.

O365_email A functional demo of the O365 Module to send an email on an authenticated, tokenized account. Prep Create an app in Azure Developer's porta

2 Oct 14, 2022
Kanmail - An email client that functions like a kanban board, for Mac/Windows/Docker

Kanmail - An email client that functions like a kanban board, for Mac/Windows/Docker

Oxygem 1.2k Dec 31, 2022
Django module to easily send emails/sms/tts/push using django templates stored on database and managed through the Django Admin

Django-Db-Mailer Documentation available at Read the Docs. What's that Django module to easily send emails/push/sms/tts using django templates stored

LPgenerator 250 Dec 21, 2022
Read/sync your IMAP mailboxes (python2)

Upstream status (master branch): Upstream status (next branch): Financial contributors: Links: Official github code repository: offlineimap Website: w

OfflineIMAP 1.7k Dec 29, 2022
This python script will generate passwords for your emails, With certain lengths, And saves them into plain text files.

How to use. Change the Default length of genereated password in default.length.txt Type the email for your account. Type the website that the email an

2 Dec 26, 2021