luigi.notifications module

Supports sending emails when tasks fail.

This needs some more documentation. See Configuration for configuration options. In particular using the config receiver should set up Luigi so that it will send emails when tasks fail.

[email]
receiver=foo@bar.baz

class luigi.notifications.TestNotificationsTask(*args, **kwargs)

Bases: luigi.task.Task

You may invoke this task to quickly check if you correctly have setup your notifications Configuration. You can run:

$ luigi TestNotificationsTask --local-scheduler --email-force-send

And then check your email inbox to see if you got an error email or any other kind of notifications that you expected.

raise_in_complete = BoolParameter (defaults to False): If true, fail in complete() instead of run()

run()

The task run method, to be overridden in a subclass.

See Task.run

complete()

If the task has any outputs, return True if all outputs exist. Otherwise, return False.

However, you may freely override this method with custom logic.

class luigi.notifications.email(*args, **kwargs)

Bases: luigi.task.Config

force_send = BoolParameter (defaults to False): Send e-mail even from a tty

format = ChoiceParameter (defaults to plain): Format type for sent e-mails Choices: {plain, none, html}

method = ChoiceParameter (defaults to smtp): Method for sending e-mail Choices: {ses, sendgrid, smtp, sns}

prefix = Parameter (defaults to ): Prefix for subject lines of all e-mails

receiver = Parameter (defaults to ): Address to send error e-mails to

sender = Parameter (defaults to luigi-client@build-11151103-project-12134-luigi): Address to send e-mails from

class luigi.notifications.smtp(*args, **kwargs)

Bases: luigi.task.Config

host = Parameter (defaults to localhost): Hostname of smtp server

local_hostname = Parameter (defaults to None): If specified, local_hostname is used as the FQDN of the local host in the HELO/EHLO command

no_tls = BoolParameter (defaults to False): Do not use TLS in SMTP connections

password = Parameter (defaults to None): Password for the SMTP server login

port = IntParameter (defaults to 0): Port number for smtp server

ssl = BoolParameter (defaults to False): Use SSL for the SMTP connection.

timeout = FloatParameter (defaults to 10.0): Number of seconds before timing out the smtp connection

username = Parameter (defaults to None): Username used to log in to the SMTP host

class luigi.notifications.sendgrid(*args, **kwargs)

Bases: luigi.task.Config

apikey = Parameter: API key for SendGrid login

luigi.notifications.generate_email(sender, subject, message, recipients, image_png)

luigi.notifications.wrap_traceback(traceback)

For internal use only (until further notice)

luigi.notifications.send_email_smtp(sender, subject, message, recipients, image_png)

luigi.notifications.send_email_ses(sender, subject, message, recipients, image_png)

Sends notification through AWS SES.

Does not handle access keys. Use either

1/ configuration file 2/ EC2 instance profile

See also https://boto3.readthedocs.io/en/latest/guide/configuration.html.

luigi.notifications.send_email_sendgrid(sender, subject, message, recipients, image_png)

luigi.notifications.send_email_sns(sender, subject, message, topic_ARN, image_png)

Sends notification through AWS SNS. Takes Topic ARN from recipients.

Does not handle access keys. Use either

1/ configuration file 2/ EC2 instance profile

See also https://boto3.readthedocs.io/en/latest/guide/configuration.html.

luigi.notifications.send_email(subject, message, sender, recipients, image_png=None)

Decides whether to send notification. Notification is cancelled if there are no recipients or if stdout is onto tty or if in debug mode.

Dispatches on config value email.method. Default is ‘smtp’.

luigi.notifications.send_error_email(subject, message, additional_recipients=None)

Sends an email to the configured error email, if it’s configured.

luigi.notifications.format_task_error(headline, task, command, formatted_exception=None)

Format a message body for an error email related to a luigi.task.Task

Parameters:

• headline – Summary line for the message
• task – luigi.task.Task instance where this error occurred
• formatted_exception – optional string showing traceback

Returns:

message body