luigi.batch_notifier

Library for sending batch notifications from the Luigi scheduler. This module is internal to Luigi and not designed for use in other contexts.

Classes

BatchNotifier(**kwargs)
ExplQueue(num_items)
batch_email(*args, **kwargs)

class luigi.batch_notifier.batch_email(*args, **kwargs)

email_interval

Parameter whose value is an int.

batch_mode

A parameter which takes two values:

1. an instance of Iterable and

2. the class of the variables to convert to.

In the task definition, use

class MyTask(luigi.Task):
    my_param = luigi.ChoiceParameter(choices=[0.1, 0.2, 0.3], var_type=float)

At the command line, use

$ luigi --module my_tasks MyTask --my-param 0.1

Consider using EnumParameter for a typed, structured alternative. This class can perform the same role when all choices are the same type and transparency of parameter value on the command line is desired.

error_lines

Parameter whose value is an int.

error_messages

Parameter whose value is an int.

group_by_error_messages

A Parameter whose value is a bool. This parameter has an implicit default value of False. For the command line interface this means that the value is False unless you add "--the-bool-parameter" to your command without giving a parameter value. This is considered implicit parsing (the default). However, in some situations one might want to give the explicit bool value ("--the-bool-parameter true|false"), e.g. when you configure the default value to be True. This is called explicit parsing. When omitting the parameter value, it is still considered True but to avoid ambiguities during argument parsing, make sure to always place bool parameters behind the task family on the command line when using explicit parsing.

You can toggle between the two parsing modes on a per-parameter base via

class MyTask(luigi.Task):
    implicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.IMPLICIT_PARSING)
    explicit_bool = luigi.BoolParameter(parsing=luigi.BoolParameter.EXPLICIT_PARSING)

or globally by

luigi.BoolParameter.parsing = luigi.BoolParameter.EXPLICIT_PARSING

for all bool parameters instantiated after this line.

class luigi.batch_notifier.ExplQueue(num_items)

enqueue(item)

class luigi.batch_notifier.BatchNotifier(**kwargs)

add_failure(task_name, family, unbatched_args, expl, owners)

add_disable(task_name, family, unbatched_args, owners)

add_scheduling_fail(task_name, family, unbatched_args, expl, owners)

send_email()

update()