luigi.contrib.pig module

Apache Pig support. Example configuration section in luigi.cfg:

[pig]
# pig home directory
home: /usr/share/pig
class luigi.contrib.pig.PigJobTask(*args, **kwargs)[source]

Bases: luigi.task.Task

pig_home()[source]
pig_command_path()[source]
pig_env_vars()[source]

Dictionary of environment variables that should be set when running Pig.

Ex::
return { ‘PIG_CLASSPATH’: ‘/your/path’ }
pig_properties()[source]

Dictionary of properties that should be set when running Pig.

Example:

return { 'pig.additional.jars':'/path/to/your/jar' }
pig_parameters()[source]

Dictionary of parameters that should be set for the Pig job.

Example:

return { 'YOUR_PARAM_NAME':'Your param value' }
pig_options()[source]

List of options that will be appended to the Pig command.

Example:

return ['-x', 'local']
output()[source]
pig_script_path()[source]

Return the path to the Pig script to be run.

run()[source]
track_and_progress(cmd)[source]
class luigi.contrib.pig.PigRunContext[source]

Bases: object

kill_job(captured_signal=None, stack_frame=None)[source]
exception luigi.contrib.pig.PigJobError(message, out=None, err=None)[source]

Bases: exceptions.RuntimeError