luigi.contrib.sge_runner module

The SunGrid Engine runner

The main() function of this module will be executed on the compute node by the submitted job. It accepts as a single argument the shared temp folder containing the package archive and pickled task to run, and carries out these steps:

• extract tarfile of package dependencies and place on the path
• unpickle SGETask instance created on the master node
• run SGETask.work()

On completion, SGETask on the master node will detect that the job has left the queue, delete the temporary folder, and return from SGETask.run()

luigi.contrib.sge_runner.main(args=['/home/docs/checkouts/readthedocs.org/user_builds/luigi/envs/latest/bin/sphinx-build', '-T', '-E', '-b', 'readthedocs', '-d', '_build/doctrees-readthedocs', '-D', 'language=en', '.', '_build/html'])

Run the work() method from the class instance in the file “job-instance.pickle”.