luigi.local_target module

LocalTarget provides a concrete implementation of a Target class that uses files on the local file system

class luigi.local_target.atomic_file(path)[source]

Bases: luigi.target.AtomicLocalFile

Simple class that writes to a temp file and moves it on close() Also cleans up the temp file if close is not invoked

move_to_final_destination()[source]
generate_tmp_path(path)[source]
class luigi.local_target.LocalFileSystem[source]

Bases: luigi.target.FileSystem

Wrapper for access to file system operations.

Work in progress - add things as needed.

copy(old_path, new_path, raise_if_exists=False)[source]

Copy a file or a directory with contents. Currently, LocalFileSystem and MockFileSystem support only single file copying but S3Client copies either a file or a directory as required.

exists(path)[source]

Return True if file or directory at path exist, False otherwise

Parameters:path (str) – a path within the FileSystem to check for existence.
mkdir(path, parents=True, raise_if_exists=False)[source]

Create directory at location path

Creates the directory at path and implicitly create parent directories if they do not already exist.

Parameters:
  • path (str) – a path within the FileSystem to create as a directory.
  • parents (bool) – Create parent directories when necessary. When parents=False and the parent directory doesn’t exist, raise luigi.target.MissingParentDirectory
  • raise_if_exists (bool) – raise luigi.target.FileAlreadyExists if the folder already exists.
isdir(path)[source]

Return True if the location at path is a directory. If not, return False.

Parameters:path (str) – a path within the FileSystem to check as a directory.

Note: This method is optional, not all FileSystem subclasses implements it.

listdir(path)[source]

Return a list of files rooted in path.

This returns an iterable of the files rooted at path. This is intended to be a recursive listing.

Parameters:path (str) – a path within the FileSystem to list.

Note: This method is optional, not all FileSystem subclasses implements it.

remove(path, recursive=True)[source]

Remove file or directory at location path

Parameters:
  • path (str) – a path within the FileSystem to remove.
  • recursive (bool) – if the path is a directory, recursively remove the directory and all of its descendants. Defaults to True.
move(old_path, new_path, raise_if_exists=False)[source]

Move file atomically. If source and destination are located on different filesystems, atomicity is approximated but cannot be guaranteed.

rename_dont_move(path, dest)[source]

Rename path to dest, but don’t move it into the dest folder (if it is a folder). This method is just a wrapper around the move method of LocalTarget.

class luigi.local_target.LocalTarget(path=None, format=None, is_tmp=False)[source]

Bases: luigi.target.FileSystemTarget

fs = <luigi.local_target.LocalFileSystem object>
makedirs()[source]

Create all parent folders if they do not exist.

open(mode='r')[source]

Open the FileSystem target.

This method returns a file-like object which can either be read from or written to depending on the specified mode.

Parameters:mode (str) – the mode r opens the FileSystemTarget in read-only mode, whereas w will open the FileSystemTarget in write mode. Subclasses can implement additional options.
move(new_path, raise_if_exists=False)[source]
move_dir(new_path)[source]
remove()[source]

Remove the resource at the path specified by this FileSystemTarget.

This method is implemented by using fs.

copy(new_path, raise_if_exists=False)[source]
fn