luigi.contrib.mysqldb module

class luigi.contrib.mysqldb.MySqlTarget(host, database, user, password, table, update_id, **cnx_kwargs)[source]


Target for a resource in MySql.

Initializes a MySqlTarget instance.

marker_table = 'table_updates'

Mark this update as complete.

IMPORTANT, If the marker table doesn’t exist, the connection transaction will be aborted and the connection reset. Then the marker table will be created.


Returns True if the Target exists and False otherwise.


Create marker table if it doesn’t exist.

Using a separate connection since the transaction might have to be reset.

class luigi.contrib.mysqldb.CopyToTable(*args, **kwargs)[source]

Bases: luigi.contrib.rdbms.CopyToTable

Template task for inserting a data set into MySQL

Usage: Subclass and override the required host, database, user, password, table and columns attributes.

To customize how to access data from an input task, override the rows method with a generator that yields each row as a tuple with fields ordered according to columns.


Return/yield tuples or lists corresponding to each row to be inserted.


Returns a MySqlTarget representing the inserted dataset.

Normally you don’t override this.

copy(cursor, file=None)[source]

Inserts data generated by rows() into target table.

If the target table doesn’t exist, self.create_table will be called to attempt to create the table.

Normally you don’t want to override this.