luigi.contrib.mysqldb module

class luigi.contrib.mysqldb.MySqlTarget(host, database, user, password, table, update_id, **cnx_kwargs)

Bases: Target

Target for a resource in MySql.

Initializes a MySqlTarget instance.

Parameters:

• host (str) – MySql server address. Possibly a host:port string.

• database (str) – database name.

• user (str) – database user

• password (str) – password for specified user.

• update_id (str) – an identifier for this data set.

• cnx_kwargs – optional params for mysql connector constructor. See https://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html.

marker_table = 'table_updates'

touch(connection=None)

Mark this update as complete.

IMPORTANT, If the marker table doesn’t exist, the connection transaction will be aborted and the connection reset. Then the marker table will be created.

exists(connection=None)

Returns True if the Target exists and False otherwise.

connect(autocommit=False)

create_marker_table()

Create marker table if it doesn’t exist.

Using a separate connection since the transaction might have to be reset.

class luigi.contrib.mysqldb.CopyToTable(*args, **kwargs)

Bases: CopyToTable

Template task for inserting a data set into MySQL

Usage: Subclass and override the required host, database, user, password, table and columns attributes.

To customize how to access data from an input task, override the rows method with a generator that yields each row as a tuple with fields ordered according to columns.

rows()

Return/yield tuples or lists corresponding to each row to be inserted.

output()

Returns a MySqlTarget representing the inserted dataset.

Normally you don’t override this.

copy(cursor, file=None)

run()

Inserts data generated by rows() into target table.

If the target table doesn’t exist, self.create_table will be called to attempt to create the table.

Normally you don’t want to override this.

property bulk_size