fusetools.db_etl_tools.Generic

class fusetools.db_etl_tools.Generic

Bases: object

Generic functions for SQL queries and ETL.

Methods

db_apply_schema	Converts Pandas DataFrame columns based on schema DataFrame provided.
make_db_cols	Returns a Pandas DataFrame column names that are converted for database standards.
make_db_schema	Creates a mapping of Pandas data types to SQL data types.
make_groupby	Creates a dynmaically generated GROUP BY clause for a given SQL statement.
run_query	Executes a SQL query.

classmethod db_apply_schema(df, schema_df)

Converts Pandas DataFrame columns based on schema DataFrame provided.

Parameters

• df – A Pandas DataFrame with column types to be converted.

• schema_df – A Pandas DataFrame of columns with corresponding SQL data types.

Returns

Pandas DataFrame with columns converted to SQL schema.

classmethod make_db_cols(df)

Returns a Pandas DataFrame column names that are converted for database standards.

Parameters

df – A Pandas DataFrame with columns to be transformed

Returns

Pandas DataFrame column names that are converted for database standards.

classmethod make_db_schema(df)

Creates a mapping of Pandas data types to SQL data types.

Parameters

df – A Pandas DataFrame with column types to be converted.

Returns

A Pandas DataFrame of columns with corresponding SQL data types.

classmethod make_groupby(sql, dim_fact_delim)

Creates a dynmaically generated GROUP BY clause for a given SQL statement.

Parameters

• sql – SQL statement provided.

• dim_fact_delim – Delimiter between selected columns.

Returns

A complete SQL statement with dynamically generated GROUP BY clause.

classmethod run_query(engine, sql)

Executes a SQL query.

Parameters

• engine – A database engine object.

• sql – A SQL statement to be executed.

Returns

Time for execution of SQL query.