dqm.completeness package

Submodules

dqm.completeness.main module

dqm.completeness.metric module

Data Completeness Evaluation Module

This module provides tools to assess the completeness of tabular data. It is especially useful in data preprocessing and cleaning stages of a data analysis workflow. The module includes a class, DataCompleteness, with methods to calculate completeness scores for dataframes and individual columns. These methods help in identifying columns with missing data and quantifying the extent of missingness.

Authors:: Faouzi ADJED Anani DJATO
Classes:: DataCompleteness: A class that encapsulates the methods for evaluating data completeness.

dqm.completeness.metric.completeness_tabular(): Calculates the average completeness score for a dataframe.

dqm.completeness.metric.data_completion(): Calculates the completeness score for an individual data column.

Dependencies:

numpy pandas matplotlib scipy seaborn warnings

Usage:

The DataCompleteness class can be used as follows:

from data_completeness import DataCompleteness

# Create an instance of the class completeness_evaluator = DataCompleteness()

# Load your data into a pandas DataFrame df = pd.read_csv(‘your_data_path.csv’)

# Calculate the overall completeness score for the DataFrame overall_score = completeness_evaluator.completeness_tabular(df)

# Calculate the completeness score for a single column column_score = completeness_evaluator.data_completion(df[‘your_column’])

# Print the results print(f’Overall Data Completeness Score: {overall_score}’) print(f’Completeness Score for Column: {column_score}’)

class dqm.completeness.metric.DataCompleteness[source]

Bases: object

This class provides methods to evaluate the completeness of tabular data.

It includes methods to calculate completeness scores for individual columns and for entire dataframes by assessing the presence of non-null data.

completeness_tabular()[source]: Calculate the average completeness score of a dataframe.

data_completion()[source]: Calculate the completeness score of a single data column.

completeness_tabular(data)[source]

Calculate the average completeness score of the entire dataframe.

Parameters:

data (pd.DataFrame) – The dataframe to be evaluated for completeness.

Returns:

The average completeness score of: all columns in the dataframe.

Return type:

score_total(float)

data_completion(data)[source]

Calculate the completeness score of a single data column.