dqm.domain_gap package

Submodules

dqm.domain_gap.main module

dqm.domain_gap.metrics module

This module defines a GapMetric class responsible for calculating the Domain Gap distance between source and target data using various methods and models. It utilizes various methods and models for this purpose.

Authors:

Yoann RANDON Sabrina CHAOUCHE Faouzi ADJED

Dependencies:

time json argparse typing mlflow torchvision.models (resnet50, ResNet50_Weights, resnet18, ResNet18_Weights) utils (DomainMeter) twe_logger

Classes:

DomainGapMetrics: Class for calculating Central Moment Discrepancy (CMD)

distance between source and target data.

Functions: None

Usage: 1. Create an instance of GapMetric. 2. Parse the configuration using parse_config(). 3. Load the CNN model using load_model(cfg). 4. Compute the CMD distance using compute_distance(cfg). 5. Log MLflow parameters using set_mlflow_params(cfg). 6. Process multiple tasks using process_tasks(cfg, tsk).

class dqm.domain_gap.metrics.CMD

Bases: Metric

compute(cfg)

Compute the Central Moment Discrepancy (CMD) loss between source and target datasets using a pre-trained model.

This method calculates the CMD loss, which measures the discrepancy between the distributions of features extracted from source and target datasets. The features are extracted from specified layers of the model, and the loss is computed as a weighted sum of the differences in moments of the feature distributions.

Parameters:

cfg (Dict) –

A configuration dictionary containing the following keys: - “DATA”: Dictionary with data-related configurations:

• ”source” (str): Path to the source folder containing images.

• ”target” (str): Path to the target folder containing images.

• ”batch_size” (int): The batch size for data loading.

• ”width” (int): The width of the images.

• ”height” (int): The height of the images.

• ”norm_mean” (list of float): Mean values for image normalization.

• ”norm_std” (list of float): Standard deviation values for image normalization.

• ”MODEL”: Dictionary with model-related configurations:

  • ”arch” (str): The architecture of the model to use.

  • ”n_layer_feature” (list of int): List of layer numbers from which to extract features.

  • ”feature_extractors_layers_weights” (list of float): Weights for each feature layer.

  • ”device” (str): The device to run the model on (e.g., “cpu” or “cuda”).

• ”METHOD”: Dictionary with method-related configurations:

  • ”k” (int): The number of moments to consider in the CMD calculation.

Returns:

The computed CMD loss between the source and target datasets.

Return type:

float

The method performs the following steps: 1. Constructs data loaders for the source and target datasets with specified transformations. 2. Loads the model and sets it up on the specified device. 3. Extracts features from the specified layers of the model for both datasets. 4. Computes the moments of the feature distributions for both datasets. 5. Calculates the CMD loss as a weighted sum of the differences in moments. 6. Returns the total CMD loss.

Raises:

• AssertionError – If the source and target datasets do not have the same number of samples.

• AssertionError – If the keys of the feature weights dictionary do not match the specified feature layers.

class dqm.domain_gap.metrics.FID

Bases: Metric

calculate_statistics(features)

Calculate the mean and covariance matrix of a set of features.

This method computes the mean vector and the covariance matrix for a given set of features. It converts the features from a PyTorch tensor to a NumPy array for easier manipulation and statistical calculations.

Parameters:

features (torch.Tensor) – A 2D tensor where each row represents a feature vector. The tensor should have shape (N, D), where N is the number of samples and D is the dimensionality of the features.

Returns:

A tuple containing:

• mu (numpy.ndarray): The mean vector of the features, with shape (D,).

• sigma (numpy.ndarray): The covariance matrix of the features, with shape (D, D).

Return type:

tuple

The function performs the following steps: 1. Converts the features tensor to a NumPy array for easier manipulation. 2. Computes the mean vector of the features. 3. Computes the covariance matrix of the features.

compute_image_distance(cfg)

Compute the Frechet Inception Distance (FID) between two sets of images.

This method calculates the FID between images from a source and target dataset using a pre-trained InceptionV3 model to extract features. The FID is a measure of the similarity between two distributions of images, commonly used to evaluate the quality of generated images.

Parameters:

cfg (dict) –

A configuration dictionary containing the following keys: - “MODEL”: Dictionary with model-related configurations:

• ”device” (str): The device to run the model on (e.g., “cpu” or “cuda”).

• ”n_layer_feature” (int): The layer number from which to extract features.

• ”DATA”: Dictionary with data-related configurations:

  • ”width” (int): The width of the images.

  • ”height” (int): The height of the images.

  • ”norm_mean” (list of float): Mean values for image normalization.

  • ”norm_std” (list of float): Standard deviation values for image normalization.

  • ”batch_size” (int): The batch size for data loading.

  • ”source” (str): Path to the source folder containing images.

  • ”target” (str): Path to the target folder containing images.

Returns:

The computed FID score, representing the distance between the source and target image distributions.

Return type:

torch.Tensor

The method performs the following steps: 1. Loads the InceptionV3 model and sets it up on the specified device. 2. Constructs data loaders for the source and target datasets with specified transformations. 3. Extracts features from the specified layer of the model for both datasets. 4. Calculates the mean and covariance of the features for both datasets. 5. Computes the FID score using the means and covariances of the features. 6. Ensures the FID score is positive by taking the absolute value.

class dqm.domain_gap.metrics.KLMVN

Bases: Metric

Instanciate KLMVN class to compute KLMVN metrics

calculate_statistics(features)

Calculate the mean and covariance matrix of a set of features.

This function computes the mean vector and the covariance matrix for a given set of features. It ensures that the feature matrix has full rank, which is necessary for certain statistical operations.

Parameters:

features (torch.Tensor) – A 2D tensor where each row represents a feature vector. The tensor should have shape (N, D), where N is the number of samples and D is the dimensionality of the features.

Returns:

A tuple containing:

• mu (torch.Tensor): The mean vector of the features, with shape (D,).

• sigma (torch.Tensor): The covariance matrix of the features, with shape (D, D).

Return type:

tuple

Raises:

AssertionError – If the feature matrix does not have full rank.

The function performs the following steps: 1. Computes the mean vector of the features. 2. Centers the features by subtracting the mean vector. 3. Computes the covariance matrix using the centered features. 4. Checks the rank of the feature matrix to ensure it has full rank.

compute_image_distance(cfg)

Compute the distance between image features from source and target datasets using a pre-trained model.

This method calculates the distance between the statistical representations of image features extracted from two datasets. It uses a pre-trained model to extract features from specified layers and computes the Kullback-Leibler divergence between the distributions of these features.

Parameters:

cfg (dict) –

A configuration dictionary containing the following keys: - “MODEL”: Dictionary with model-related configurations:

• ”device” (str): The device to run the model on (e.g., “cpu” or “cuda”).

• ”arch” (str): The architecture of the model to use.

• ”n_layer_feature” (int): The layer number from which to extract features.

• ”DATA”: Dictionary with data-related configurations:

  • ”width” (int): The width of the images.

  • ”height” (int): The height of the images.

  • ”norm_mean” (list of float): Mean values for image normalization.

  • ”norm_std” (list of float): Standard deviation values for image normalization.

  • ”batch_size” (int): The batch size for data loading.

  • ”source” (str): Path to the source folder containing images.

  • ”target” (str): Path to the target folder containing images.

Returns:

The computed distance between the source and target image features.

Return type:

float

The method performs the following steps: 1. Loads the model and sets it up on the specified device. 2. Constructs data loaders for the source and target datasets with specified transformations. 3. Extracts features from the specified layer of the model for both datasets. 4. Calculates the mean and covariance of the features for both datasets. 5. Regularizes the covariance matrices to ensure numerical stability. 6. Computes the Kullback-Leibler divergence between the feature distributions.

klmvn(mu1, cov1, mu2, cov2, device)

Compute the Kullback-Leibler (KL) divergence between two multivariate normal distributions.

This method calculates the KL divergence between two multivariate normal distributions defined by their mean vectors and covariance matrices. It assumes that the covariance matrices are diagonal.

Parameters:

• mu1 (torch.Tensor) – Mean vector of the first multivariate normal distribution.

• cov1 (torch.Tensor) – Diagonal elements of the covariance matrix of the first distribution.

• mu2 (torch.Tensor) – Mean vector of the second multivariate normal distribution.

• cov2 (torch.Tensor) – Diagonal elements of the covariance matrix of the second distribution.

• device (torch.device) – The device (CPU or GPU) on which to perform the computation.

Returns:

The computed KL divergence between the two distributions.

Return type:

torch.Tensor

The method performs the following steps: 1. Constructs diagonal covariance matrices from the provided diagonal elements. 2. Creates multivariate normal distributions using the mean vectors and covariance matrices. 3. Computes the KL divergence between the two distributions.

regularize_covariance(cov_matrix, epsilon=1e-06)

Regularize a covariance matrix by adding a small value to its diagonal elements.

This function enhances the numerical stability of a covariance matrix by adding a small constant to its diagonal. This is particularly useful when the covariance matrix is nearly singular or when performing operations that require the matrix to be positive definite.

Parameters:

• cov_matrix (numpy.ndarray) – The covariance matrix to be regularized. It should be a square matrix.

• epsilon (float, optional) – A small value to add to the diagonal elements of the covariance matrix. Default is 1e-6.

Returns:

The regularized covariance matrix with the small value added to its diagonal.

Return type:

numpy.ndarray

The function performs the following steps: 1. Adds the specified epsilon value to the diagonal elements of the input covariance matrix. 2. Returns the modified covariance matrix.

class dqm.domain_gap.metrics.MMD

Bases: Metric

Maximum Mean Discrepancy metric class defintion

compute(cfg)

Computes a domain gap metric between two datasets using a specified kernel method.

This function extracts features from source and target datasets using a deep learning model, applies a specified kernel function (linear, RBF, or polynomial), and computes a similarity measure between the datasets.

Parameters:

cfg (dict) –

Configuration dictionary containing: - DATA:

• source (str): Path to the source dataset.

• target (str): Path to the target dataset.

• batch_size (int): Batch size for dataloaders.

• width (int): Width of input images.

• height (int): Height of input images.

• norm_mean (tuple): Mean for normalization.

• norm_std (tuple): Standard deviation for normalization.

• MODEL:

  • arch (str): Model architecture.

  • n_layer_feature (int): Layer from which features are extracted.

  • device (str): Device to run computations (‘cpu’ or ‘cuda’).

• METHOD:

  • kernel (str): Kernel type (‘linear’, ‘rbf’, ‘poly’).

  • kernel_params (dict): Parameters for the chosen kernel.

Returns:

Computed domain gap value based on the selected kernel.

Return type:

float

Raises:

AssertionError – If source and target datasets have different sizes.

class dqm.domain_gap.metrics.Metric

Bases: object

Base class for defining a metric.

compute()

Compute the value of the metric.

Return type:

float

class dqm.domain_gap.metrics.ProxyADistance

Bases: Metric

adapt_format_like_pred(y, pred)

Convert a list of class indices into a one-hot encoded tensor matching the format of the predictions.

This method takes a list of class indices and converts it into a one-hot encoded tensor that matches the shape and format of the provided predictions tensor. This is useful for comparing ground truth labels with model predictions in a consistent format.

Parameters:

• y (torch.Tensor or list) – A 1D tensor or list containing class indices. Each element should be an integer representing the class index.

• pred (torch.Tensor) – A 2D tensor containing predicted probabilities or scores for each class. The shape should be (N, C), where N is the number of samples and C is the number of classes.

Returns:

A one-hot encoded tensor of the same shape as pred, where each row has a 1 at the

index of the true class and 0 elsewhere.

Return type:

torch.Tensor

The method performs the following steps: 1. Initializes a zero tensor with the same shape as pred. 2. Iterates over each class index in y and sets the corresponding position in the new tensor to 1.

compute_image_distance(cfg)

Compute the average image distance between source and target datasets using multiple models.

This method calculates the average image distance between features extracted from source and target image datasets using multiple pre-trained models. The distance is computed using a specified evaluation function for each model, and the average distance across all models is returned.

Parameters:

cfg (Dict) –

A configuration dictionary containing the following keys: - “DATA”: Dictionary with data-related configurations:

• ”source” (str): Path to the source folder containing images.

• ”target” (str): Path to the target folder containing images.

• ”batch_size” (int): The batch size for data loading.

• ”width” (int): The width of the images.

• ”height” (int): The height of the images.

• ”norm_mean” (list of float): Mean values for image normalization.

• ”norm_std” (list of float): Standard deviation values for image normalization.

• ”MODEL”: Dictionary with model-related configurations:

  • ”arch” (list of str): List of model architectures to use.

  • ”n_layer_feature” (int): The layer number from which to extract features.

  • ”device” (str): The device to run the models on (e.g., “cpu” or “cuda”).

• ”METHOD”: Dictionary with method-related configurations:

  • ”evaluator” (str): The evaluation function to use for computing the distance.

Returns:

The computed average image distance between the source and target datasets across all models.

Return type:

float

The method performs the following steps: 1. Constructs data loaders for the source and target datasets with specified transformations. 2. Iterates over each model specified in the configuration. 3. Loads each model and sets it up on the specified device. 4. Extracts features from the specified layer of the model for both datasets. 5. Computes the combined features and labels for the source and target datasets. 6. Calculates the distance using the specified evaluation function. 7. Returns the average distance across all models.

function_pad(x, y, error_metric)

Computes the PAD (Presentation Attack Detection) value using SVM classifier.

Parameters:

• x (np.ndarray) – Training features.

• y (np.ndarray) – Training labels.

• x_test (np.ndarray) – Test features.

• y_test (np.ndarray) – Test labels.

Returns:

A dictionary containing PAD either using MSE or MAE metric.

Return type:

dict

class dqm.domain_gap.metrics.RMSELoss(eps=0)

Bases: Module

Compute the Root Mean Squared Error (RMSE) loss between the predicted values and the target values.

This class provides a PyTorch module for calculating the RMSE loss, which is a common metric for evaluating the accuracy of regression models. The RMSE is the square root of the average of squared differences between predicted values and target values.

mse

Mean Squared Error loss module with reduction set to “sum”.

Type:

nn.MSELoss

eps

A small value added to the loss to prevent division by zero and ensure numerical stability.

Type:

float

forward(yhat, y)

Compute the RMSE loss between the predicted values yhat and the target values y.

forward(yhat, y)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class dqm.domain_gap.metrics.Wasserstein

Bases: object

compute_1D_distance(cfg)

Compute the average 1D Wasserstein Distance between corresponding features from source and target datasets.

This method calculates the average 1D Wasserstein Distance between features extracted from source and target image datasets using a pre-trained model. The features are extracted from a specified layer of the model, and the distance is computed for each corresponding feature dimension.

Parameters:

cfg (dict) –

A configuration dictionary containing the following keys: - “MODEL”: Dictionary with model-related configurations:

• ”arch” (str): The architecture of the model to use.

• ”device” (str): The device to run the model on (e.g., “cpu” or “cuda”).

• ”n_layer_feature” (int): The layer number from which to extract features.

• ”DATA”: Dictionary with data-related configurations:

  • ”width” (int): The width of the images.

  • ”height” (int): The height of the images.

  • ”norm_mean” (list of float): Mean values for image normalization.

  • ”norm_std” (list of float): Standard deviation values for image normalization.

  • ”batch_size” (int): The batch size for data loading.

  • ”source” (str): Path to the source folder containing images.

  • ”target” (str): Path to the target folder containing images.

Returns:

The computed average 1D Wasserstein Distance between the source and target image features.

Return type:

float

The method performs the following steps: 1. Loads the model and sets it up on the specified device. 2. Constructs data loaders for the source and target datasets with specified transformations. 3. Extracts features from the specified layer of the model for both datasets. 4. Computes the 1D Wasserstein Distance for each corresponding feature dimension. 5. Returns the average distance across all feature dimensions.

compute_cov_matrix(tensor)

Compute the covariance matrix of a given tensor.

This method calculates the covariance matrix for a given tensor, which represents a set of feature vectors. The covariance matrix provides a measure of how much the dimensions of the feature vectors vary from the mean with respect to each other.

Parameters:

tensor (torch.Tensor) – A 2D tensor where each row represents a feature vector. The tensor should have shape (N, D), where N is the number of samples and D is the dimensionality of the features.

Returns:

The computed covariance matrix of the feature vectors, with shape (D, D).

Return type:

torch.Tensor

The method performs the following steps: 1. Computes the mean vector of the feature vectors. 2. Centers the feature vectors by subtracting the mean vector. 3. Computes the covariance matrix using the centered feature vectors.

compute_slice_wasserstein_distance(cfg)

Compute the Sliced Wasserstein Distance between two sets of image features.

This method calculates the Sliced Wasserstein Distance between features extracted from source and target image datasets using a pre-trained model. The features are projected onto a lower-dimensional space using the eigenvectors corresponding to the largest eigenvalues of the covariance matrix. The distance is then computed between these projections.

Parameters:

cfg (dict) –

A configuration dictionary containing the following keys: - “MODEL”: Dictionary with model-related configurations:

• ”arch” (str): The architecture of the model to use.

• ”device” (str): The device to run the model on (e.g., “cpu” or “cuda”).

• ”n_layer_feature” (int): The layer number from which to extract features.

• ”DATA”: Dictionary with data-related configurations:

  • ”width” (int): The width of the images.

  • ”height” (int): The height of the images.

  • ”norm_mean” (list of float): Mean values for image normalization.

  • ”norm_std” (list of float): Standard deviation values for image normalization.

  • ”batch_size” (int): The batch size for data loading.

  • ”source” (str): Path to the source folder containing images.

  • ”target” (str): Path to the target folder containing images.

Returns:

The computed Sliced Wasserstein Distance between the source and target image features.

Return type:

float

The method performs the following steps: 1. Loads the model and sets it up on the specified device. 2. Constructs data loaders for the source and target datasets with specified transformations. 3. Extracts features from the specified layer of the model for both datasets. 4. Concatenates the features and computes the covariance matrix. 5. Computes the eigenvalues and eigenvectors of the covariance matrix. 6. Projects the features onto a lower-dimensional space using the eigenvectors. 7. Computes the Sliced Wasserstein Distance between the projected features.

dqm.domain_gap.utils module

Domain Gap Metric Calculation Script

This script defines a PyTorch module, DomainMeter, for domain adaptation using Central Moment Discrepancy (CMD) and Kullback-Leibler (KL) divergence. The script also includes custom dataset classes (RMSELoss and PandasDatasets) for loading images from a Pandas DataFrame and implementing a custom root mean square error (RMSE) loss.

Authors:

Sabrina CHAOUCHE Yoann RANDON Faouzi ADJED

Classes:

ModelConfiguration RMSELoss PandasDatasets DomainMeter

Dependencies:

torch torchvision mlflow cmd (DomainMeter class from cmd module) twe_logger (Custom logging module)

Usage: Run the script with optional arguments ‘–cfg’ for the JSON config file path and ‘–tsk’ for the JSON task config file path.

dqm.domain_gap.utils.compute_features(dataloader, model, device)

Compute features from a model for images in the DataLoader batch by batch.

Parameters:

• dataloader (DataLoader) – DataLoader object to load images in batches.

• model (torch.nn.Module) – Pre-trained model to extract features.

• device (torch.device) – Device to run the model (e.g., CPU or GPU).

Returns:

A concatenated tensor of features for all images.

Return type:

torch.Tensor

dqm.domain_gap.utils.construct_dataloader(folder_path, transform, batch_size)

Loads images from a folder and returns a DataLoader for batch-wise processing.

Parameters:

• folder_path (str) – Path to the folder containing images.

• transform (transform) – Transform object to fine-tune data for model input.

• batch_size (int) – Number of images per batch.

Returns:

A DataLoader object that yields batches of transformed images.

Return type:

DataLoader

dqm.domain_gap.utils.display_resume(cfg, dist, time_lapse)

dqm.domain_gap.utils.extract_nth_layer_feature(model, n)

dqm.domain_gap.utils.generate_transform(img_size, norm_mean, norm_std)

Generate transform to change data input into compatible model inputs

Parameters:

• image_size (Tuple[int, int]) – value to resize image

• norm_mean (List[float]) – normalization mean

• norm_std (List[float]) – normalization standard deviation

Returns:

a function which apply multiple changes to data

Return type:

transform

dqm.domain_gap.utils.load_config(config_file)

Load configuration from a JSON file.

dqm.domain_gap.utils.load_model(model_str, device)

Loads a model based on the input string.

If the string contains ‘.pt’ or ‘.pth’, tries to load a saved PyTorch model from a file. If the string matches a known torchvision model (e.g., ‘resnet18’), it loads the corresponding model.

Parameters: model_str (str): The model string or file path.

Returns: model (torch.nn.Module): The loaded PyTorch model.

Return type:

Module

Module contents