uqmodels.postprocessing package

Submodules

uqmodels.postprocessing.UQKPI_Processor module

class uqmodels.postprocessing.UQKPI_Processor.Anomscore_processor(name='Anomscore', KPI_parameters={}, cache=None, **kwargs)

Bases: UQKPI_Processor

Processor aiming to produce an contextual deviation anomaly score from UQmeasure prediction and observation

fit(UQ, type_UQ, pred, y=None, type_UQ_params=None, ctx_mask=None, **kwargs)

Fitting procedure aim to estimate and store Anomscore_processor params for Anomscore computation

Parameters:

• UQ (np.array) – UQmeasure provide by the UQestiamtor.

• type_UQ (str) – Type_UQ of the UQestimators.

• pred (np.array) – Prediction of the predictor or the UQestimator.

• y (np.array, optional) – Targets/Observations, can be None if processor does’t need y to fit

• type_UQ_params (type_UQ_params, optional) – Additional information about type_UQ parameters. Defaults to None.

transform(UQ, type_UQ, pred, y=None, type_UQ_params=None, ctx_mask=None, **kwargs)

Transform procedure aim to transform (predictor) & UQestimator output into a Epistemic_scorelvl according Episticscore quantile fitted during training procedure.

Parameters:

• UQ (np.array) – UQmeasure provide by the UQestimator.

• type_UQ (str) – Type_UQ of the UQestimators.

• pred (np.array) – Prediction of the predictor or the UQestimator.

• y (np.array, optional) – Targets/Observations, can be None if processor does’t need y to fit

• type_UQ_params (type_UQ_params, optional) – Additional information about type_UQ parameters. Defaults to None.

Returns: anom_score : Deviation score (pred-y) process by anom_proc.compute_anom_score

class uqmodels.postprocessing.UQKPI_Processor.Epistemicscorelvl_processor(name='Epistemicscorelvl', KPI_parameters={}, cache=None, **kwargs)

Bases: UQKPI_Processor

Processor aiming to transform UQmeasure (containing epistemic measure) into Epistemic_score_lvl (ADD link)

fit(UQ, type_UQ, pred, y=None, type_UQ_params=None, **kwargs)

Fitting procedure aim to estimate and store KPI_processor params for Epistemicscorelvl computation

Parameters:

• UQ (np.array) – UQmeasure provide by the UQestiamtor.

• type_UQ (str) – Type_UQ of the UQestimators.

• pred (np.array) – Prediction of the predictor or the UQestimator.

• y (np.array, optional) – Targets/Observations, can be None if processor does’t need y to fit

• type_UQ_params (type_UQ_params, optional) – Additional information about type_UQ parameters. Defaults to None.

transform(UQ, type_UQ, pred, y=None, type_UQ_params=None, **kwargs)

Transform procedure aim to transform (predictor) & UQestimator output into a Epistemic_scorelvl according Episticscore quantile fitted during training procedure.

Parameters:

• UQ (np.array) – UQmeasure provide by the UQestiamtor.

• type_UQ (str) – Type_UQ of the UQestimators.

• pred (np.array) – Prediction of the predictor or the UQestimator.

• y (np.array, optional) – Targets/Observations, can be None if processor does’t need y to fit

• type_UQ_params (type_UQ_params, optional) – Additional information about type_UQ parameters. Defaults to None.

Returns: Epistemic_scorelvlEpistemic_scorelvl that express the quantile class of Epistemic values

amoung quantile [0.50, 0.80, 0.90, 0.95, 0.975, 0.99, 0.999]

class uqmodels.postprocessing.UQKPI_Processor.NormalPIs_processor(name='Normal_PIs_processor', KPI_parameters={'list_alpha': [0.05, 0.95], 'with_epistemic': True}, cache=None, **kwargs)

Bases: UQKPI_Processor

Processor aiming to transform UQmeasure in Normal predictive intervals

fit(UQ, type_UQ, pred, y=None, type_UQ_params=None, **kwargs)

fitting procedure aim to estimate and store KPI_processor params for NormalPIs computation :param UQ: UQmeasure provide by the UQestiamtor. :type UQ: np.array :param type_UQ: Type_UQ of the UQestimators. :type type_UQ: str :param pred: Prediction of the predictor or the UQestimator. :type pred: np.array :param y: Targets/Observations, can be None if processor does’t need y to fit :type y: np.array, optional :param type_UQ_params: Additional information about type_UQ parameters.

Defaults to None.

transform(UQ, type_UQ, pred, y=None, type_UQ_params=None, **kwargs)

Transform procedure aim to transform (predictor) & UQestimator output into normal predictive intervales according to the list_alpha stored parameters.

Parameters:

• UQ (np.array) – UQmeasure provide by the UQestiamtor.

• type_UQ (str) – Type_UQ of the UQestimators.

• pred (np.array) – Prediction of the predictor or the UQestimator.

• y (np.array, optional) – Targets/Observations, can be None if processor does’t need y to fit

• type_UQ_params (type_UQ_params, optional) – Additional information about type_UQ parameters. Defaults to None.

Returns:

Quantiles parametrized by list_alpha provided in the init procedure

Return type:

list_PIs

class uqmodels.postprocessing.UQKPI_Processor.UQKPI_Processor(name='UQ_processor', KPI_parameters={}, cache=<uqmodels.processing.Cache_manager object>, random_state=None, **kwargs)

Bases: Processor

KPI processor that aim to process (predictor) & UQestimator output into a KPI.

fit(UQ=None, type_UQ=None, pred=None, y=None, type_UQ_params=None, **kwargs)

fitting procedure aim to estimate and store KPI_processor params

Parameters:

• UQ (np.array) – UQmeasure provide by the UQestiamtor.

• type_UQ (str) – Type_UQ of the UQestimators.

• pred (np.array) – Prediction of the predictor or the UQestimator.

• y (np.array, optional) – Targets/Observations, can be None if processor does’t need y to fit

• type_UQ_params (type_UQ_params, optional) – Additional information about type_UQ parameters. Defaults to None.

transform(UQ=None, type_UQ=None, pred=None, y=None, type_UQ_params=None, **kwargs)

Transform procedure aim to transform (predictor) & UQestimator output into a KPI using fitted parameters.

by default return row UQ provided by UQEstimators.

Parameters:

• UQ (np.array) – UQmeasure provide by the UQestiamtor.

• type_UQ (str) – Type_UQ of the UQestimators.

• pred (np.array) – Prediction of the predictor or the UQestimator.

• y (np.array, optional) – Targets/Observations, can be None if processor does’t need y to fit

• type_UQ_params (type_UQ_params, optional) – Additional information about type_UQ parameters. Defaults to None.

uqmodels.postprocessing.UQ_processing module

uqmodels.postprocessing.UQ_processing.check_UQ(UQ, type_UQ, type_UQ_params)

Check if UQ and type_UQ are compatible. :param UQ: UQ object :type UQ: _type_ :param type_UQ: type_UQ specification :type type_UQ: _type_

uqmodels.postprocessing.UQ_processing.check_y_vs_pred_and_UQ_shape(y, pred, UQ=None)

Check if y, pred and UQ have compatible shape

Parameters:

• y (_type_) – _description_

• pred (_type_) – _description_

• UQ (_type_) – _description_

Returns:

Reshaped if needed

Return type:

y,pred,UQ

uqmodels.postprocessing.UQ_processing.compute_Epistemic_score(UQ, type_UQ, pred=None, y=None, type_UQ_params=None, list_percent=[0.8, 0.9, 0.99, 0.999, 1], var_min=0, var_max=None, min_cut=0, max_cut=1, q_var=1, q_Eratio=3, mode='levels', params_=None, reduc_filter=None, **kwargs)

Function that compute Epistemic_score_lvl from (predictor) & UQEstimor outputs & fitted parameters provided by fit_Epistemic_score

Parameters:

• pred (_type_) – Mean prediction

• UQ (_type_) – UQmeasure

• type_UQ (_type_) – UQmeasure hypothesis

• list_percent (list, optional) – Quantile values. Defaults to [0.025, 0.975].

• type_UQ_params (_type_, optional) – UQmeasure params. Defaults to None.

• params (_type_, optional) – May use externally fit_Epistemic_score to obtain it. If None, internal called to fit

Returns:

Epistemic_score_lvl : Quantile class values of Epistemic score params : Parameters provided or computed

Return type:

Epistemic_scorelvl

uqmodels.postprocessing.UQ_processing.compute_PI(UQ, type_UQ, pred, y=None, type_UQ_params=None, list_alpha=[0.025, 0.975], var_min=0, var_max=None, min_cut=0, max_cut=1, q_var=1, params_=None, reduc_filter=None, **kwargs)

Compute according to normal assumption a list of quantile parametrized by list alpha from the UQmeasre

Parameters:

• pred (_type_) – Mean prediction

• UQ (_type_) – UQmeasure

• type_UQ (_type_) – UQmeasure hypothesis

• list_alpha (list, optional) – Quantile values. Defaults to [0.025, 0.975].

• type_UQ_params (_type_, optional) – UQmeasure params. Defaults to None.

• params (_type_, optional) – params : May use externally fit_PIs to obtain it. If None, fit_PIs is used internally.

Raises:

ValueError – type_UQ not covered

Returns:

List of computed quantiles params : Params provided or computed

Return type:

list_PIs

uqmodels.postprocessing.UQ_processing.fit_Epistemic_score(UQ, type_UQ, pred=None, y=None, type_UQ_params=None, list_percent=[0.5, 0.8, 0.95, 0.98, 0.995, 1], var_min=0, var_max=None, min_cut=0.1, max_cut=0.97, q_var=1, q_Eratio=3, mode='score', reduc_filter=None, **kwargs)

Function that estimate parameters link to Epistemic_score_normalisation based on quantile lvl

Parameters:

• pred (_type_) – Mean prediction

• UQ (_type_) – UQmeasure

• type_UQ (_type_) – UQmeasure hypothesis

• list_percent (list, optional) – Quantile values to normalise. Defaults to [0.50, 0.80, 0.90, 0.95, 0.975, 0.99, 0.999].

• type_UQ_params (_type_, optional) – UQmeasure params. Defaults to None.

Returns:

Params needed by compute_Epistemic_score

Return type:

params

uqmodels.postprocessing.UQ_processing.fit_PI(UQ, type_UQ, pred, y=None, list_alpha=[0.025, 0.975], type_UQ_params=None, reduc_filter=None, **kwargs)

Function that estimate PIs parameters according to normal assumption a list of quantile parametrized by list alpha from the UQmeasure

Parameters:

• pred (_type_) – Mean prediction

• UQ (_type_) – UQmeasure

• type_UQ (_type_) – UQmeasure hypothesis

• list_alpha (list, optional) – Quantile values. Defaults to [0.025, 0.975].

• type_UQ_params (_type_, optional) – UQmeasure params. Defaults to None.

• PIs_params (_type_, optional) – PIs_params : May use externally fit_PIs to obtain it. If None, fit_PIs is used internally.

Raises:

ValueError – type_UQ not covered

Returns:

List of computed quantiles

Return type:

list_PIs

uqmodels.postprocessing.UQ_processing.get_extremum_var(var, min_cut=0, max_cut=1, var_min=0, var_max=None, factor=1, mode_multidim=True)

uqmodels.postprocessing.UQ_processing.get_extremum_var_TOT_and_ndUQ_ratio(UQ, type_UQ, pred=None, y=None, type_UQ_params=None, min_cut=0, max_cut=1, var_min=0, var_max=None, factor=2, q_var=1, q_Eratio=2, mode_multidim=True, E_cut_in_var_nominal=True, A_res_in_var_atypic=False, **kwargs_process_TOT)

Estimate parameters use to seperate from var_A & var_E theirs

Respective nominal part (Affected in TOT,ATYPIC) et atypical part using empirical threeshold

Parameters:

• UQ (_type_) – _description_

• type_UQ (_type_, optional) – _description_. Defaults to None.

• pred (_type_, optional) – _description_. Defaults to None.

• y (_type_, optional) – _description_. Defaults to None.

• type_UQ_params (_type_, optional) – _description_. Defaults to None.

• min_cut (int, optional) – _description_. Defaults to 0.

• max_cut (int, optional) – _description_. Defaults to 1.

• var_min (int, optional) – _description_. Defaults to 0.

• var_max (_type_, optional) – _description_. Defaults to None.

• factor (int, optional) – _description_. Defaults to 2.

• q_var (int, optional) – _description_. Defaults to 1.

• q_Eratio (int, optional) – _description_. Defaults to 2.

• mode_multidim (bool, optional) – _description_. Defaults to False.

Returns:

_description_

Return type:

_type_

uqmodels.postprocessing.UQ_processing.get_nominal_disentangled_UQ_ratio(UQ, q_var=1, q_Eratio=2)

Compute nominal_disentangled_UQ_ration form set of data and k_var_e

Parameters:

• UQ (_type_) – _description_

• k_var_e (_type_) – nominal considerate ratio (high-pass filters)

Returns:

nominal disentangled_UQ_ratio

Return type:

ndUQ_ratio

uqmodels.postprocessing.UQ_processing.norm_var(var, min_cut=0, max_cut=1, var_min=0, var_max=None, q_var=1, epsilon=1e-08)

variance normalisation :param var: variance to normalise :type var: np.array :param min_cut: Bottom extremun percentile : [0,1]. :type min_cut: float :param max_cut: Bottom upper percentile: [0,1]. :type max_cut: float :param var_min: minimal variance assumption. Defaults to 0. :type var_min: float, optional :param q_var: Power coefficent :type q_var: float

Returns:

normalised variance

Return type:

var

uqmodels.postprocessing.UQ_processing.process_UQmeasure_to_Epistemicscore(UQ, type_UQ, pred=None, y=None, type_UQ_params=None, var_min=0, var_max=None, min_cut=0, max_cut=1, q_var=1, q_var_e=None, k_var_e=1, mode='relative_likelihood', reduc_filter=None, roll=0, **kwargs)

Process UQ measure to epistemicvalues according UQmeasure

Parameters:

• UQ (np.array or list) – UQ measure obtain from UQ-estimator

• type_UQ (_type_) – Type UQ that caracterise UQ measre

• pred (np.array) – prediction unused

• y (np.array) – Targets/Observation unused

• type_UQ_params – additional parameters unused

Returns:

Epistemic unconfidence score.

Return type:

Eval

uqmodels.postprocessing.UQ_processing.process_UQmeasure_to_TOT_and_E_sigma(UQ, type_UQ, pred=None, y=None, type_UQ_params=None, var_min=0, var_max=None, min_cut=0, max_cut=1, q_var=1, q_var_e=1, k_var_e=1, q_Eratio=None, ndUQ_ratio=None, extremum_var_TOT=(None, None), reduc_filter=None, roll=0, **kwargs)

Process UQ measure into sigma_tot & sigma_E (sqrt(varirance)) according prediction, UQmeasure and observation.

Parameters:

• UQ (np.array or list) – UQmeasure obtain from UQEstimator

• type_UQ (_type_) – Type UQ that the nature of UQmeasure

• pred (np.array) – prediction provide by a predictor or an UQEstimator

• y (np.array) – Targets/Observation

• type_UQ_params – additional parameters link to type paradigm (ex : alpha for quantile)

• min_cut (_type_) – Bottom extremun percentile.

• max_cut (_type_) – Bottom upper percentile.

• q_var (_type_) – Power coefficent

• sigma_min (float, optional) – Minimum values of UQ considered.

Returns:

standards deviation estimation link to ML-uncertainty

Return type:

sigma

uqmodels.postprocessing.UQ_processing.process_UQmeasure_to_quantile(UQ, type_UQ, pred, y=None, type_UQ_params=None, alpha=0.05, var_min=0, var_max=None, min_cut=0, max_cut=1, q_var=1, reduc_filter=None, roll=0)

Process UQ measure into gaussian_quantile according prediction, UQmeasure and observation and a alpha quantile lvl

Parameters:

• UQ (np.array or list) – UQmeasure obtain from UQEstimator

• type_UQ (_type_) – Type UQ that the nature of UQmeasure

• pred (np.array) – prediction provide by a predictor or an UQEstimator

• y (np.array) – Targets/Observation

• type_UQ_params – additional parameters link to type paradigm (ex : alpha for quantile)

• min_cut (_type_) – Bottom extremun percentile.

• max_cut (_type_) – Bottom upper percentile.

• q_var (_type_) – Power coefficent

• sigma_min (float, optional) – Minimum values of UQ considered.

Returns:

alpha quantile lvl based on gaussian assumption

Return type:

gaussian_quantile

uqmodels.postprocessing.UQ_processing.process_UQmeasure_to_residu(UQ, type_UQ, pred, y, type_UQ_params=None, d=2, min_cut=0, max_cut=1, q_var=1, var_min=0, var_max=None, with_born=False, k_var_e=0, q_var_e=None, q_Eratio=None, extremum_var_TOT=(None, None), ndUQ_ratio=None, reduc_filter=None, roll=0, debug=False)

Process UQ measure to residu according prediction, UQmeasure and observation.

Parameters:

• UQ (np.array or list) – UQ measure obtain from UQ-estimator

• type_UQ (_type_) – Type UQ that caracterise UQ measre

• pred (np.array) – prediction

• y (np.array) – Targets/Observation

• type_UQ_params – additional parameters link to type paradigm (ex : alpha for quantile)

• min_cut (_type_) – Bottom extremun percentile.

• max_cut (_type_) – Bottom upper percentile.

• min_cut – Bottom extremun percentile.

• max_cut – Bottom upper percentile.

• q_var (_type_) – Power coefficent

• sigma_min (float, optional) – Minimum values of UQ considered.

Returns:

residu

Return type:

res

uqmodels.postprocessing.UQ_processing.process_UQmeasure_to_sigma(UQ, type_UQ, pred=None, y=None, type_UQ_params=None, var_min=0, var_max=None, min_cut=0, max_cut=1, q_var=1, reduc_filter=None)

Process UQ measure into sigma (sqrt(varirance)) according prediction, UQmeasure and observation.

Parameters:

• UQ (np.array or list) – UQmeasure obtain from UQEstimator

• type_UQ (_type_) – Type UQ that the nature of UQmeasure

• pred (np.array) – prediction provide by a predictor or an UQEstimator

• y (np.array) – Targets/Observation

• type_UQ_params – additional parameters link to type paradigm (ex : alpha for quantile)

• min_cut (_type_) – Bottom extremun percentile.

• max_cut (_type_) – Bottom upper percentile.

• q_var (_type_) – Power coefficent

• sigma_min (float, optional) – Minimum values of UQ considered.

Returns:

standards deviation estimation link to ML-uncertainty

Return type:

sigma

uqmodels.postprocessing.UQ_processing.renormalise_UQ(UQ, type_UQ, scaler=None, var_min=0, var_max=None)

UQmeasure to normalised

Parameters:

• UQ (np.array) – UQmeasure provided by an UQEstimator

• type_UQ (_type_) – nature of the UQeeasure

• rescale_val (np.array) – rescale variance term (may be obtain by scikilearn Standard normalizer)

• var_min (float, optional) – minimal variance assumption. Defaults to 0.

Returns:

normalised UQmeasure

Return type:

UQ

uqmodels.postprocessing.UQ_processing.split_var_dUQ(UQ, q_var=1, q_var_e=1, ndUQ_ratio=None, extremum_var_TOT=(None, None), E_cut_in_var_nominal=True, A_res_in_var_atypic=False)

split var_A&E into var_nominal & var_atypique based threshold

Parameters:

• UQ (_type_) – _description_

• q_var (int, optional) – _description_. Defaults to 1.

• k_var_e (int, optional) – _description_. Defaults to 1.

• q_var_e (int, optional) – _description_. Defaults to 1.

• ndUQ_ratio (int, optional) – _description_. Defaults to 0.

• extremum_var_TOT (tuple, optional) – _description_. Defaults to (None, None).

Returns:

_description_

Return type:

_type_

uqmodels.postprocessing.anomaly_processing module

uqmodels.postprocessing.anomaly_processing.compute_anom_score(UQ, type_UQ, pred, y, type_UQ_params=None, ctx_mask=None, beta=0.005, per_seuil=0.9995, min_cut=0.01, max_cut=0.97, var_min=0, var_max=None, d=2, type_norm='quantiles_local', empiric_rescale=False, global_median_normalization=False, filt=None, reduc_filter=None, q_var=1, q_var_e=1, q_Eratio=3, k_var_e=1, with_born=False, params_=None, debug=False, roll=0, **kwargs)

Compute contextual deviation score from observation, Prediction and UQ measure.

Then apply normalise considering threeshold based on beta miss_covered target

Parameters:

• UQ (UQ-measure) – UQ measure from UQ estimateurs

• type_UQ (str, optional) – _description_. Defaults to “sigma”.

• pred (array) – prediction

• y (array) – Real values

• type_UQ_params (str, optional) – _description_. Defaults to “sigma”.

• beta (float, optional) – Anom target Defaults to 0.05.

• per_seuil (float, optional) – Threshold Defaults to 0.995.

• min_cut (float, optional) – extremum cut Defaults to 0.005.

• max_cut (float, optional) – extremum cut Defaults to 0.995.

• var_min (float, optional) – Threshold of minimum var, Default to 0

• d (int, optional) – residual power Defaults to 2.

• type_norm (str, optional) – Type of normalisation see norm Defaults to “quantiles_local”.

• empiric_rescale (bool,optional) – Force empiric rescale based on train percentile

• global_median_normalization (bool,optional) – Normalisation of residu by each step by the median residu of all dimension.

• filt (list, optional) – fitler applied to the score . Defaults to [0, 1, 0].

• q_var (int, optional) – power coefficent apply to the score. Defaults to 1.

• params – params provide by fit function. Defaults to None imply internal call to fit

Type_norm :

“Nsigma_local”Fexible anomalie thresold (N-sgima) by dimension | Hypothesis :

Anomaly distributed homegeneously by dimension.

“Nsigma_global” : Fexible anomalie thresold (N-sgima) global : homogeneous anomaly distribution by dimension. “Chi2”: Theorical Chi2 correction “quantiles_local” : rigid anomalie thresold (quantiles) by dimension :

heterogeneos anomaly distrubtion by dimension.

“quantiles_global”(Default) rigid anomalie thresold (quantiles) global :

homogeneous anomaly distribution by dimension.

“None” : No normalisation

Returns:

2D anomaly score matrix params : parameters provided or computed

Return type:

anom_score

uqmodels.postprocessing.anomaly_processing.compute_born_calibrated(UQ, type_UQ, pred, y=None, type_UQ_params=None, beta=0.1, type_res='res', ctx_mask=None, params_=None)

!!!!Depreciated !!! Compute_UQ_calibration from UQ measure, pred, observation and target

Parameters:

• UQ (np.array or list) – UQmeasure obtain from UQEstimator

• type_UQ (str) – Type UQ that the nature of UQmeasure

• pred (np.array) – prediction provide by a predictor or an UQEstimator

• y (np.array) – Targets/Observation

• type_UQ_params (_type_, optional) – additional parameters link to type paradigm (ex : alpha for quantile)

• beta (float, optional) – target miss-coverage of the borns

• type_res – “no_calib : No calibration “res” : Calibration based on mean residuals “w_res” : Calibration based on weigthed mean residuals “cqr” : Calibration based on quantile residuals

• ctx_mask (_type_, optional) – _description_. Defaults to None.

• params (_type_, optional) – _description_. Defaults to None.

Raises:

ValueError – _description_

Returns:

(upper & lower calibrated bound) & compute params

Return type:

(y_pred_lower, y_pred_upper), params

uqmodels.postprocessing.anomaly_processing.compute_calibrate(residuals, ctx_mask=None, beta=0.05, type_norm=None, empiric_rescale=True, deg=1, params_=None, mode='score', debug=False)

Function that apply calibration on residu based on some hypothesis

Parameters:

• res (_type_) – residu to calibrate

• ctx_mask (None) – mask for ctx calib

• beta (_type_) – Anomaly target ratio

• type_norm (_type_, optional) – assymption

• empiric_rescale (bool, optional) – _description_. Defaults to True.

• deg (int, optional) – Power to apply to the residu. Defaults to 1.

type_norm assymption :

“Nsigma_local”Fexible anomalie thresold (N-sgima) by dimension | Hypothesis :

Anomaly distributed homegeneously by dimension.

“Nsigma_global” : Fexible anomalie thresold (N-sgima) global : homogeneous anomaly distribution by dimension. “Chi2”: Theorical Chi2 correction “quantiles_local” : rigid anomalie thresold (quantiles) by dimension :

heterogeneos anomaly distrubtion by dimension.

“quantiles_global”(Default) rigid anomalie thresold (quantiles) global :

homogeneous anomaly distribution by dimension.

“None” : No normalisation

Returns:

calibrated residu params: Params provided or computed

Return type:

res

uqmodels.postprocessing.anomaly_processing.compute_score_fusion(score, ctx_mask=None, type_fusion='mahalanobis', type_norm='quantiles_global', beta=0.05, per_seuil=0.995, filt=[0, 1, 0], d=1, fusion_reduc_filter=None, params_=None, fusion_debug=False, **kwargs)

Compute and aggregate and 2D anomaly score matrix into an aggregated 1D

Parameters:

• score (_type_) – _description_

• ctx_mask (_type_) – _description_

• type_fusion (str, optional) – mahalanobis or mean. Defaults to “mahalanobis”.

• type_norm (str, optional) – _description_. Defaults to “quantiles_global”.

• beta (float, optional) – anom ratio target. Defaults to 0.95.

• per_seuil (float, optional) – _description_. Defaults to 0.995.

• filt (list, optional) – _description_. Defaults to [0, 1, 0].

• d (int, optional) – _description_. Defaults to 2.

• list_norm_val (_type_, optional) – _description_. Defaults to None.

• list_ctx_val (_type_, optional) – _description_. Defaults to None.

• params – params provided by fit function. Defaults to None imply interal call to fit

Returns:

1D anomaly score matrix

Return type:

score_agg

uqmodels.postprocessing.anomaly_processing.fit_anom_score(UQ, type_UQ, pred, y, type_UQ_params=None, ctx_mask=None, beta=0.01, per_seuil=0.9995, min_cut=0.01, max_cut=0.97, var_min=0, d=2, q_var=1, k_var_e=1, q_var_e=1, q_Eratio=3, type_norm='quantiles_local', empiric_rescale=False, global_median_normalization=False, filt=None, reduc_filter=None, roll=0, debug=False, **kwargs)

Fit parameters link to anomaly score calibration using Prediction and UQ measure.

Parameters:

• UQ (UQ-measure) – UQ measure from UQ estimateurs

• type_UQ (str, optional) – _description_. Defaults to “sigma”.

• pred (array) – prediction

• y (array) – Real values

• type_UQ_params (str, optional) – additional information about UQ. Defaults to None.

• ctx_mask – mask for ctx_calib

• beta (float, optional) – Anom target Defaults to 0.05.

• per_seuil (float, optional) – Threshold Defaults to 0.995.

• min_cut (float, optional) – extremum cut Defaults to 0.005.

• max_cut (float, optional) – extremum cut Defaults to 0.995.

• var_min (float, optional) – Threshold of minimum var, Default to 0

• d (int, optional) – residual power Defaults to 2.

• type_norm (str, optional) – Type of normalisation see Defaults to “quantiles_local”.

• global_median_normalization (bool,optional) – Normalisation of residu by each step by the median residu of all dimension.

• filt (list, optional) – _description_. Defaults to [0, 1, 0].

• q_var (int, optional) – _description_. Defaults to 1.

Returns:

parameters to calibrate anomaly score

Return type:

params

uqmodels.postprocessing.anomaly_processing.fit_born_calibrated(UQ, type_UQ, pred, y, type_UQ_params=None, beta=0.1, type_res='res', ctx_mask=None, var_min=0, min_cut=0, max_cut=0, q_var=1, empiric_rescale=True)

!!!!Depreciated !!! Estimate calibration parameters in order to calibrate born from UQ measure, pred, observation and target

Parameters:

• UQ (np.array or list) – UQmeasure obtain from UQEstimator

• type_UQ (_type_) – Type UQ that the nature of UQmeasure

• pred (np.array) – prediction provide by a predictor or an UQEstimator

• y (np.array) – Targets/Observation

• type_UQ_params – additional parameters link to type paradigm (ex : alpha for quantile)

• beta – target miss-coverage of the borns

• min_cut (_type_) – Bottom extremun percentile.

• max_cut (_type_) – Bottom upper percentile.

• q_var (_type_) – Power coefficent

• sigma_min (float, optional) – Minimum values of UQ considered.

Returns:

list of calibration term for each ctx_values,

list_ctx_id : list of ctx_values)

Return type:

params = (list_norm_val,

uqmodels.postprocessing.anomaly_processing.fit_calibrate(residuals, ctx_mask=None, beta=0.05, type_norm=None, empiric_rescale=False, deg=1, d=1, debug=False)

Function that estimate calibrate parameters form residu link to some hypothesis

Parameters:

• s (_type_) – score

• ctx_mask – mask for ctx calib

• beta (_type_) – Anomaly target ratio

• type_norm (_type_, optional) – assymption

• empiric_rescale (bool, optional) – _description_. Defaults to True.

• deg (int, optional) – _description_. Defaults to 1.

Type_norm :

“Nsigma_local”Fexible anomalie thresold (N-sgima) by dimension | Hypothesis :

Anomaly distributed homegeneously by dimension.

“Nsigma_global” : Fexible anomalie thresold (N-sgima) global : homogeneous anomaly distribution by dimension. “Chi2”: Theorical Chi2 correction “quantiles_local” : rigid anomalie thresold (quantiles) by dimension :

heterogeneos anomaly distrubtion by dimension.

“quantiles_global”(Default) rigid anomalie thresold (quantiles) global :

homogeneous anomaly distribution by dimension.

“None” : No normalisation

Returns:

(list_norm_valcalibration coeficient for each ctx_values

list_ctx_id : list of ctx_values)

Return type:

params

uqmodels.postprocessing.anomaly_processing.fit_score_fusion(score, ctx_mask=None, type_fusion='mahalanobis', type_norm='quantiles_global', beta=0.05, per_seuil=0.995, filt=[0, 1, 0], fusion_reduc_filter=None, fusion_debug=False, **kwargs)

uqmodels.postprocessing.anomaly_processing.score_seuil(s, per_seuil=0.995, local=True)

Thresholding of extrem values by percentile by dimension if local = True or globally else.

Parameters:

• s (_type_) – _description_

• per_seuil (float, optional) – _description_. Defaults to 0.995.

• local (bool, optional) – _description_. Defaults to True.

Returns:

_description_

Return type:

_type_

uqmodels.postprocessing.custom_UQKPI_Processor module

class uqmodels.postprocessing.custom_UQKPI_Processor.Multiscale_Anomscore_processor(name='Multiscale_anomscore', KPI_parameters={}, cache=None, random_state=None, **kwargs)

Bases: UQKPI_Processor

Processor aiming to produce an contextual deviation anomaly score from UQmeasure prediction and observation

fit(list_UQ, type_UQ, list_pred, list_y, type_UQ_params=None, ctx_mask=None, **kwargs)

Fitting procedure aim to estimate and store Multiscale_Anomscore_processor params for Multiscale_Anomscore computation

Parameters:

• UQ (np.array) – UQmeasure provide by the UQestiamtor.

• type_UQ (str) – Type_UQ of the UQestimators.

• pred (np.array) – Prediction of the predictor or the UQestimator.

• y (np.array, optional) – Targets/Observations, can be None if processor does’t need y to fit

• type_UQ_params (type_UQ_params, optional) – Additional information about type_UQ parameters. Defaults to None.

transform(list_UQ, type_UQ, list_pred, list_y, type_UQ_params=None, ctx_mask=None, **kwargs)

Transform procedure aim to transform (predictor) & UQestimator output into a Multiscale_Anomscore according to Multiscale_Anomscore_params

Parameters:

• UQ (np.array) – UQmeasure provide by the UQestiamtor.

• type_UQ (str) – Type_UQ of the UQestimators.

• pred (np.array) – Prediction of the predictor or the UQestimator.

• y (np.array, optional) – Targets/Observations, can be None if processor does’t need y to fit

• type_UQ_params (type_UQ_params, optional) – Additional information about type_UQ parameters. Defaults to None.

Returns:

A tuple (S_anom_chan, S_anom_agg, S_anom_source, list_bot, list_top) where S_anom_chan is a mutli-dimensional providing Anom score for each channels of each source on compute_anom_score, S_anom_source is an aggregated score providing Anom score at source lvl on compute_score_fusion, S_anom_agg is an aggregated score providing 1D Anom score based on compute_score_fusion, list_bot is an anomalie lower threeshold for each chan of each sensors provided by compute_anom_score and list_top is an anomalie upper threeshold for each chan of each sensors provided by compute_anom_score

Module contents