Plots and reports

The snippets on this page start from this setup:

import os

import pandas as pd

from avatars.constants import PlotKind
from avatars.manager import Manager

manager = Manager(api_key=os.environ["AVATAR_API_KEY"])
df = pd.read_csv("fixtures/wbcd.csv")

runner = manager.create_runner("wbcd_reporting")
runner.add_table("wbcd", df)
runner.advise_parameters("wbcd")
runner.run()

How to render plots

Plots help you tune the parameters and understand the results. They compare the original data and the avatars at four levels:

  • Univariate: PlotKind.DISTRIBUTION (one plot per column) and PlotKind.AGGREGATE_STATS (mean and standard deviation of the first 10 columns).

  • Bivariate: PlotKind.CORRELATION and PlotKind.CORRELATION_DIFFERENCE.

  • Multivariate: PlotKind.PROJECTION_2D and PlotKind.PROJECTION_3D.

  • Data structure: PlotKind.CONTRIBUTION (contribution of each variable to the axes of the model).

runner.render_plot("wbcd", PlotKind.PROJECTION_2D)
runner.render_plot("wbcd", PlotKind.DISTRIBUTION)
runner.render_plot("wbcd", PlotKind.CORRELATION_DIFFERENCE)

Specific plots:

  • time series: PlotKind.RAW_SERIES, PlotKind.NORMALIZED_SERIES;

  • class rebalancing: PlotKind.CLASS_PROJECTION_2D;

  • multi-table: PlotKind.METRICS_SUMMARY.

If the plot does not show in your notebook (for example in VS Code), or if you want an HTML file, use open_in_browser=True. The plot is saved as an HTML file and opened in your browser:

runner.render_plot("wbcd", PlotKind.PROJECTION_3D, open_in_browser=True)

In the web app: results page → Projections.

How to download the reports

A run produces two reports:

  • the technical report (PDF) quantifies the re-identification risk and the utility, with each metric next to its target;

  • the PIA report (DOCX, so that you can edit it) is the input for your Privacy Impact Assessment. It evaluates how likely an attack is and what an attacker would need, and helps you decide if the residual risk is compatible with the intended use of the data.

from avatar_yaml.models.parameters import ReportType

runner.download_report("technical_report.pdf")
runner.download_report("pia_report.docx", report_type=ReportType.PIA)

For a multi-table run, there is one PIA report for each table, with the prefixes 0_, 1_, …

In the web app: results page → Technical Assessment / Impact Assessment.

How to describe the context for the PIA report

The context of the processing sets the targets of the metrics and fills the PIA report. Set it when you create the runner:

from avatar_yaml.models.avatar_metadata import (
    DataRecipient,
    DataSubject,
    DataType,
    SensitivityLevel,
)

runner = manager.create_runner(
    set_name="wbcd_pia_report",
    pia_data_recipient=DataRecipient.TRUSTED_THIRDPARTY,
    pia_data_type=DataType.HEALTH,
    pia_data_subject=DataSubject.PATIENTS,
    pia_sensitivity_level=SensitivityLevel.HIGH,
)

  • DataRecipient: who receives the data (OPENDATA, INTERNAL, TRUSTED_THIRDPARTY, CONTRACTUAL_THIRDPARTY, OUTSIDE_EU, UNKNOWN). It has the largest effect on the targets.

  • DataType: the type of data (HEALTH, HR, FINANCE, INSURANCE, MOBILITY, EDUCATION, UNKNOWN).

  • DataSubject: the people the data is about (PATIENTS, EMPLOYEES, CLIENTS, USERS, STUDENTS, UNKNOWN).

  • SensitivityLevel: how sensitive the data is (from VERY_HIGH to NEGLIGIBLE, or UNDEFINED).

Without a context, the strictest targets (open data) apply.

In the web app: step Name & input → Job Context.

How to set the report language

The reports are in English (ReportLanguage.EN, default) or in French (ReportLanguage.FR):

from avatar_yaml.models.parameters import ReportLanguage

runner = manager.create_runner("wbcd_report_fr", report_language=ReportLanguage.FR)