# Augmentation, differential privacy and pseudonymization

The snippets on this page start from this setup:

```python
import os

import pandas as pd

from avatars.manager import Manager

manager = Manager(api_key=os.environ["AVATAR_API_KEY"])
df = pd.read_csv("fixtures/wbcd.csv")

runner = manager.create_runner("wbcd_processing")
runner.add_table("wbcd", df)
runner.advise_parameters("wbcd")
```

## How to augment your dataset

Data augmentation generates more avatars than there are original rows, with
the same statistical patterns. Use it for example to train a model on a
larger dataset. Set `data_augmentation_strategy` to a factor:

```python
runner.update_parameters("wbcd", data_augmentation_strategy=3.0)  # 3 times more rows
runner.run()
avatars_df = runner.shuffled("wbcd")
```

**In the web app:** **Expert mode → Data Augmentation → Factor**.

## How to rebalance a minority class

```python
from avatar_yaml.models.parameters import AugmentationStrategy

runner.update_parameters(
    "wbcd",
    data_augmentation_strategy=AugmentationStrategy.minority,
    data_augmentation_target_column="Class",
)
# minority     : the smallest class grows to the size of the second smallest class
# not_majority : all the classes grow to the size of the largest class
```

The `PlotKind.CLASS_PROJECTION_2D` plot shows the result (see
{doc}`reporting`).

**In the web app:** **Expert mode → Imbalanced Data**.

## How to augment each modality with its own ratio

```python
runner.update_parameters(
    "wbcd",
    data_augmentation_strategy={"2": 1.0, "4": 2.0},  # ratio for each modality of the target
    data_augmentation_target_column="Class",
)
```

## How to augment and keep the original rows (sensitive)

By default, the original rows are anonymized too. With
`data_augmentation_should_anonymize_original_table=False`, the real rows stay
in the output, and only the new rows are synthetic.

```python
runner.update_parameters(
    "wbcd",
    data_augmentation_strategy=2.0,
    data_augmentation_should_anonymize_original_table=False,
)
```

:::{warning}
The output then contains real data: it is **not** anonymous. Use it only
inside your organization, for example to enlarge a training set that does
not leave it.
:::

## How to use differential privacy (FastDP)

FastDP applies differential privacy mechanisms in the latent space. A smaller
epsilon gives a stronger protection and a lower utility. `k` and an epsilon
cannot be used together.

```python
# From scratch
runner.set_parameters("wbcd", fast_dp_epsilon=5)

# After advise_parameters (which sets a k), remove k explicitly
runner.update_parameters("wbcd", k=None, fast_dp_epsilon=5)
```

**In the web app:** use case **Avatar DP Anonymization**.

:::{warning}
`update_parameters("wbcd", fast_dp_epsilon=5)` after `advise_parameters`
raises `ValueError: Expected either k or fast_dp_epsilon to be set, not both`.
Pass `k=None`.
:::

## How to use differential privacy (OpenDP, MST)

```python
runner.set_parameters("wbcd", open_dp_epsilon=1.0)
```

The same rules apply: no `k` with `open_dp_epsilon`, and pass `k=None` with
`update_parameters`.

## How to pseudonymize PII columns (Beta)

> **⚠️ Beta Feature** — Pseudonymization is currently in beta. The API and
> behavior may change in future releases.

Pseudonymization replaces direct identifiers (PII) with artificial values.
It works with avatarization: the PII columns are set aside before the
anonymization, are not modelled, and come back with their pseudonymized
values in the output.

```python
from avatars import (
    ConstantStrategy,
    FakeDataStrategy,
    HashSha256Strategy,
    PiiType,
    SpecificIdStrategy,
)

people = pd.read_csv("fixtures/pseudo_data.csv")

runner = manager.create_runner("pseudonymization_example")
runner.add_table("people", people)
runner.advise_parameters("people")
runner.update_parameters(
    "people",
    pseudonymized_columns={
        "first_name": FakeDataStrategy(pii_type=PiiType.FIRST_NAME),
        "last_name": FakeDataStrategy(pii_type=PiiType.LAST_NAME),
        "email": FakeDataStrategy(pii_type=PiiType.EMAIL, consistent=False),
        "ssn": SpecificIdStrategy(pattern="###-##-####"),
        "phone": HashSha256Strategy(),
        "address": ConstantStrategy(value="REDACTED"),
    },
)
runner.run()
result = runner.shuffled("people")
```

**In the web app:** step **Configure tables → Transformation =
Pseudonymization**.

### Available strategies

The strategies can be imported from `avatars` or from `avatar_yaml`.

- **`FakeDataStrategy(pii_type=PiiType.EMAIL)`** — Generates realistic fake
  values. **Requires** a `pii_type`. By default (`consistent=True`), the same
  source value always gives the same fake value. Pass `consistent=False` for a
  new value on each row. `high_variability=True` generates a unique value for
  each row (slower, no collisions).
- **`HashSha256Strategy()`** — Replaces values with their SHA-256 hex digest.
  Always deterministic: the same input gives the same hash in all runs. Good
  to link datasets; avoid it on low-entropy values (short codes, integers),
  which are open to dictionary attacks.
- **`Uuid4Strategy()`** — Replaces each unique value with a random UUID v4.
  Pass `consistent=False` for a new UUID on each row.
- **`ConstantStrategy(value="REDACTED")`** — Replaces every value with a fixed
  string. Use it when a column must be fully suppressed.
- **`IntegerStrategy()`** — Maps each unique value to a unique pseudonymous
  integer in random order. Pass `consistent=False` for an independent integer
  on each row.
- **`SpecificIdStrategy(pattern="EMP-####")`** — Generates structured IDs from
  a pattern. Placeholders: `#` (digit), `?` (letter), `^` (alphanumeric),
  `{{col}}` (value of another column). Set the letter case with
  `letter_case=SpecificIdLetterCase.UPPER`, `LOWER` or `BOTH`.

### Supported PII types

`FakeDataStrategy` needs a `PiiType`: `EMAIL`, `FIRST_NAME`, `LAST_NAME`,
`FULL_NAME`, `PHONE`, `SSN`, `ADDRESS`, `FREE_TEXT`.

### Pseudonymize identifiers in several tables

Pseudonymize the primary key of the parent table. The foreign keys that
point to it in the child tables get the same mapping automatically: do not
configure them.

```python
from avatars import Uuid4Strategy

patient = pd.read_csv("fixtures/patient.csv")
visit = pd.read_csv("fixtures/visit.csv").drop(columns=["doctor_id"])

runner = manager.create_runner("pseudonymization_multitable")
runner.add_table("patient", patient, primary_key="p_id", individual_level=True)
runner.add_table(
    "visit", visit, primary_key="visit_id", foreign_keys=["patient_id"], individual_level=False
)
runner.add_link("patient", "p_id", "visit", "patient_id")
runner.advise_parameters()
runner.update_parameters("patient", pseudonymized_columns={"p_id": Uuid4Strategy()})
runner.run()
```