# Avatarization parameters

The snippets on this page start from this setup:

```python
import os

import pandas as pd

from avatars.manager import Manager

manager = Manager(api_key=os.environ["AVATAR_API_KEY"])
df = pd.read_csv("fixtures/wbcd.csv")

runner = manager.create_runner("wbcd_parameters")
runner.add_table("wbcd", df)
runner.advise_parameters("wbcd")
```

## How to get the recommended parameters

`advise_parameters` asks the server for parameters adapted to your data
(`k`, `ncp`, imputation, ...). It is the recommended starting point.

```python
runner.advise_parameters("wbcd")  # without argument: all the tables
runner.print_parameters("wbcd")  # show what will be sent
```

**In the web app:** the values filled in at step **Configure job**, and the
**Run with recommended parameters** button.

## `set_parameters` or `update_parameters`?

- `update_parameters` keeps the current parameters (for example the
  recommended ones) and changes only the parameters you pass.
- `set_parameters` **replaces all** the parameters of the table, including
  the recommended ones.

```python
runner.advise_parameters("wbcd")

# Keep the recommendation, change only k
runner.update_parameters("wbcd", k=20)

# Replace everything: ncp, imputation, ... from the recommendation are lost
runner.set_parameters("wbcd", k=20)
```

:::{warning}

- `update_parameters` raises `ValueError: No existing parameters found` if you
  did not call `advise_parameters` or `set_parameters` before.
- `set_parameters` without `k` (or a DP epsilon) creates **no** avatarization
  parameters: only the metrics are configured. This is what you want to
  {ref}`evaluate an existing synthetic dataset <evaluate-existing-dataset>`,
  not to anonymize.
  :::

To remove parameters, use `runner.delete_parameters("wbcd")`.

## How to set k (privacy / utility trade-off)

`k` is the number of neighbours used to generate each avatar. It is the
parameter with the largest effect on the result. Change it step by step.

```python
runner.update_parameters("wbcd", k=30)
# small k (e.g. 5)  -> avatars close to the originals: more utility, less privacy
# large k (e.g. 50) -> smoother avatars: more privacy, less utility
```

`k=2` is the minimum and emits a warning. A `k` larger than the number of
rows makes the job fail.

**In the web app:** step **Configure job → k**.

## How to set ncp (number of components)

`ncp` is the number of dimensions of the projection used to find the
neighbours.

```python
runner.update_parameters("wbcd", ncp=5)
# low ncp  -> neighbours are found on the axes with the most variance
# high ncp -> more dimensions are used, including the "secondary" variables
```

**In the web app:** **Expert mode → ncp**.

## How to give more weight to a variable

A larger weight separates the individuals more along this variable. Use it to
preserve the variable of interest of your study better. The default weight is

1.

```python
runner.update_parameters("wbcd", column_weights={"Class": 3.0, "Bare_Nuclei": 2.0})
```

**In the web app:** step **Configure tables → Column weight**.

## How to handle categorical variables with many modalities

```python
runner.update_parameters("wbcd", use_categorical_reduction=True)
```

The categorical variables are encoded as numeric vectors before the
projection. Use it when the total number of modalities is large (more than
about 20). It can reduce the utility. See also
{ref}`group-rare-modalities`.

**In the web app:** **Expert mode → Categorical reduction for variables with
many modalities**.

(keep-a-column-out)=

## How to keep a column out of anonymization

Choose by what you want in the output:

| You want the column...                         | Use                                                     |
| ---------------------------------------------- | ------------------------------------------------------- |
| removed from the output                        | `columns_to_drop` in `add_table` ({doc}`data_handling`) |
| in the output, not modelled, reattached by row | `exclude_variable_names` + `ROW_ORDER` (freeze)         |
| in the output, reattached by similarity        | `exclude_variable_names` + `COORDINATE_SIMILARITY`      |
| in the output, replaced by fake values (PII)   | `pseudonymized_columns` ({doc}`processing`)             |

### Freeze a column (reattach by row order)

The column does not enter the projection. Its values are reattached to the
avatars by row order.

```python
from avatar_yaml.models.parameters import ExcludeVariablesMethod

runner.update_parameters(
    "wbcd",
    exclude_variable_names=["Mitoses"],
    exclude_variable_method=ExcludeVariablesMethod.ROW_ORDER,  # default
)
```

**In the web app:** step **Configure tables → Transformation = Freeze**.

### Exclude a column and reattach it by similarity

Recommended for a categorical variable with very many modalities: the
variable is excluded from the projection, then each avatar gets the value of
its closest original.

```python
from avatar_yaml.models.parameters import ExcludeVariablesMethod

runner.update_parameters(
    "wbcd",
    exclude_variable_names=["Mitoses"],
    exclude_variable_method=ExcludeVariablesMethod.COORDINATE_SIMILARITY,
    use_excluded_variables_in_metrics=True,  # measure the risk of this variable too
)
```

:::{warning}

- By default, an excluded variable is also excluded from the metrics: its
  risk is **not** measured. Set `use_excluded_variables_in_metrics=True` to
  measure it.
- `ROW_ORDER` keeps more identifying information than
  `COORDINATE_SIMILARITY`.
- Do not exclude primary keys or foreign keys. If the column contains PII,
  pseudonymize it instead.
  :::

## How to choose the imputation of missing values

Missing values are modelled: the server adds an indicator, fills the gaps
for the computation, then puts the missing values back in the avatars, in
the same proportions as in the original data.

```python
from avatar_yaml.models.parameters import ImputeMethod

runner.update_parameters(
    "wbcd",
    imputation_method=ImputeMethod.KNN,  # FAST_KNN, KNN, MEAN, MEDIAN, MODE
    imputation_k=5,  # neighbours for KNN and FAST_KNN
    imputation_training_fraction=0.5,  # fraction of the data used to fit the imputation
)
```

**In the web app:** **Expert mode → Imputation for missing data**.

## How to get avatars without missing values

By default, the avatars have missing values in the same proportions as the
original data. With `imputation_return_data_imputed=True`, you get complete
avatars.

```python
from avatar_yaml.models.parameters import ImputeMethod

runner.update_parameters(
    "wbcd",
    imputation_method=ImputeMethod.FAST_KNN,  # required, see the warning below
    imputation_return_data_imputed=True,
)
```

**In the web app:** **Expert mode → Return imputed data**.

:::{warning}
`imputation_return_data_imputed=True` is ignored, with no error, when no
imputation parameter is set (for example `set_parameters("wbcd", k=10,
imputation_return_data_imputed=True)`). Always pass `imputation_method` with
it. Check the result with `runner.print_parameters("wbcd")`.
:::

## How to reproduce a run exactly

The same seed, the same data and the same parameters give the same avatars.
Set the seed when you create the runner:

```python
runner = manager.create_runner("wbcd_seed_42", seed=42)
runner.add_table("wbcd", df)
runner.advise_parameters("wbcd")
runner.run()
```

To reuse the exact configuration of a past run, export it as YAML (see
{doc}`advanced`).