Reuse past runs and configurations

The snippets on this page start from a manager (see Getting started):

import os

from avatars.manager import Manager

manager = Manager(api_key=os.environ["AVATAR_API_KEY"])

How to view my last jobs

Jobs are listed by creation date, with their identifier, creation date and status.

manager.get_last_jobs(count=10)  # the last 10 jobs
manager.get_jobs_by_name("wbcd_tuning")  # all the jobs of a run name
manager.get_last_results(count=10)  # a summary of the last 10 results

You can also see your last jobs and results in the web application.

How to reload results from a previous run

Use create_runner_from_name with the name you gave to the run. This works for runs launched from Python and from the web application.

runner = manager.create_runner_from_name("wbcd_tuning")
df = runner.shuffled("wbcd")
metrics = runner.privacy_metrics("wbcd")

If several runs have the same name, the most recent one is used. To choose another one, list them with find_ids_by_name and load one by its identifier with create_runner_from_id:

results = manager.find_ids_by_name("wbcd_tuning")
for set_name_id, jobs in results:
    print(set_name_id, [job.kind for job in jobs])

runner = manager.create_runner_from_id(results[0][0])

The identifier of a run is also in runner.set_name after run().

Run again from a loaded runner

A loaded runner is first a way to access existing results. When you call run() on it, a new run starts, and a warning gives the identifier of the previous run so that you can still reload it:

runner = manager.create_runner_from_name("wbcd_tuning")
runner.update_parameters("wbcd", k=20)
runner.run(ignore_warnings=True)
# If the data expired on the server, upload it again first:
# runner.upload_file("wbcd", "fixtures/wbcd.csv")

How to export a configuration as YAML

The YAML file holds the whole configuration of a run: tables, schema, parameters, metrics, report and PIA context. It does not hold the data. Use it to version a configuration or to share it.

runner.get_yaml("wbcd_tuning.yaml")  # writes the file and returns the YAML text

In the web app: run step → Configuration and execution script → Generated configuration → Download YAML configuration.

How to launch a job from a YAML file

Load a YAML file exported from Python or downloaded from the web application, upload the data of each table, and run:

runner = manager.create_runner_from_yaml("fixtures/yaml_from_web.yaml", set_name="iris_from_yaml")
runner.upload_file("iris", "fixtures/iris.csv")  # the data is not in the YAML
runner.jobs.config = runner.config  # needed with client 1.30.0 and earlier, see the warning
runner.run()

create_runner_from_yaml adds a version suffix to the name of the run (iris_from_yaml-v1). You can also load a YAML file in an existing runner with runner.from_yaml("path/to/config.yaml").

Warning

With client 1.30.0 and earlier, run() on a runner loaded from a YAML file raises ValueError: Expected k or epsilon to be set, because the jobs still use the configuration from before the YAML was loaded. Add runner.jobs.config = runner.config after create_runner_from_yaml or from_yaml, as in the snippet above.

Warning

upload_file names the file <table>.<extension>, and the YAML must refer to the same file name (for example file: iris.csv). A configuration created from a DataFrame refers to <table>.parquet: upload a Parquet file or a DataFrame in this case. Otherwise, the server does not find the data.

How to continue in Python a job launched in the web app

The run step of the web application shows a Simple python script to run the job. To use it reliably:

  1. Click Run now in the web application first: the script loads the job by its name, so it fails with No jobs found if the job does not exist yet. Give the job a unique name: if an older job has the same name, the script can load the results of that job.

  2. On a dedicated or on-premise instance, replace the base_url of the script with the URL of your server.

  3. Uncomment the upload_file lines only if the data has expired on the server (after 30 days by default), and use the current name of each table.

import os

from avatars.manager import Manager

manager = Manager(api_key=os.environ["AVATAR_API_KEY"])  # + base_url for a dedicated instance

# 1) Load the job launched in the web application
runner = manager.create_runner_from_name("my_web_job")  # exact name of the job
runner.statuses()  # check that the jobs are 'finished'

# 2) Get the results
avatars_df = runner.shuffled("my_table")
runner.privacy_metrics("my_table")
runner.signal_metrics("my_table")
runner.download_report("report.pdf")

# 3) Run again with other parameters (this starts a new run)
runner.update_parameters("my_table", k=20)
runner.run(ignore_warnings=True)

The YAML configuration is the exact reference of the parameters of a web job. To reproduce a web job from scratch, use the YAML (see above) rather than the script.