Azure Machine Learning Environments

According to Microsoft:

https://docs.microsoft.com/en-us/python/api/azureml-core/azureml.core.environment.environment?view=azure-ml-py

An Environment defines Python packages, environment variables, and Docker settings that are used in machine learning experiments, including in data preparation, training, and deployment to a web service. An Environment is managed and versioned in an Azure Machine Learning Workspace. You can update an existing environment and retrieve a version to reuse. Environments are exclusive to the workspace they are created in and can't be used across different workspaces.

In basic terms for a developer, it's basically a Docker Image with all the needed dependencies (conda/pip packages) to run your experiment.

This has been there in the Azure ML SDK since the very first versions, but recently Microsoft added the Environments option in Azure ML Studio, however at the time of this writing, it shows as a Preview option.

Microsoft by default gives us 17 environments to choose from, we can reuse them in our experiments or we can create our own.

How to create and use our own environment?

First we create the Environment variable and add conda dependencies we will need

from azureml.core.environment import CondaDependencies

myenv = Environment(name="myenv")
conda_dep = CondaDependencies()
conda_dep.add_conda_package("scikit-learn")

If needed, we can also add pip dependencies

conda_dep.add_pip_package("pillow==5.4.1")
myenv.python.conda_dependencies=conda_dep

If required for your experiments, also define environment variables

myenv.environment_variables = {"MESSAGE":"Hello from Azure Machine Learning"}

And finally we can use it in our experiments

from azureml.core import ScriptRunConfig, Experiment

myexp = Experiment(workspace=ws, name = "environment-example")

To submit a run, create a run configuration that combines the script file and environment, and pass it to Experiment.submit. In this example, the script is submitted to local computer, but you can specify other compute targets such as remote clusters as well.

src = ScriptRunConfig(source_directory=".",
                      script="test.py",
                      compute_target="local",
                      environment=myenv)

run = myexp.submit(config=src)

run.wait_for_completion(show_output=True)

If you want to know which environment you used for an experiment just run:

run.get_environment()

Full documentation here: