To integrate labeled data from Azure Machine Learning data labeling into machine learning pipelines, you can follow these general steps:
- Set up the Azure Machine Learning workspace: Ensure you have an Azure Machine Learning workspace set up. You can create one using the Azure portal.
- Data labeling: Use the Azure Machine Learning data labeling capabilities to label your data. You can use Azure Machine Learning Studio to create labeling projects, upload data, and manage labeling tasks.
- Store the labeled data: After data labeling is complete, the labeled data is typically stored in storage. You can create a dataset in Azure Machine Learning that points to the location of your labeled data.
- Define the machine learning pipeline: Create an Azure Machine Learning pipeline that includes steps for data preprocessing, model training, and evaluation. You can use the Azure Machine Learning SDK to define these steps in a Python script.
- Reference the labeled dataset: In the pipeline, reference the labeled dataset you created in the Data labeling step. This dataset will be used for training your machine learning model.
- Run the pipeline: Execute the pipeline in your Azure Machine Learning workspace. This will trigger the data preprocessing, model training, and evaluation steps consistently and repeatably.
- Monitor and Iterate: Monitor the pipeline execution and evaluate model performance. If necessary, iterate on the pipeline to improve your model by adjusting hyperparameters or using different algorithms.
Here is a simplified example using the Azure Machine Learning SDK to give you an idea:
from azureml.core import Dataset, Workspace
from azureml.core.experiment import Experiment
from azureml.core.runconfig import RunConfiguration
from azureml.core.conda_dependencies import CondaDependencies
Load your Azure ML workspace
ws = Workspace.from_config()
Reference the labeled dataset
labeled_dataset = Dataset.get_by_name(ws, name=’your_labeled_dataset_name’)
Define a machine learning experiment
experiment_name = ‘your_experiment_name’
experiment = Experiment(workspace=ws, name=experiment_name)
Define a run configuration with necessary dependencies
run_config = RunConfiguration()
run_config.environment.python.user_managed_dependencies = False
run_config.environment.python.conda_dependencies = CondaDependencies.create(conda_packages=[‘your_required_packages’])
Define your machine learning pipeline steps
…
Reference the labeled dataset in your pipeline steps
…
Submit the pipeline run
pipeline_run = experiment.submit(pipeline)
Remember to replace placeholders such as ‘your_labeled_dataset_name’ and ‘your_required_packages’ with your actual dataset name and required Python packages.
Adjust the pipeline steps according to your specific use case and requirements. The Azure Machine Learning SDK documentation provides detailed information on how to define and run pipelines, as ML pipeline implementation is beyond the scope of this book.
Now, let’s see how to label the data using the open source tool Label Studio.
Exploring Label Studio
Label Studio (https://labelstud.io/) is an open source data labeling and annotation platform designed to streamline the process of labeling diverse data types, including images, text, and audio. With a user-friendly interface, Label Studio empowers machine learning practitioners and data scientists to efficiently label and annotate datasets for training and evaluating models. Its versatility, collaborative features, and support for multiple labeling tasks make it a valuable tool in the development of robust and accurate machine learning models.
In this section, we are going to label four types of data: image, video, text, and audio.