OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Control where Source Code for Azure ML Command gets Uploaded

  • Thread starter Thread starter Matt_Haythornthwaite
  • Start date Start date
M

Matt_Haythornthwaite

Guest
I'm working in a notebook in Azure Machine Learning Studio and I'm using the following code block to instantiate a job using the command function.

Code:
from azure.ai.ml import command, Input, Output
from azure.ai.ml.entities import Data
from azure.ai.ml.constants import AssetTypes

subscription_id = "<subscription_id>"
resource_group = "<resource_group>"
workspace = "<workspace>"
storage_account = "<storage_account>"
input_path = "<input_path>"
output_path = "<output_path>"

input_dict = {
    "input_data_object": Input(
        type=AssetTypes.URI_FILE, 
        path=f"azureml://subscriptions/{subscription_id}/resourcegroups/{resource_group}/workspaces/{workspace}/datastores/{storage_account}/paths/{input_path}"
    )
}

output_dict = {
    "output_folder_object": Output(
        type=AssetTypes.URI_FOLDER,
        path=f"azureml://subscriptions/{subscription_id}/resourcegroups/{resource_group}/workspaces/{workspace}/datastores/{storage_account}/paths/{output_path}",
    )
}

job = command(
    code="./src", 
    command="python 01_read_write_data.py -v --input_data=${{inputs.input_data_object}} --output_folder=${{outputs.output_folder_object}}",
    inputs=input_dict,
    outputs=output_dict,
    environment="<asset_env>",
    compute="<compute_cluster>",
)

returned_job = ml_client.create_or_update(job)

This runs successfully but with each run, if the code stored within the ./src directory changes then a new copy is uploaded to the default blob storage account. I don't mind this, but with each run, the code is uploaded to a new container at the root of my blob storage account. Therefore my default storage account is getting cluttered with containers. I've read the docs for instantiating a command object using the command() function, but I see no parameter available to control where my ./src code gets uploaded. Is there any way to control this?
<p>I'm working in a notebook in Azure Machine Learning Studio and I'm using the following code block to instantiate a job using the <a href="https://learn.microsoft.com/en-us/p...e.ai.ml?view=azure-python#azure-ai-ml-command" rel="nofollow noreferrer">command function</a>.</p>
<pre><code>from azure.ai.ml import command, Input, Output
from azure.ai.ml.entities import Data
from azure.ai.ml.constants import AssetTypes

subscription_id = "<subscription_id>"
resource_group = "<resource_group>"
workspace = "<workspace>"
storage_account = "<storage_account>"
input_path = "<input_path>"
output_path = "<output_path>"

input_dict = {
"input_data_object": Input(
type=AssetTypes.URI_FILE,
path=f"azureml://subscriptions/{subscription_id}/resourcegroups/{resource_group}/workspaces/{workspace}/datastores/{storage_account}/paths/{input_path}"
)
}

output_dict = {
"output_folder_object": Output(
type=AssetTypes.URI_FOLDER,
path=f"azureml://subscriptions/{subscription_id}/resourcegroups/{resource_group}/workspaces/{workspace}/datastores/{storage_account}/paths/{output_path}",
)
}

job = command(
code="./src",
command="python 01_read_write_data.py -v --input_data=${{inputs.input_data_object}} --output_folder=${{outputs.output_folder_object}}",
inputs=input_dict,
outputs=output_dict,
environment="<asset_env>",
compute="<compute_cluster>",
)

returned_job = ml_client.create_or_update(job)
</code></pre>
<p>This runs successfully but with each run, if the code stored within the <code>./src</code> directory changes then a new copy is uploaded to the default blob storage account. I don't mind this, but with each run, the code is uploaded to a new container at the root of my blob storage account. Therefore my default storage account is getting cluttered with containers. I've read the docs for instantiating a <code>command</code> object using the <code>command()</code> function, but I see no parameter available to control where my <code>./src</code> code gets uploaded. Is there any way to control this?</p>
 

Latest posts

Online statistics

Members online
0
Guests online
6
Total visitors
6
Top