OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Azure Data Factory global parameters don't work with new deployment method

  • Thread starter Thread starter Nick.Mc
  • Start date Start date
N

Nick.Mc

Guest
I'm using the "new" ADF deployment approach as defined here:

https://learn.microsoft.com/en-us/a...provements#continuous-deployment-improvements

Which pulls code out of the collabaration branch and validates and compiles it using this code:

(sample of validation code straight from the site - my code uses real values and runs as expected)

Code:
run build validate $(Build.Repository.LocalPath)/<Root-folder-from-Git-configuration-settings-in-ADF> /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/testResourceGroup/providers/Microsoft.DataFactory/factories/<Your-Factory-Name>

This works fine for me. The output of the build stage is an ARM template that can be deployed.

However when I start using global parameters I get this error:

Validator: Error for Expression: '@pipeline().globalParameters.gAFConnectivityTestEndPoint', Expression Error Message: 'Parameter gAFConnectivityTestEndPoint was not found under MYRESOURCEGROUP-001', attribute: '{"name":"url","display":"url","constraint":{},"primitive":true,"templatable":true,"_owner":{"name":"WebActivity","display":["Activity","Activities"]}}', obj: '{"_object":{},"_owned":{},"_ancestor":{},"_model":{},"_aObject":{}}', resolution: '{"kind":0,"details":{}}', resolution.error: '{"kind":0,"details":{}}'

The doco seems to indicate that global parameters are not published to the collabaration branch, however I've found that global parameters are acually written into the factory/ADFName.JSON file in the collabaration branch. So it appears to have all required information.

I've tried copying the global parameters JSON file into the collabaration branch from the adf_publish branch with the same filename and get the same error

Does anyone know what is required to get this working? How do I tell the npm task about my global parameters JSON?

EDIT​


I tried to repro for a much simpler pipeline and the npm task executed with no errors so it seems like this does work. Ticking "Include in ARM template" was not necessary for this to work. I need to work out why it fails for my more complicated pipeline. Curiously the error refers to the resource group, not the adf instance, so perhaps it is because the failing environment is heavily locked down.
<p>I'm using the "new" ADF deployment approach as defined here:</p>
<p><a href="https://learn.microsoft.com/en-us/a...provements#continuous-deployment-improvements" rel="nofollow noreferrer">https://learn.microsoft.com/en-us/a...provements#continuous-deployment-improvements</a></p>
<p>Which pulls code out of the collabaration branch and validates and compiles it using this code:</p>
<p>(sample of validation code straight from the site - my code uses real values and runs as expected)</p>
<pre><code>run build validate $(Build.Repository.LocalPath)/<Root-folder-from-Git-configuration-settings-in-ADF> /subscriptions/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/resourceGroups/testResourceGroup/providers/Microsoft.DataFactory/factories/<Your-Factory-Name>
</code></pre>
<p>This works fine for me. The output of the build stage is an ARM template that can be deployed.</p>
<p>However when I start using global parameters I get this error:</p>
<blockquote>
<p>Validator: Error for Expression:
'@pipeline().globalParameters.gAFConnectivityTestEndPoint', Expression
Error Message: 'Parameter gAFConnectivityTestEndPoint was not found
under MYRESOURCEGROUP-001', attribute:
'{"name":"url","display":"url","constraint":{},"primitive":true,"templatable":true,"_owner":{"name":"WebActivity","display":["Activity","Activities"]}}',
obj:
'{"_object":{},"_owned":{},"_ancestor":{},"_model":{},"_aObject":{}}',
resolution: '{"kind":0,"details":{}}', resolution.error:
'{"kind":0,"details":{}}'</p>
</blockquote>
<p>The doco seems to indicate that global parameters are not published to the collabaration branch, <em>however</em> I've found that global parameters are acually written into the <code>factory/ADFName.JSON</code> file in the collabaration branch. So it appears to have all required information.</p>
<p>I've tried copying the global parameters JSON file into the collabaration branch from the adf_publish branch with the same filename and get the same error</p>
<p>Does anyone know what is required to get this working? How do I tell the npm task about my global parameters JSON?</p>
<h1>EDIT</h1>
<p>I tried to repro for a much simpler pipeline and the npm task executed with no errors so it seems like this does work. Ticking "Include in ARM template" was not necessary for this to work. I need to work out why it fails for my more complicated pipeline. Curiously the error refers to the resource group, not the adf instance, so perhaps it is because the failing environment is heavily locked down.</p>
Continue reading...
 
Top