OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

"requests.exceptions.HTTPError: 400 Client Error: BadRequest for url" when querying an Azure Data Explorer table

  • Thread starter Thread starter Sheldon
  • Start date Start date
S

Sheldon

Guest
I am running a Python script from the Azure Machine Learning (AML) environment. This script queries data from an Azure Data Explorer (ADX) table, using the Kusto Query Language (KQL).

Here is an example KQL query:

Code:
QUERY = "my_adx_table | where relative_timestamp >= 1896 and relative_timestamp <= 2396 and my_file_id == 640"

Most of the time, this query works as expected, but for a handful of examples, AML returns the following error:

Code:
requests.exceptions.HTTPError: 400 Client Error: BadRequest for url: https://myclustername.myclusterregion.kusto.windows.net/v2/rest/query

According to this SO answer, "a 400 means that the request was malformed. In other words, the data stream sent by the client to the server didn't follow the rules." I therefore assumed that I was dealing a data-related issue.

However when trying to reproduce this error locally, I noticed that:

  • running the aforementioned query directly in the ADX query pane succeeds
  • calling the query from a Python script executed on my local computer succeeds as well (see code below).

Why am I getting this 400 Client Error when running the code from the AML environment?



APPENDIX: Example code to run the KQL query from a local computer:

Code:
from azure.kusto.data import KustoClient, KustoConnectionStringBuilder
from azure.kusto.data.helpers import dataframe_from_result_table
from azure.kusto.data.exceptions import KustoServiceError


QUERY = "my_adx_table | where relative_timestamp >= 1896 and relative_timestamp <= 2396 and my_file_id == 640"
print("QUERY =",QUERY)

adxconn = {
                "cluster":"https://myclustername.myclusterregion.kusto.windows.net",
                "client_id":"XXX",
                "client_secret":"YYY",
                "authority_id":"ZZZ",
                "kusto_db":"mydbname",
                "kusto_ingest_uri": "https://ingest-myclustername.myclusterregion.kusto.windows.net"               
            }

kcsb = KustoConnectionStringBuilder.with_aad_application_key_authentication(adxconn['cluster'], adxconn['client_id'], adxconn['client_secret'], adxconn['authority_id'])
client = KustoClient(kcsb)
RESPONSE = client.execute_query(adxconn['kusto_db'], QUERY)
print('response',RESPONSE)

df = dataframe_from_result_table(RESPONSE.primary_results[0])

This sample code returns a pandas dataframe containing the desired data.
<p>I am running a <strong>Python</strong> script from the <strong>Azure Machine Learning</strong> (AML) environment. This script queries data from an <strong>Azure Data Explorer</strong> (ADX) table, using the <strong>Kusto Query Language</strong> (KQL).</p>
<p>Here is an example KQL query:</p>
<pre><code>QUERY = "my_adx_table | where relative_timestamp >= 1896 and relative_timestamp <= 2396 and my_file_id == 640"
</code></pre>
<p>Most of the time, this query works as expected, but for a handful of examples, AML returns the following error:</p>
<pre><code>requests.exceptions.HTTPError: 400 Client Error: BadRequest for url: https://myclustername.myclusterregion.kusto.windows.net/v2/rest/query
</code></pre>
<p>According to <a href="https://stackoverflow.com/a/19671511/6440589">this SO answer</a>, <em>"a 400 means that the request was malformed. In other words, the data stream sent by the client to the server didn't follow the rules</em>." I therefore assumed that I was dealing a data-related issue.</p>
<p>However when trying to reproduce this error locally, I noticed that:</p>
<ul>
<li>running the aforementioned query directly in the ADX query pane succeeds</li>
<li>calling the query from a Python script executed on my local computer succeeds as well (see code below).</li>
</ul>
<p><strong>Why am I getting this 400 Client Error when running the code from the AML environment?</strong></p>
<hr />
<p><strong>APPENDIX: Example code to run the KQL query from a local computer:</strong></p>
<pre><code>from azure.kusto.data import KustoClient, KustoConnectionStringBuilder
from azure.kusto.data.helpers import dataframe_from_result_table
from azure.kusto.data.exceptions import KustoServiceError


QUERY = "my_adx_table | where relative_timestamp >= 1896 and relative_timestamp <= 2396 and my_file_id == 640"
print("QUERY =",QUERY)

adxconn = {
"cluster":"https://myclustername.myclusterregion.kusto.windows.net",
"client_id":"XXX",
"client_secret":"YYY",
"authority_id":"ZZZ",
"kusto_db":"mydbname",
"kusto_ingest_uri": "https://ingest-myclustername.myclusterregion.kusto.windows.net"
}

kcsb = KustoConnectionStringBuilder.with_aad_application_key_authentication(adxconn['cluster'], adxconn['client_id'], adxconn['client_secret'], adxconn['authority_id'])
client = KustoClient(kcsb)
RESPONSE = client.execute_query(adxconn['kusto_db'], QUERY)
print('response',RESPONSE)

df = dataframe_from_result_table(RESPONSE.primary_results[0])
</code></pre>
<p>This sample code returns a pandas dataframe containing the desired data.</p>
 

Latest posts

U
Replies
0
Views
1
user3658366
U
G
Replies
0
Views
1
Giampaolo Levorato
G
M
Replies
0
Views
1
Marcelo Rodrigo Nascimento
M
Top