OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Transfer Issue with tsop-agent v3.0.5219: Update to v3.0.5235

  • Thread starter Thread starter Jinja_dude
  • Start date Start date
J

Jinja_dude

Guest
We recently encountered a problem with our file transfer configuration using the tsop-agent for Google Cloud Storage. Here is a summary of the situation:

Problem Description​

  1. Launch an STS job to synchronize a POSIX filesystem on-prem to a GCP bucket.
  2. The synchronization consistently fails since 06/20.

Environment​

  • Environment: Production
  • GCP Service: Storage Transfer Service / Storage Service
  • DNS Configuration: DNS resolution for Google URLs (googleapis.com) is intentionally disabled by our security policies.
  • Proxy Configuration: All HTTP requests are routed through a proxy that can resolve and route these requests.

Diagnosis​

  • DNS: No resolution for www.googleapis.com and google.com, as expected.
  • Proxy: Correctly configured, curl commands work via the proxy.
  • STS Logs: Transfers fail for files approximately 35 MB or larger.

Solution​


After various tests and verifications, we discovered the problem stemmed from the version of the tsop-agent. An automatic update occurred, upgrading the agent from version v3.0.5219 to version v3.0.5235, which resolved the issue.

Question​


Has anyone else experienced this problem with tsop-agent version v3.0.5219? Do you know what was fixed in version v3.0.5235 to resolve this issue? I searched the documentation but couldn't find the release notes to compare the two versions.

Thanks in advance for your help!

Additional Details for Context:​

  1. DNS Configuration: DNS resolution for Google URLs is intentionally disabled by our security policies.
  2. Proxy Configuration: Verified and correct, with an initial warning about Boto configuration resolved.
  3. File Size: Specific issue with files around 35 MB or larger.
  4. Logs: Consistent error message on DNS resolution.

Thank you for your attention, and I hope someone can provide insights on the differences between these two versions of the tsop-agent.
<p>We recently encountered a problem with our file transfer configuration using the <strong>tsop-agent</strong> for Google Cloud Storage. Here is a summary of the situation:</p>
<h4>Problem Description</h4>
<ul>
<li><p><strong>Observed Error</strong>:
E0621 13:52:09.606702 13863 handler.go:168] failed to initialize upload: generic::unknown: retry budget exhausted (10 attempts): Post "https://www.googleapis.com/upload/s...json&ifGenerationMatch=0&uploadType=resumable": dial tcp: lookup <a href="http://www.googleapis.com" rel="nofollow noreferrer">www.googleapis.com</a> on DNS_IP:35 no such host</p>
</li>
<li><p><strong>Scenario</strong>:</p>
</li>
</ul>
<ol>
<li>Launch an STS job to synchronize a POSIX filesystem on-prem to a GCP bucket.</li>
<li>The synchronization consistently fails since 06/20.</li>
</ol>
<h4>Environment</h4>
<ul>
<li><strong>Environment</strong>: Production</li>
<li><strong>GCP Service</strong>: Storage Transfer Service / Storage Service</li>
<li><strong>DNS Configuration</strong>: DNS resolution for Google URLs (googleapis.com) is intentionally disabled by our security policies.</li>
<li><strong>Proxy Configuration</strong>: All HTTP requests are routed through a proxy that can resolve and route these requests.</li>
</ul>
<h4>Diagnosis</h4>
<ul>
<li><strong>DNS</strong>: No resolution for <code>www.googleapis.com</code> and <code>google.com</code>, as expected.</li>
<li><strong>Proxy</strong>: Correctly configured, <code>curl</code> commands work via the proxy.</li>
<li><strong>STS Logs</strong>: Transfers fail for files approximately 35 MB or larger.</li>
</ul>
<h4>Solution</h4>
<p>After various tests and verifications, we discovered the problem stemmed from the version of the <strong>tsop-agent</strong>. An automatic update occurred, upgrading the agent from version <strong>v3.0.5219</strong> to version <strong>v3.0.5235</strong>, which resolved the issue.</p>
<h4>Question</h4>
<p>Has anyone else experienced this problem with <strong>tsop-agent</strong> version <strong>v3.0.5219</strong>? Do you know what was fixed in version <strong>v3.0.5235</strong> to resolve this issue? I searched the documentation but couldn't find the release notes to compare the two versions.</p>
<p>Thanks in advance for your help!</p>
<h4>Additional Details for Context:</h4>
<ol>
<li><strong>DNS Configuration</strong>: DNS resolution for Google URLs is intentionally disabled by our security policies.</li>
<li><strong>Proxy Configuration</strong>: Verified and correct, with an initial warning about Boto configuration resolved.</li>
<li><strong>File Size</strong>: Specific issue with files around 35 MB or larger.</li>
<li><strong>Logs</strong>: Consistent error message on DNS resolution.</li>
</ol>
<p>Thank you for your attention, and I hope someone can provide insights on the differences between these two versions of the <strong>tsop-agent</strong>.</p>
Continue reading...
 

Latest posts

Top