OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Download SAM Exclusions List Using a Python script

  • Thread starter Thread starter Michael Keiffer
  • Start date Start date
M

Michael Keiffer

Guest
I am attempting to write a Python script that will let me download the latest SAM Exclusions list from a government website. I get very close, but when my Python script clicks on the link to download the file, a pop-up window appears and the user must scroll to the bottom of the pop-up box before the Accept button becomes active and you can click on it. Once the Accept button is active and you click on it, the file will download.

Here is the website link to the SAM Exclusions file - https://sam.gov/data-services/Exclusions/Public V2?privacy=Public

When you put in this URL, you have to click on the OK button, select the very first Zip file link that is at the top, and then the other pop-up box appears where you have to scroll down in the box before the Accept button becomes active.

I am using a MacBook Pro and the MacOS to run my Python script. Below is the Python script that I have developed so far. Any assistance to allow the Python script to fully run and download the latest Exclusions file would be appreciated.

Here is my current script:

Code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import os
import time

def download_latest_file():
try:
    print("Initializing Chrome WebDriver...")
    options = webdriver.ChromeOptions()
    prefs = {'download.default_directory': '/Users/m_keiffer/Downloads'}  # Replace with     
    options.add_experimental_option('prefs', prefs)
    driver = webdriver.Chrome(options=options)

    print("Opening the webpage...")
    driver.get("https://sam.gov/data-services/Exclusions/Public%20V2?privacy=Public")

    wait = WebDriverWait(driver, 30)

    # Handle initial OK pop-up
    try:
        print("Handling initial OK pop-up...")
        ok_button = wait.until(EC.element_to_be_clickable((By.XPATH, 
        ok_button.click()
        time.sleep(2)
        print("OK button clicked.")
    except Exception as e:
        print(f"Error handling initial OK pop-up: {str(e)}")

    # Find and click the correct download link
    try:
        print("Finding download links...")
        exclusions_link = wait.until(EC.element_to_be_clickable((By.XPATH, 
        print(f"Clicking on the download link: {exclusions_link.text}")
        exclusions_link.click()
        time.sleep(5)
    except Exception as e:
        print(f"Error finding or clicking download link: {str(e)}")
        driver.quit()
        return

    # Handle Accept pop-up
    try:
        print("Handling Accept pop-up...")
        modal = wait.until(EC.presence_of_element_located((By.XPATH, '//sa-security-
        
        # Click inside the modal to activate it
        driver.execute_script("arguments[0].click();", modal)
        time.sleep(1)

        # Scroll to the bottom of the modal content
        driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", modal)
        time.sleep(2)  # Adjust time as necessary to ensure scroll is complete
        
        # Find and click the Accept button
        accept_button = wait.until(EC.element_to_be_clickable((By.XPATH, 
        driver.execute_script("arguments[0].click();", accept_button)
        print("Accept button clicked.")
    except Exception as e:
        print(f"Error handling Accept pop-up: {str(e)}")
        driver.quit()
        return

    # Wait for the download to complete
    time.sleep(20)

    download_path = "/Users/m_keiffer/Downloads"  # Replace with your actual username
    files_after = os.listdir(download_path)
    print(f"Files in Downloads: {files_after}")
    new_files = [f for f in files_after if f.endswith('.zip') or f.endswith('.csv')]
    if new_files:
        latest_file = max(new_files, key=lambda x:       
        print(f"Latest downloaded file: {latest_file}")
    else:
        print("No new files found in the Downloads folder.")

    driver.quit()
except Exception as e:
    print(f"Failed to initialize the WebDriver or other error: {str(e)}")

Replace your-username with your actual Mac username​


download_latest_file()

I have tried using ChatGPT to help me with the final step but so far nothing has worked. I get to where the last pop-up box appears but I cannot get the Accept button to become active and to be selected. I am not sure if it is possible to do this in Python, so any help would be greatly appreciated!
<p>I am attempting to write a Python script that will let me download the latest SAM Exclusions list from a government website. I get very close, but when my Python script clicks on the link to download the file, a pop-up window appears and the user must scroll to the bottom of the pop-up box before the Accept button becomes active and you can click on it. Once the Accept button is active and you click on it, the file will download.</p>
<p>Here is the website link to the SAM Exclusions file - <a href="https://sam.gov/data-services/Exclusions/Public V2?privacy=Public" rel="nofollow noreferrer">https://sam.gov/data-services/Exclusions/Public V2?privacy=Public</a></p>
<p>When you put in this URL, you have to click on the OK button, select the very first Zip file link that is at the top, and then the other pop-up box appears where you have to scroll down in the box before the Accept button becomes active.</p>
<p>I am using a MacBook Pro and the MacOS to run my Python script. Below is the Python script that I have developed so far. Any assistance to allow the Python script to fully run and download the latest Exclusions file would be appreciated.</p>
<p>Here is my current script:</p>
<pre><code>from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import os
import time

def download_latest_file():
try:
print("Initializing Chrome WebDriver...")
options = webdriver.ChromeOptions()
prefs = {'download.default_directory': '/Users/m_keiffer/Downloads'} # Replace with
options.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome(options=options)

print("Opening the webpage...")
driver.get("https://sam.gov/data-services/Exclusions/Public V2?privacy=Public")

wait = WebDriverWait(driver, 30)

# Handle initial OK pop-up
try:
print("Handling initial OK pop-up...")
ok_button = wait.until(EC.element_to_be_clickable((By.XPATH,
ok_button.click()
time.sleep(2)
print("OK button clicked.")
except Exception as e:
print(f"Error handling initial OK pop-up: {str(e)}")

# Find and click the correct download link
try:
print("Finding download links...")
exclusions_link = wait.until(EC.element_to_be_clickable((By.XPATH,
print(f"Clicking on the download link: {exclusions_link.text}")
exclusions_link.click()
time.sleep(5)
except Exception as e:
print(f"Error finding or clicking download link: {str(e)}")
driver.quit()
return

# Handle Accept pop-up
try:
print("Handling Accept pop-up...")
modal = wait.until(EC.presence_of_element_located((By.XPATH, '//sa-security-

# Click inside the modal to activate it
driver.execute_script("arguments[0].click();", modal)
time.sleep(1)

# Scroll to the bottom of the modal content
driver.execute_script("arguments[0].scrollTop = arguments[0].scrollHeight", modal)
time.sleep(2) # Adjust time as necessary to ensure scroll is complete

# Find and click the Accept button
accept_button = wait.until(EC.element_to_be_clickable((By.XPATH,
driver.execute_script("arguments[0].click();", accept_button)
print("Accept button clicked.")
except Exception as e:
print(f"Error handling Accept pop-up: {str(e)}")
driver.quit()
return

# Wait for the download to complete
time.sleep(20)

download_path = "/Users/m_keiffer/Downloads" # Replace with your actual username
files_after = os.listdir(download_path)
print(f"Files in Downloads: {files_after}")
new_files = [f for f in files_after if f.endswith('.zip') or f.endswith('.csv')]
if new_files:
latest_file = max(new_files, key=lambda x:
print(f"Latest downloaded file: {latest_file}")
else:
print("No new files found in the Downloads folder.")

driver.quit()
except Exception as e:
print(f"Failed to initialize the WebDriver or other error: {str(e)}")
</code></pre>
<h1>Replace your-username with your actual Mac username</h1>
<p>download_latest_file()</p>
<p>I have tried using ChatGPT to help me with the final step but so far nothing has worked. I get to where the last pop-up box appears but I cannot get the Accept button to become active and to be selected. I am not sure if it is possible to do this in Python, so any help would be greatly appreciated!</p>
 

Latest posts

ن
Replies
0
Views
1
نعمان منذر محمود الجميلي
ن
Top