OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

How can I input data into a webpage to scrape the resulting output using Python?

  • Thread starter Thread starter user728166
  • Start date Start date
U

user728166

Guest
I am familiar with BeautifulSoup and urllib2 to scrape data from a webpage. However, what if a parameter needs to be entered into the page before the result that I want to scrape is returned?

I'm trying to obtain the geographic distance between two addresses using this website: http://www.freemaptools.com/how-far-is-it-between.htm

I want to be able to go to the page, enter two addresses, click "Show", and then extract the "Distance as the Crow Flies" and "Distance by Land Transport" values and save them to a dictionary.

Is there any way to input data into a webpage using Python?
<p>I am familiar with BeautifulSoup and urllib2 to scrape data from a webpage. However, what if a parameter needs to be entered into the page before the result that I want to scrape is returned?</p>

<p>I'm trying to obtain the geographic distance between two addresses using this website: <a href="http://www.freemaptools.com/how-far-is-it-between.htm" rel="noreferrer">http://www.freemaptools.com/how-far-is-it-between.htm</a></p>

<p>I want to be able to go to the page, enter two addresses, click "Show", and then extract the "Distance as the Crow Flies" and "Distance by Land Transport" values and save them to a dictionary.</p>

<p>Is there any way to input data into a webpage using Python?</p>
 

Latest posts

Top