OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Web scraping google flight prices

  • Thread starter Thread starter Andrea Barnabò
  • Start date Start date
A

Andrea Barnabò

Guest
I am trying to learn to use the python library BeautifulSoup, I would like to, for example, scrape a price of a flight on Google Flights. So I connected to Google Flights, for example at this link, and I want to get the cheapest flight price.

So I would get the value inside the div with this class "gws-flights-results__itinerary-price" (as in the figure).

figure example


Here is the simple code I wrote:

Code:
from bs4 import BeautifulSoup
import urllib.request

url = 'https://www.google.com/flights?hl=it#flt=/m/07_pf./m/05qtj.2019-04-27;c:EUR;e:1;sd:1;t:f;tt:o'
page = urllib.request.urlopen(url)
soup = BeautifulSoup(page, 'html.parser')
div = soup.find('div', attrs={'class': 'gws-flights-results__itinerary-price'})

But the resulting div has class NoneType.

I also try with

Code:
find_all('div')

but within all the div I found in this way, there was not the div I was interested in. Can someone help me?
<p>I am trying to learn to use the python library BeautifulSoup, I would like to, for example, scrape a price of a flight on Google Flights.
So I connected to Google Flights, for example at <a href="https://www.google.com/flights?hl=it#flt=/m/07_pf./m/05qtj.2019-04-27;c:EUR;e:1;sd:1;t:f;tt:o" rel="nofollow noreferrer">this link</a>, and I want to get the cheapest flight price.</p>

<p>So I would get the value inside the div with this class "gws-flights-results__itinerary-price" (as in the figure).</p>

<p><img src="https://i.sstatic.net/RpWDF.png" alt="figure example"></p>

<p>Here is the simple code I wrote:</p>

<pre><code>from bs4 import BeautifulSoup
import urllib.request

url = 'https://www.google.com/flights?hl=it#flt=/m/07_pf./m/05qtj.2019-04-27;c:EUR;e:1;sd:1;t:f;tt:o'
page = urllib.request.urlopen(url)
soup = BeautifulSoup(page, 'html.parser')
div = soup.find('div', attrs={'class': 'gws-flights-results__itinerary-price'})
</code></pre>

<p>But the resulting div has class <code>NoneType</code>.</p>

<p>I also try with </p>

<pre><code>find_all('div')
</code></pre>

<p>but within all the div I found in this way, there was not the div I was interested in.
Can someone help me?</p>
 

Latest posts

Top