OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

BeautifulSoup AttributeError: 'NoneType' object has no attribute 'get_text' when calling .get_text() directly

  • Thread starter Thread starter Kopsie
  • Start date Start date
K

Kopsie

Guest
I am using BeautifulSoup with Selenium to scrape data from a website. I have encountered an issue where accessing the get_text() method directly results in an AttributeError, while accessing it indirectly does not cause any error.

Here is the relevant part of my code:

Working Code (No Error)​


Code:
page_content = BeautifulSoup(driver.page_source, "html.parser")
article_rows = page_content.select(".project-row.published")

for i, article_row in enumerate(article_rows):
    title = article_row.select_one(".project-title")
    print("title", title.get_text())

Output:

Code:
title 

title for the article

Failing Code (AttributeError)​


Code:
page_content = BeautifulSoup(driver.page_source, "html.parser")
article_rows = page_content.select(".project-row.published")

for i, article_row in enumerate(article_rows):
    title = article_row.select_one(".project-title").get_text()

Error:

Code:
AttributeError: 'NoneType' object has no attribute 'get_text'

Details:

  • In the first code block, I am storing the result of select_one in a variable title and then calling get_text() on this variable. This works without any issues.
  • In the second code block, I am trying to call get_text() directly on the result of select_one, but this results in an AttributeError.

Question: Why does the first approach work without error, while the second approach causes an AttributeError? How can I safely call get_text() directly?

Additional Information:

  • I have ensured that the elements with class project-title do exist on the page.
  • Using Python 3.12.4 and BeautifulSoup 4.12.3.
  • The page is loaded correctly and all elements seem to be in place.

Any insights or solutions to prevent the AttributeError when calling get_text() directly would be greatly appreciated.

Thank you!
<p>I am using BeautifulSoup with Selenium to scrape data from a website. I have encountered an issue where accessing the <code>get_text()</code> method directly results in an <code>AttributeError</code>, while accessing it indirectly does not cause any error.</p>
<p>Here is the relevant part of my code:</p>
<h3>Working Code (No Error)</h3>
<pre class="lang-py prettyprint-override"><code>page_content = BeautifulSoup(driver.page_source, "html.parser")
article_rows = page_content.select(".project-row.published")

for i, article_row in enumerate(article_rows):
title = article_row.select_one(".project-title")
print("title", title.get_text())
</code></pre>
<p><strong>Output:</strong></p>
<pre><code>title

title for the article
</code></pre>
<h3>Failing Code (AttributeError)</h3>
<pre class="lang-py prettyprint-override"><code>page_content = BeautifulSoup(driver.page_source, "html.parser")
article_rows = page_content.select(".project-row.published")

for i, article_row in enumerate(article_rows):
title = article_row.select_one(".project-title").get_text()
</code></pre>
<p><strong>Error:</strong></p>
<pre><code>AttributeError: 'NoneType' object has no attribute 'get_text'
</code></pre>
<p><strong>Details:</strong></p>
<ul>
<li>In the first code block, I am storing the result of <code>select_one</code> in a variable <code>title</code> and then calling <code>get_text()</code> on this variable. This works without any issues.</li>
<li>In the second code block, I am trying to call <code>get_text()</code> directly on the result of <code>select_one</code>, but this results in an <code>AttributeError</code>.</li>
</ul>
<p><strong>Question:</strong>
Why does the first approach work without error, while the second approach causes an <code>AttributeError</code>? How can I safely call <code>get_text()</code> directly?</p>
<p><strong>Additional Information:</strong></p>
<ul>
<li>I have ensured that the elements with class <code>project-title</code> do exist on the page.</li>
<li>Using Python 3.12.4 and BeautifulSoup 4.12.3.</li>
<li>The page is loaded correctly and all elements seem to be in place.</li>
</ul>
<p>Any insights or solutions to prevent the <code>AttributeError</code> when calling <code>get_text()</code> directly would be greatly appreciated.</p>
<p>Thank you!</p>
 

Latest posts

S
Replies
0
Views
1
Souvik Manna
S
S
Replies
0
Views
1
Shelling ford
S
Top