OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Why can't we filter a pandas dataframe by column != None

  • Thread starter Thread starter AlanSTACK
  • Start date Start date
A

AlanSTACK

Guest
Let's say we have the following example pandas dataframe:

Code:
import pandas as pd

data = {'A': [1, 2, 3, 4], 'B': [None, 'x', 'y', None]}
df = pd.DataFrame(data)
print(df)

# we get the following
#    A     B
# 0  1  None
# 1  2     x
# 2  3     y
# 3  4  None

However, problems occur when attempting to filter using df['column'] != None

Code:
print(df[df['B'] != None])

# we get the following
#    A     B
# 0  1  None
# 1  2     x
# 2  3     y
# 3  4  None

It only functions correctly when we use the specific method df['column'].notna()

Code:
print(df[df['B'].notna()])

# we get the following
#    A  B
# 1  2  x
# 2  3  y

Why? Can someone explain this behavior?
<p>Let's say we have the following example pandas dataframe:</p>
<pre><code>import pandas as pd

data = {'A': [1, 2, 3, 4], 'B': [None, 'x', 'y', None]}
df = pd.DataFrame(data)
print(df)

# we get the following
# A B
# 0 1 None
# 1 2 x
# 2 3 y
# 3 4 None
</code></pre>
<p>However, problems occur when attempting to filter using <code>df['column'] != None</code></p>
<pre><code>print(df[df['B'] != None])

# we get the following
# A B
# 0 1 None
# 1 2 x
# 2 3 y
# 3 4 None
</code></pre>
<p>It only functions correctly when we use the specific method <code>df['column'].notna()</code></p>
<pre><code>print(df[df['B'].notna()])

# we get the following
# A B
# 1 2 x
# 2 3 y
</code></pre>
<p>Why? Can someone explain this behavior?</p>
 
Top