Check if all values in column are NaN in Pandas
source link: https://thispointer.com/check-if-all-values-in-column-are-nan-in-pandas/
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
Check if all values in column are NaN in Pandas
This article will discuss checking if all values in a DataFrame column are NaN.
First of all, we will create a DataFrame from a list of tuples,
import pandas as pd
import numpy as np
# List of Tuples
empoyees = [('Jack', np.NaN, 34, 'Sydney', np.NaN, 5),
('Riti', np.NaN, 31, 'Delhi' , np.NaN, 7),
('Aadi', np.NaN, 16, 'London', np.NaN, np.NaN),
('Mark', np.NaN, 41, 'Delhi' , np.NaN, np.NaN)]
# Create a DataFrame object
df = pd.DataFrame( empoyees,
columns=['A', 'B', 'C', 'D', 'E', 'F'])
# Display the DataFrame
print(df)
Output:
A B C D E F 0 Jack NaN 34 Sydney NaN 5.0 1 Riti NaN 31 Delhi NaN 7.0 2 Aadi NaN 16 London NaN NaN 3 Mark NaN 41 Delhi NaN NaN
This DataFrame has four rows and six columns, out of which two columns (‘B’ & ‘E’) have all NaN values. Let’s see how we can verify if a column contains all NaN values or not in a DataFrame.
Check if all values are NaN in a column
Select the column as a Series object and then use isnull() and all() methods of the Series to verify if all values are NaN or not. The steps are as follows,
Advertisements
- Select the column by name using subscript operator of DataFrame i.e. df[‘column_name’]. It gives the column contents as a Pandas Series object.
- Call the isnull() function of the Series object. It returns a boolean Series of the same size. Each True value in this boolean Series indicates that the corresponding value in the Original Series (selected column) is NaN.
- Check if all values in the boolean Series are True or not. If yes, then it means all values in that column are NaN.
For example, let’s check if all values are NaN in column ‘B’ from the above created DataFrame,
# Check if all values in column 'B' are NaN
if df['B'].isnull().all():
print("All values in the column 'B' are NaN")
else:
print("All values in the column 'B' are not NaN")
Output:
All values in the column 'B' are NaN
We selected the column and then got a boolean series using the isnull() method. Then using the all() function, we checked if all the values in Boolean Series are True or not. If all values are True, then it means that all elements in the column are NaN.
In this example, the ‘B’ column had all values; therefore, the returned boolean Series had all True values, and the Series.all() function returned True in this case. Let’s check out a negative example,
Let’s check if all values are NaN in column ‘F’ in the above created DataFrame,
# Check if all values in column 'F' are NaN
if df['F'].isnull().all():
print("All values in the column 'F' are NaN")
else:
print("All values in the column 'F' are not NaN")
Output:
All values in the column 'F' are not NaN
In this example, all values in column ‘F’ are not NaN; therefore, the returned boolean Series had some True and few False values, and the Series.all() function returned False in this case. It proved that all elements in column ‘F’ are not NaN.
The complete working example is as follows,
import pandas as pd
import numpy as np
# List of Tuples
empoyees = [('Jack', np.NaN, 34, 'Sydney', np.NaN, 5),
('Riti', np.NaN, 31, 'Delhi' , np.NaN, 7),
('Aadi', np.NaN, 16, 'London', np.NaN, np.NaN),
('Mark', np.NaN, 41, 'Delhi' , np.NaN, np.NaN)]
# Create a DataFrame object
df = pd.DataFrame( empoyees,
columns=['A', 'B', 'C', 'D', 'E', 'F'])
# Display the DataFrame
print(df)
# Check if all values in column 'B' are NaN
if df['B'].isnull().all():
print("All values in the column 'B' are NaN")
else:
print("All values in the column 'B' are not NaN")
Output:
A B C D E F 0 Jack NaN 34 Sydney NaN 5.0 1 Riti NaN 31 Delhi NaN 7.0 2 Aadi NaN 16 London NaN NaN 3 Mark NaN 41 Delhi NaN NaN All values in the column 'B' are NaN
Summary
We learned how to check if all values in a DataFrame column are NaN.
Pandas Tutorials -Learn Data Analysis with Python
Are you looking to make a career in Data Science with Python?
Data Science is the future, and the future is here now. Data Scientists are now the most sought-after professionals today. To become a good Data Scientist or to make a career switch in Data Science one must possess the right skill set. We have curated a list of Best Professional Certificate in Data Science with Python. These courses will teach you the programming tools for Data Science like Pandas, NumPy, Matplotlib, Seaborn and how to use these libraries to implement Machine learning models.
Checkout the Detailed Review of Best Professional Certificate in Data Science with Python.
Remember, Data Science requires a lot of patience, persistence, and practice. So, start learning today.
Join a LinkedIn Community of Python Developers
Recommend
-
10
Create a pandas data frame with the date index and the random values in the column advertisements How do I create a pandas dataframe with da...
-
13
Count Unique Values in a Column – thisPointer.comThis article will discuss different ways to Count unique values in a Dataframe Column in Python. First of all, we will create a sample Dataframe from a list of tuples i.e. ...
-
10
Pandas | Count non-zero values in Dataframe Column This article will discuss how to count the number of non-zero values in one or more Dataframe columns in Pandas. Let’s first create a Dataframe from a...
-
15
Pandas – Count True Values in a Dataframe Column In this article, we will discuss different ways to count True values in a Dataframe Column. First of all, we will create a Dataframe from a list of tuples...
-
14
Pandas – Check if all values in a Column are Equal This article will discuss how to check if all values in a DataFrame Column are the same. First of all, we will create a DataFrame from a list of tuples,
-
11
Pandas: Check if all values in column are zeros This article will discuss checking if all values in a DataFrame column are zero (0) or not. First of all, we will create a DataFrame from a list of tuples,
-
14
Check if a Column exists in Pandas DataFrame In this article, we will discuss how to check if a column or multiple columns exist in a Pandas DataFrame or not. Suppose we have a DataFrame, ...
-
7
Replace NaN Values with Zeros in Pandas DataFrameReplace NaN Values with Zeros in Pandas DataFrame10 Views06/06/2022In this video, we are...
-
5
Remap values in Pandas Column with Dictionary In Pandas, A DataFrame is a two-dimensional array. Many times while working with pandas DataFrame, we need to remap the values of a specific column with dicti...
-
4
In this article, we will discuss various methods to replace the column values based on conditions in a pandas DataFrame. Let’s look at the table of contents describing the list of methods. Table of Contents
About Joyk
Aggregate valuable and interesting links.
Joyk means Joy of geeK