lassence.net

Field notes from the AI frontier

Select rows with NA values in any pandas DataFrame column

--- archive: codebook, 2020–2023 · topic: Pandas ---
# Import libraries
import pandas as pd
import numpy as np

# Create sample dataframe
df = pd.DataFrame({
    'population':[2148000, np.nan, 861635, 232741, np.nan], 
    'area_km':[105.4, 47.87, 240.6, np.nan, 78.26]},
    index=['Paris', 'Lyon',  'Marseille', 'Lille', 'Strasbourg']
)
df
populationarea_km
Paris2148000.0105.40
LyonNaN47.87
Marseille861635.0240.60
Lille232741.0NaN
StrasbourgNaN78.26

Select rows that have at least one NA value

# Standard way
df[df.isnull().any(axis=1)]
populationarea_km
LyonNaN47.87
Lille232741.0NaN
StrasbourgNaN78.26
# With .loc and lambda (useful for method chaining)
df.loc[lambda x: x.isnull().any(axis=1)]
populationarea_km
LyonNaN47.87
Lille232741.0NaN
StrasbourgNaN78.26

Filter out rows with NA values

# Select rows that do not have any NA value
df.dropna()
populationarea_km
Paris2148000.0105.4
Marseille861635.0240.6