lassence.net

Field notes from the AI frontier

Cross-join two pandas DataFrames

--- archive: codebook, 2020–2023 · topic: Pandas ---

In data manipulation and analysis, joining different DataFrames is a common operation. One specific type of join is the cross-join, which combines all possible combinations of rows from two DataFrames. This can be particularly useful when working with hierarchical data, such as groups and subgroups, where you want to explore all possible combinations. Let’s see how to perform a cross-join using Pandas.

Consider two DataFrames, one representing groups and the other representing subgroups:

# Import libraries
import pandas as pd
from IPython.display import display

# Create DataFrames
df_groups = pd.DataFrame({'group': ['A', 'B', 'C']})
display(df_groups)

df_subgroups = pd.DataFrame({'subgroup': list(range(5))})
display(df_subgroups)
group
0A
1B
2C
subgroup
00
11
22
33
44

Here, df_groups contains three groups 'A', 'B', and 'C', and df_subgroups contains five subgroups ranging from 0 to 4.

Cross-Join with merge()

A cross-join can be performed using the merge() function in Pandas with the argument how='cross'. This will create a new DataFrame containing all possible combinations of groups and subgroups:

# Combine DataFrames with a cross-join
df_groups.merge(df_subgroups, how='cross')

The resulting DataFrame will have 15 rows, representing all combinations of the 3 groups and 5 subgroups:

groupsubgroup
0A0
1A1
2A2
3A3
4A4
5B0
6B1
7B2
8B3
9B4
10C0
11C1
12C2
13C3
14C4