lassence.net

Field notes from the AI frontier

Go up a hierarchical tree with pandas

--- archive: codebook, 2020–2023 · topic: Pandas ---

A common way of storing hierarchical data is the tree:

Tree data structure with a root node and nested children

In SQL, the methods for going up a tree are very different between SQL dialects. In pandas however, it is relatively simple.

For this example, we’ll use the list of Google Ads geotargets, that stores locations in a hierarchical structure, and get the list of all parents for each location.

Setup

# Import libraries
import pandas as pd
import numpy as np

# Import Google Ads location data
df = (
    pd.read_csv('https://developers.google.com/adwords/api/docs/appendix/geo/geotargets-2021-02-24.csv')
    .rename(columns=lambda x: x.lower().replace(' ', '_'))
    .loc[lambda x: x['country_code'] == 'FR']
    .reset_index(drop=True)
    .assign(parent_id=lambda x: x['parent_id'].fillna(0).astype('int'))
    [['criteria_id', 'canonical_name', 'parent_id']]
)
df
criteria_idcanonical_nameparent_id
01005781Algolsheim,Grand Est,France9068896
11005782Cernay,Grand Est,France9068896
21005783Colmar,Grand Est,France9068896
31005784Ensisheim,Grand Est,France9068896
41005785Erstein,Grand Est,France9068896
…………
37559060754Petite-France,Grand Est,France9068896
37569072487Les Halles,Ile-de-France,France20321
37579072488Place Vendome,Ile-de-France,France20321
37589072489Sorbonne,Ile-de-France,France20321
37599072490Saint-Germain-des-Pres,Ile-de-France,France20321

Recursive function

# Recursive algorithm: list the next parent if any, otherwise exit
def recursive(row, parents):
    if row['parent_id'] == 0:
        return parents
    else:
        parents.append(row['parent_id'])
        return recursive(df.loc[df['criteria_id'] == row['parent_id'], :].squeeze(), parents)

# Call the recursive function for each row
def get_parents(row):
    parents = []
    return recursive(row, parents)

# Apply the function and create a column with the parents list for each location
df['parents_list'] = df.apply(get_parents, axis=1)
# Display resulting DataFrame
df.sort_values('parents_list')
criteria_idcanonical_nameparent_idparents_list
6832250France0[]
66920320Franche-Comte,France2250[2250]
67520326Nord-Pas-de-Calais,France2250[2250]
67420325Midi-Pyrenees,France2250[2250]
67320324Lorraine,France2250[2250]
……………
3088905609173370,Auvergne-Rhone-Alpes,France9069525[9069525, 2250]
3089905609273400,Auvergne-Rhone-Alpes,France9069525[9069525, 2250]
3090905609373600,Auvergne-Rhone-Alpes,France9069525[9069525, 2250]
6381006423Saint-Agreve,Auvergne-Rhone-Alpes,France9069525[9069525, 2250]
3112905611574500,Auvergne-Rhone-Alpes,France9069525[9069525, 2250]