lassence.net

Field notes from the AI frontier

Training: introduction to SQL

--- date: · theme: data · words: 2,727 ---

Setup

Google Analytics data

### Check first rows

SELECT *
FROM analytics
LIMIT 5
visitor_idvisit_start_timedatevisitspageviewstime_on_sitetransactionsrevenuechannelsourcemediumis_mobilecountry
0216463892932268303914936264072017-05-0111000.0Organic SearchgoogleorganicFalseGermany
1907476176218667191214936758162017-05-0111000.0Referral(direct)(none)FalseBelgium
2587893516712831607214936616062017-05-0111000.0Organic SearchgoogleorganicTrueGermany
354419503437407998914937070652017-05-0111000.0Organic SearchgoogleorganicFalseIndonesia
4388645666439540129314936522072017-05-0111000.0Organic SearchgoogleorganicFalseUnited Kingdom

Countries data

### Check first rows

SELECT *
FROM countries
LIMIT 5
countrypopulation
0China1407794600
1India1376519831
2United States331611270
3Indonesia271350000
4Pakistan225200000

Tutorial

1. Select - From - Limit

### ✅ 1.1. Select all rows from table

SELECT *
FROM analytics
visitor_idvisit_start_timedatevisitspageviewstime_on_sitetransactionsrevenuechannelsourcemediumis_mobilecountry
0216463892932268303914936264072017-05-0111000.0Organic SearchgoogleorganicFalseGermany
1907476176218667191214936758162017-05-0111000.0Referral(direct)(none)FalseBelgium
2587893516712831607214936616062017-05-0111000.0Organic SearchgoogleorganicTrueGermany
354419503437407998914937070652017-05-0111000.0Organic SearchgoogleorganicFalseIndonesia
4388645666439540129314936522072017-05-0111000.0Organic SearchgoogleorganicFalseUnited Kingdom
..........................................
653663530673266201216014962731482017-05-311519100.0Organic SearchgoogleorganicFalseNew Zealand
653671225749775589867114962370322017-05-311516700.0Organic SearchgoogleorganicFalseUnited States
65368858692843109634818114962381162017-05-31158700.0Organic SearchgoogleorganicFalseSri Lanka
65369168615503927473379114962263112017-05-311545400.0Organic SearchgoogleorganicFalseSpain
65370165642735611770937214962324102017-05-3115144000.0AffiliatesPartnersaffiliateTrueUnited Kingdom
### ✅ 1.2. Show first 5 rows

SELECT *
FROM analytics
LIMIT 5
visitor_idvisit_start_timedatevisitspageviewstime_on_sitetransactionsrevenuechannelsourcemediumis_mobilecountry
0216463892932268303914936264072017-05-0111000.0Organic SearchgoogleorganicFalseGermany
1907476176218667191214936758162017-05-0111000.0Referral(direct)(none)FalseBelgium
2587893516712831607214936616062017-05-0111000.0Organic SearchgoogleorganicTrueGermany
354419503437407998914937070652017-05-0111000.0Organic SearchgoogleorganicFalseIndonesia
4388645666439540129314936522072017-05-0111000.0Organic SearchgoogleorganicFalseUnited Kingdom
### ✅ 1.3. Show only visitor_id, date, channel for first 10 rows

SELECT
    visitor_id,
    date,
    channel
FROM analytics
LIMIT 10
visitor_iddatechannel
021646389293226830392017-05-01Organic Search
190747617621866719122017-05-01Referral
258789351671283160722017-05-01Organic Search
35441950343740799892017-05-01Organic Search
438864566643954012932017-05-01Organic Search
532392586349873579452017-05-01Organic Search
627534623319530415912017-05-01Organic Search
772907100666380064622017-05-01Referral
892709773863490358632017-05-01Organic Search
94557330855028660302017-05-01Direct

2. Conditions

### ✅ 2.1. Select rows with more than 100 pageviews

SELECT *
FROM analytics
WHERE pageviews > 100
visitor_idvisit_start_timedatevisitspageviewstime_on_sitetransactionsrevenuechannelsourcemediumis_mobilecountry
0749869596335463519914937714132017-05-021132210700.00Socialplus.google.comreferralFalseUnited States
148519678941802156414953593792017-05-21111852871199.18Referral(direct)(none)FalseUnited States
2970364459783640930414955681202017-05-231136700700.00Direct(direct)(none)TrueIndia
356711119202401320914959897012017-05-281102402200.00Referral(direct)(none)FalseUnited States
4385547380447299831714960854132017-05-29111843891129.37Referral(direct)(none)FalseUnited States
5863715893230165831814962507832017-05-31113935951525.00Referral(direct)(none)FalseUnited States
### ✅ 2.2. Select 10 visits with exactly 2 transactions

SELECT *
FROM analytics
WHERE transactions = 2
LIMIT 10
visitor_idvisit_start_timedatevisitspageviewstime_on_sitetransactionsrevenuechannelsourcemediumis_mobilecountry
096500184523399589814937355472017-05-021242464267.18Direct(direct)(none)FalseUnited States
1853259348805764552314937421192017-05-021312017233.59Direct(direct)(none)FalseUnited States
2475085530542044502214937798782017-05-021322499217.58Referral(direct)(none)FalseUnited States
3752860528147195760314937411532017-05-021401929292.91Referral(direct)(none)FalseUnited States
4950361299318927842714939164642017-05-04121843233.59Direct(direct)(none)FalseUnited States
5370408188148890519914940315242017-05-05155116821521.51Direct(direct)(none)FalseJapan
6598536823337698036314942724152017-05-08129926255.80Referral(direct)(none)FalseUnited States
7383553463773670607314942841842017-05-08131663243.98Referral(direct)(none)FalseUnited States
8703337092659025257914943703532017-05-091541422285.58Referral(direct)(none)FalseUnited States
9786788224708816338314945394362017-05-111501553262.60Direct(direct)(none)FalseSweden
### ✅ 2.3. Select visits with 3 to 5 transactions

SELECT *
FROM analytics
WHERE transactions BETWEEN 3 AND 5
visitor_idvisit_start_timedatevisitspageviewstime_on_sitetransactionsrevenuechannelsourcemediumis_mobilecountry
0240252719973115093214937395872017-05-021752893462.27Direct(direct)(none)FalseUnited Kingdom
1492141380694078467614942852292017-05-081641232452.07Organic SearchgoogleorganicFalseUnited States
2498851793713993714514950396312017-05-17126557318.99Organic SearchgoogleorganicFalseUnited States
3902684071808201004014951997252017-05-1915122223272.79Direct(direct)(none)FalseUnited States

3. Functions and columns renaming

### ✅ 3.1. Show number of rows
### Hint: use COUNT()

SELECT COUNT(*)
FROM analytics
count
065371
### ✅ 3.2. Show sum of visits, transactions, and revenue
### Hint: use SUM()

SELECT
    SUM(visits),
    SUM(transactions),
    SUM(revenue)
FROM analytics
sumsumsum
065371.01160.0121711.48
### ✅ 3.3. Show sum of visits, transactions, and revenue, and rename the columns properly
### Hint: use AS

SELECT
    SUM(visits) AS visits,
    SUM(transactions) AS transactions,
    SUM(revenue) AS revenue
FROM analytics
visitstransactionsrevenue
065371.01160.0121711.48
### ✅ 3.4. Show first and last date, and rename them

SELECT
    MIN(date) AS min_date,
    MAX(date) AS max_date
FROM analytics
min_datemax_date
02017-05-012017-05-31
### ✅ 3.5. Show conversion rate overall (transactions / visits)

SELECT
    SUM(transactions) / SUM(visits) AS conv_rate
FROM analytics
conv_rate
00.017745
### ✅ 3.6. Show conversion rate between 1st and 10 May

SELECT
    SUM(transactions) / SUM(visits) AS conv_rate
FROM analytics
WHERE date BETWEEN '2017-05-01' AND '2017-05-10'
conv_rate
00.021916
### ✅ 3.7. Show conversion rate on week-ends
### Hint: use DATE_PART('ISODOW', date) to get the weekday

SELECT
    SUM(transactions) / SUM(visits) AS conv_rate
FROM analytics
WHERE DATE_PART('ISODOW', date) > 5
conv_rate
00.011282
### ✅ 3.8. Show number of transactions, total revenue and avg basket
### for 'Organic Search' between 10/05 and 15/05

SELECT
    SUM(transactions) AS transactions,
    SUM(revenue) AS revenue,
    SUM(revenue) / SUM(transactions) AS avg_basket
FROM analytics
WHERE
    channel = 'Organic Search'
    AND date BETWEEN '2017-05-10' AND '2017-05-15'
transactionsrevenueavg_basket
066.05198.6778.767727

4. Grouping and ordering

### ✅ 4.1. Show number of visits per day

SELECT
    date,
    SUM(visits)
FROM analytics
GROUP BY 1
datesum
02017-05-312292.0
12017-05-302199.0
22017-05-262009.0
32017-05-182715.0
42017-05-141290.0
52017-05-192223.0
62017-05-291745.0
72017-05-082025.0
82017-05-281463.0
92017-05-252228.0
102017-05-201855.0
112017-05-173117.0
122017-05-242406.0
132017-05-121932.0
142017-05-222248.0
152017-05-112218.0
162017-05-131251.0
172017-05-232129.0
182017-05-211645.0
192017-05-012588.0
202017-05-102024.0
212017-05-163098.0
222017-05-071400.0
232017-05-271502.0
242017-05-042806.0
252017-05-051960.0
262017-05-032588.0
272017-05-061383.0
282017-05-022564.0
292017-05-152360.0
302017-05-092108.0
### ✅ 4.2. Show number of visits per day, ordered by day, for May 1 to 10

SELECT
    date,
    SUM(visits) AS visits
FROM analytics
WHERE date BETWEEN '2017-05-01' AND '2017-05-10'
GROUP BY 1
ORDER BY 1
datevisits
02017-05-012588.0
12017-05-022564.0
22017-05-032588.0
32017-05-042806.0
42017-05-051960.0
52017-05-061383.0
62017-05-071400.0
72017-05-082025.0
82017-05-092108.0
92017-05-102024.0
### ✅ 4.3. Show revenue per day, ordered by revenue descending, for top 5 days

SELECT
    date,
    SUM(revenue) AS revenue
FROM analytics
GROUP BY 1
ORDER BY 2 DESC
LIMIT 5
daterevenue
02017-05-3110423.59
12017-05-097329.01
22017-05-017126.08
32017-05-057099.86
42017-05-126677.05
### ✅ 4.4. Show distribution of number of transactions
### i.e. how many visits with 1, 2, 3... etc. transactions

SELECT
    transactions,
    COUNT(*)
FROM analytics
GROUP BY 1
ORDER BY 1
transactionscount
0064256
111089
2221
332
442
5151
### ✅ 4.5. Show visits and pageviews per visit by channel, ordered by pages per visit descending

SELECT
    channel,
    SUM(visits) AS visits,
    SUM(pageviews) / SUM(visits) AS pageviews_per_visit
FROM analytics
GROUP BY 1
ORDER BY 3 DESC
channelvisitspageviews_per_visit
0Referral9251.05.514863
1Paid Search1887.05.246953
2Display427.04.224824
3Direct15599.03.998077
4(Other)5.03.600000
5Organic Search33199.03.535378
6Affiliates1317.02.658314
7Social3686.02.468258

5. Joins

### ✅ 5.1. Show first 5 rows for `country` table

SELECT *
FROM countries
LIMIT 5
countrypopulation
0China1407794600
1India1376519831
2United States331611270
3Indonesia271350000
4Pakistan225200000
### ✅ 5.2. Join `analytics` table with `countries` table for 10 rows

SELECT *
FROM analytics
LEFT JOIN countries
ON analytics.country = countries.country
LIMIT 10
visitor_idvisit_start_timedatevisitspageviewstime_on_sitetransactionsrevenuechannelsourcemediumis_mobilecountrycountrypopulation
0216463892932268303914936264072017-05-0111000.0Organic SearchgoogleorganicFalseGermanyGermany83190556
1907476176218667191214936758162017-05-0111000.0Referral(direct)(none)FalseBelgiumBelgium11560220
2587893516712831607214936616062017-05-0111000.0Organic SearchgoogleorganicTrueGermanyGermany83190556
354419503437407998914937070652017-05-0111000.0Organic SearchgoogleorganicFalseIndonesiaIndonesia271350000
4388645666439540129314936522072017-05-0111000.0Organic SearchgoogleorganicFalseUnited KingdomUnited Kingdom66796807
5323925863498735794514936694472017-05-0111000.0Organic SearchgoogleorganicFalseCanadaCanada38443464
6275346233195304159114936578022017-05-0111000.0Organic SearchgoogleorganicTrueSri LankaSri Lanka21919000
7729071006663800646214936359452017-05-0111000.0Referraldocs.google.comreferralFalseIndiaIndia1376519831
8927097738634903586314936335492017-05-0111000.0Organic SearchgoogleorganicFalseIndiaIndia1376519831
945573308550286603014936433902017-05-0111000.0Direct(direct)(none)TrueUnited StatesUnited States331611270
### ✅ 5.3. Join `analytics` with `countries` using `USING`

SELECT *
FROM analytics
LEFT JOIN countries
USING (country)
LIMIT 10
countryvisitor_idvisit_start_timedatevisitspageviewstime_on_sitetransactionsrevenuechannelsourcemediumis_mobilepopulation
0Germany216463892932268303914936264072017-05-0111000.0Organic SearchgoogleorganicFalse83190556
1Belgium907476176218667191214936758162017-05-0111000.0Referral(direct)(none)False11560220
2Germany587893516712831607214936616062017-05-0111000.0Organic SearchgoogleorganicTrue83190556
3Indonesia54419503437407998914937070652017-05-0111000.0Organic SearchgoogleorganicFalse271350000
4United Kingdom388645666439540129314936522072017-05-0111000.0Organic SearchgoogleorganicFalse66796807
5Canada323925863498735794514936694472017-05-0111000.0Organic SearchgoogleorganicFalse38443464
6Sri Lanka275346233195304159114936578022017-05-0111000.0Organic SearchgoogleorganicTrue21919000
7India729071006663800646214936359452017-05-0111000.0Referraldocs.google.comreferralFalse1376519831
8India927097738634903586314936335492017-05-0111000.0Organic SearchgoogleorganicFalse1376519831
9United States45573308550286603014936433902017-05-0111000.0Direct(direct)(none)True331611270

6. Window functions

### ✅ 6.1. Visits by day vs visits on previous day

SELECT
    date,
    visits,
    LAG(visits) OVER (ORDER BY date) AS prev_visits
FROM (
    SELECT
        date,
        SUM(visits) AS visits
    FROM analytics
    GROUP BY 1
) AS sub1
ORDER BY date
datevisitsprev_visits
02017-05-012588.0NaN
12017-05-022564.02588.0
22017-05-032588.02564.0
32017-05-042806.02588.0
42017-05-051960.02806.0
52017-05-061383.01960.0
62017-05-071400.01383.0
72017-05-082025.01400.0
82017-05-092108.02025.0
92017-05-102024.02108.0
102017-05-112218.02024.0
112017-05-121932.02218.0
122017-05-131251.01932.0
132017-05-141290.01251.0
142017-05-152360.01290.0
152017-05-163098.02360.0
162017-05-173117.03098.0
172017-05-182715.03117.0
182017-05-192223.02715.0
192017-05-201855.02223.0
202017-05-211645.01855.0
212017-05-222248.01645.0
222017-05-232129.02248.0
232017-05-242406.02129.0
242017-05-252228.02406.0
252017-05-262009.02228.0
262017-05-271502.02009.0
272017-05-281463.01502.0
282017-05-291745.01463.0
292017-05-302199.01745.0
302017-05-312292.02199.0

Exercises

Exercise 1

👉 Question: Show pageviews per visit by channel, rounded at 2 decimals, in descending order.

💡 Solution:

SELECT
    channel,
    ROUND(SUM(pageviews) / SUM(visits), 2) AS pageviews_per_visit
FROM analytics
GROUP BY 1
ORDER BY 1 DESC
channelpageviews_per_visit
0Social2.47
1Referral5.51
2Paid Search5.25
3(Other)3.60
4Organic Search3.54
5Display4.22
6Direct4.00
7Affiliates2.66

Exercise 2

👉 Question: Show average number of daily visits.

💡 Solution:

SELECT
    AVG(visits) AS avg_visits_1,
    SUM(visits) / COUNT(date) AS avg_visits_2
FROM (
    SELECT
        date,
        SUM(visits) AS visits
    FROM analytics
    GROUP BY 1
) AS sub1
avg_visits_1avg_visits_2
02108.7419352108.741935

Exercise 3

👉 Question: Show visits, revenue and conversion rate by country, with their population, ordered by revenue decreasing, with non-zero revenue.

💡 Solution:

SELECT *
FROM (
    SELECT
        country,
        SUM(visits) AS visits,
        SUM(revenue) AS revenue,
        SUM(transactions) / SUM(visits) AS conv_rate
    FROM analytics
    GROUP BY 1
) AS sub1
LEFT JOIN countries
USING (country)
WHERE revenue > 0
ORDER BY revenue DESC
countryvisitsrevenueconv_ratepopulation
0United States30401.0116916.970.036545331611270
1Canada1992.01787.290.00853438443464
2Japan1860.01723.410.003226125410000
3Mexico610.0301.450.004918126014024
4Poland615.0136.040.00162638230000
5United Kingdom3160.0122.240.00158266796807
6Argentina343.0108.630.00583145808747
7Greece245.0100.770.00408210718565
8Taiwan1135.095.140.00176223525623
9Saudi Arabia116.085.150.00862134218169
10Ireland577.067.180.0034664977400
11Sweden413.062.600.00484310385347
12Brazil793.049.980.001261213088674
13Thailand378.042.930.00264666642442
14Colombia302.034.990.00331151049498
15Guatemala55.025.180.01818217109746
16Germany1556.023.950.00064383190556
17Switzerland482.013.990.0020758667100
18South Africa136.013.590.00735359622350

Exercise 4

👉 Question: Show daily variation of visits, in absolute and relative value.

💡 Solution:

SELECT
    date,
    visits,
    LEAD(visits) OVER (ORDER BY date) AS prev_visits,
    visits - LAG(visits) OVER (ORDER BY date) AS daily_var,
    (visits - LAG(visits) OVER (ORDER BY date)) / (LAG(visits) OVER (ORDER BY date)) AS daily_var_pct
FROM (
    SELECT
        date,
        SUM(visits) AS visits
    FROM analytics
    GROUP BY 1
) AS sub1
ORDER BY date
datevisitsprev_visitsdaily_vardaily_var_pct
02017-05-012588.02564.0NaNNaN
12017-05-022564.02588.0-24.0-0.009274
22017-05-032588.02806.024.00.009360
32017-05-042806.01960.0218.00.084235
42017-05-051960.01383.0-846.0-0.301497
52017-05-061383.01400.0-577.0-0.294388
62017-05-071400.02025.017.00.012292
72017-05-082025.02108.0625.00.446429
82017-05-092108.02024.083.00.040988
92017-05-102024.02218.0-84.0-0.039848
102017-05-112218.01932.0194.00.095850
112017-05-121932.01251.0-286.0-0.128945
122017-05-131251.01290.0-681.0-0.352484
132017-05-141290.02360.039.00.031175
142017-05-152360.03098.01070.00.829457
152017-05-163098.03117.0738.00.312712
162017-05-173117.02715.019.00.006133
172017-05-182715.02223.0-402.0-0.128970
182017-05-192223.01855.0-492.0-0.181215
192017-05-201855.01645.0-368.0-0.165542
202017-05-211645.02248.0-210.0-0.113208
212017-05-222248.02129.0603.00.366565
222017-05-232129.02406.0-119.0-0.052936
232017-05-242406.02228.0277.00.130108
242017-05-252228.02009.0-178.0-0.073982
252017-05-262009.01502.0-219.0-0.098294
262017-05-271502.01463.0-507.0-0.252364
272017-05-281463.01745.0-39.0-0.025965
282017-05-291745.02199.0282.00.192755
292017-05-302199.02292.0454.00.260172
302017-05-312292.0NaN93.00.042292