SQL·DS
SQL for Data Science — Cheatsheet IBM: Databases & SQL for Data Science with Python · 29 lectures

Basics · transcript: subtitle.txt (subtitle (1).txt is the same recording)

Course Welcome & Overview

Why data matters, why SQL is the key to modern data repositories, how Python connects to databases — and the six-module roadmap for the whole course.

Course roadmap

roadmap
# Module 1  SQL basics — DML: INSERT / SELECT / UPDATE / DELETE
# Module 2  DDL — CREATE / ALTER / DROP / TRUNCATE tables
# Module 3  Built-in functions, sub-queries, multiple tables, joins
# Module 4  Python for databases — Jupyter, DB-API, SQL Magic, EDA
# Module 5  Hands-on project on real-world data
# Module 6  Advanced — Views, Stored Procedures, ACID, Inner & Outer JOINs

Tips & tricks

  • Treat the roadmap as a checklist — after each module, tick what you can do from memory.
  • Don't skip the integrated labs: theory sticks only when you run the queries yourself.
  • SQL is the universal key; Python is the connector — different jobs, one workflow.

Basics · transcript: subtitle (2).txt

Data, Databases & RDBMS

The four foundations — data, database, DBMS/RDBMS, SQL — and the five commands that cover most of what people ever do with a database.

Key definitions

  • SQL — the language used for relational databases to query (get data out of) a database.
  • Data — a collection of facts: words, numbers, or even pictures.
  • Database — a repository of data; supports adding, modifying, and querying.
  • DBMS / RDBMS — the set of software tools controlling access, organization, and storage. Examples: MySQL, Oracle, DB2.

The five basic commands

sql · dialect-agnostic
CREATE TABLE ...    -- 1. create a table
INSERT INTO ...     -- 2. populate the table (Create)
SELECT ... FROM ... -- 3. read data               (Read)
UPDATE ... SET ...  -- 4. change data             (Update)
DELETE FROM ...     -- 5. remove rows             (Delete)

Tips & tricks

  • The five commands are identical across every RDBMS — product-specific syntax comes later (if ever).
  • In practice database ≈ DBMS ≈ database server ≈ data server; these terms are used interchangeably.

Basics · transcript: subtitle (8).txt

DDL vs DML

Every SQL statement belongs to one of two families: DDL touches structure (tables), DML touches content (rows) — the CRUD operations.

DDL — definition

CREATE tablesALTER structureTRUNCATE data onlyDROP table

Define, change, or drop database objects such as tables.

DML — manipulation

INSERTSELECTUPDATEDELETE

Read and modify data — Create, Read, Update, Delete rows.

Tips & tricks

  • Interview favorite: DELETE is DML; TRUNCATE is DDL.
  • TRUNCATE vs DELETE without WHERE: both empty a table, but truncate is quicker, more efficient, and resets storage.

Basics · transcript: subtitle (3).txt

Relational Model & ER Model

How the real world maps into tables: the relational model's data independence, ER entities → tables, attributes → columns, data types, and the PK/FK that later drive JOINs.

Concept → relational result

ER conceptRelational result
Entity (noun: person, place, thing — drawn as a rectangle)Table
Attribute (property of the entity — drawn as an oval)Column
Data valuesRows
Identifier attributePrimary key — unique row, no duplication
Attribute referencing another table's PKForeign key — the link between tables

Common data types

CHAR(n)VARCHAR(n)INTEGERDECIMALDATETIME

Tips & tricks

  • CHAR = truly fixed length (ISBN with dashes, 2-letter country codes); VARCHAR = anything human-entered (titles, names, emails).
  • Sketch the ERD first, SQL second — the joins define themselves from PK→FK links.
  • The relational model's superpower = logical + physical data independence and storage independence.

Best use cases

  • Designing a new database from requirements — model first, build second.
  • Explaining why a table's structure is the way it is.

Data Definition · transcript: subtitle (9).txt

CREATE TABLE

The most common DDL statement: table name, then column definitions — name, datatype, optional constraints — enclosed in one pair of parentheses, comma-separated.

Syntax & code

sql
-- Lookup table with a tiny fixed-length code
CREATE TABLE provinces (
    id   CHAR(2)    PRIMARY KEY NOT NULL,   -- 'AB', 'BC', ...
    name VARCHAR(24)                        -- 'ALBERTA', 'BRITISH COLUMBIA', ...
);

-- Library example with constraints
CREATE TABLE author (
    author_id  CHAR(2)      PRIMARY KEY NOT NULL,
    lastname   VARCHAR(15)  NOT NULL,
    firstname  VARCHAR(15)  NOT NULL,
    email      VARCHAR(40),
    city       VARCHAR(15),
    country    CHAR(2)
);

Tips & tricks

  • PRIMARY KEY prevents duplicates; NOT NULL makes the field required (an author must have a name).
  • Designate the primary key at creation time — retrofitting uniqueness into a populated table is painful.
  • The last column has no trailing comma — the classic beginner syntax error.

Data Definition · transcript: subtitle (10).txt

ALTER · DROP · TRUNCATE

DDL for changing an existing database: ALTER changes a table's structure, DROP removes the whole table, TRUNCATE empties all rows but keeps the table.

Syntax & code

sql
ALTER TABLE author ADD COLUMN telephone_number BIGINT;
ALTER TABLE author MODIFY telephone_number CHAR(20);   -- widen to allow "(" "+" "-"
ALTER TABLE author DROP COLUMN telephone_number;

DROP TABLE author;                 -- table + data gone

TRUNCATE TABLE author IMMEDIATE;   -- DB2: all rows, not undoable
-- MySQL / PostgreSQL: TRUNCATE TABLE author;

Tips & tricks

  • Store phone numbers as strings — ( + - break numeric columns.
  • ALTERing a populated column's datatype fails when existing data is incompatible — check first.
  • Mnemonic: TRUNCATE = fast empty; DROP = demolish. Neither takes a WHERE.

Basics · transcript: subtitle (6).txt

INSERT Statement

Populating a table — one row at a time, or several comma-separated value tuples in a single statement.

Syntax & code

sql · DML: create
INSERT INTO author (author_id, lastname, firstname, email, city, country)
VALUES ('A1', 'Chong', 'Raul', 'RFC@ca.ibm.com', 'Toronto', 'CA');

-- Multiple rows in one statement
INSERT INTO author (author_id, lastname, firstname, email, city, country)
VALUES
    ('A1', 'Chong', 'Raul', 'RFC@ca.ibm.com', 'Toronto', 'CA'),
    ('A2', 'Ahuja', 'Rav',  'RA@ca.ibm.com',  'Markham', 'CA');

Tips & tricks

  • Golden rule: number of values = number of listed columns — that's what gives every column a value.
  • Always name the columns instead of relying on positional order — your code survives schema changes.
  • Text values need quotes ('A1'); numbers don't. Batch small loads into multi-row INSERTs.

Basics · transcript: subtitle (4).txt

SELECT · WHERE · Predicates

A DBMS exists to facilitate retrieval, not just storage. SELECT returns a result set; WHERE restricts it via a predicate — a condition that evaluates to true, false, or unknown.

Syntax & code

sql · DML: read
SELECT * FROM book;                    -- 4 rows, all columns
SELECT book_id, title FROM book;       -- output column order = SELECT order
SELECT book_id, title
FROM   book
WHERE  book_id = 'B1';                 -- only rows where the predicate is TRUE

Comparison operators

=><>=<=<> / !=

Tips & tricks

  • Avoid SELECT * in production code: name columns for less traffic, self-documentation, and schema-change safety.
  • A WHERE on a CHAR key ('B1') compares strings — mind the quotes.
  • The "unknown" arm comes from NULL comparisons: NULL means we don't know, not false.

Basics · transcript: subtitle (5).txt

COUNT · DISTINCT · LIMIT

Three expressions that turn "give me everything" into fast, focused queries: count rows, remove duplicates, restrict how many rows come back.

Syntax & code

sql
SELECT COUNT(*)                 FROM tablename;                       -- all rows
SELECT COUNT(COUNTRY)           FROM MEDALS WHERE COUNTRY = 'CANADA'; -- matching rows
SELECT DISTINCT COUNTRY         FROM MEDALS WHERE MEDALTYPE = 'GOLD'; -- uniques
SELECT *                        FROM MEDALS WHERE YEAR = 2018 LIMIT 5; -- a peek

Tips & tricks

  • Explore with LIMIT, not SELECT * — a few rows answer "what kind of data is this?" almost instantly, even on millions-of-row tables.
  • COUNT(DISTINCT COUNTRY) answers "how many different X?" in one query.
  • Dialect spelling: LIMIT (MySQL/SQLite/PG), TOP 10 (SQL Server), FETCH FIRST 10 ROWS ONLY (DB2/Oracle 12c+).

Basics · transcript: subtitle (7).txt

UPDATE & DELETE

Altering and removing rows with DML. The central lesson: the WHERE clause is what saves your data — omit it and every row is updated or removed.

Syntax & code

sql · DML
SELECT * FROM author;  -- 1. preview current state

-- 2. change FIRSTNAME and LASTNAME of the author with AUTHOR_ID = 'A2'
UPDATE author
   SET lastname = 'Katta', firstname = 'Lakshmi'
 WHERE author_id = 'A2';

-- 3. remove the rows for two authors
DELETE FROM author WHERE author_id IN ('A2', 'A3');

Best use cases

  • Data correction / cleansing — mis-entered names, bulk code updates.
  • GDPR-style removal of specific records, always with a previewed condition.

Advanced Querying · transcript: subtitle (11).txt

Refining SELECT — LIKE · BETWEEN · IN

What if you don't know the exact value for the predicate? Three tools replace clumsy WHERE clauses: pattern matching, ranges, and value sets.

Syntax & code

sql
-- String pattern: % is the wildcard (0+ of any character)
SELECT firstname FROM author WHERE firstname LIKE 'R%';   -- starts with R
SELECT firstname FROM author WHERE firstname LIKE '%on%'; -- contains "on"

-- Range (inclusive) — same as pages >= 290 AND pages <= 300
SELECT title FROM book WHERE pages BETWEEN 290 AND 300;

-- Set of values — avoids a very long repeated predicate
SELECT * FROM author WHERE country IN ('CA', 'IN', 'CN');

Tips & tricks

  • Pattern position is the whole trick: 'R%' starts-with, '%R' ends-with, '%R%' contains.
  • LIKE is case-sensitive in some DBs — pair with UPPER()/LOWER() for robust matching.
  • BETWEEN is inclusive; with dates/timestamps the end boundary can silently bite — prefer half-open ranges.

Advanced Querying · transcript: subtitle (12).txt

Sorting Result Sets — ORDER BY

Results come back in no particular order. ORDER BY sorts by one or more columns — ascending by default, DESC for descending, or by column position.

Syntax & code

sql
SELECT title FROM book ORDER BY title;        -- alphabetical (ASC default)
SELECT title FROM book ORDER BY title DESC;   -- descending
SELECT title, pages FROM book ORDER BY 2;     -- sort by 2nd column in the list

Tips & tricks

  • Prefer ORDER BY columnname over positional ORDER BY 2 in anything saved — reordering the SELECT list silently retargets positional sorts.
  • Multi-column: ORDER BY country ASC, lastname DESC — later columns break ties.
  • ORDER BY runs after SELECT, so it can reference aliases; WHERE runs before and cannot.

Best use cases

  • Human-readable reporting; "top-N" queries: … ORDER BY score DESC LIMIT 5.

Advanced Querying · transcript: subtitle (13).txt

Grouping — DISTINCT · GROUP BY · HAVING

The classic "how many X per Y?" build-up: kill duplicates with DISTINCT, count per group with GROUP BY, name derived columns with AS, filter groups with HAVING.

Syntax & code — progression on the author table

sql
SELECT country FROM author ORDER BY 1;               -- 20 rows, duplicates
SELECT DISTINCT country FROM author;                 -- 6 unique countries

SELECT country, COUNT(*) AS "Count"                  -- per-country counts
FROM author GROUP BY country;

SELECT country, COUNT(*) AS "Count"                  -- keep only big groups
FROM author
GROUP BY country
HAVING COUNT(*) > 4;                                 -- → China 6, India 6

Tips & tricks

  • Mnemonic: WHERE filters rows; HAVING filters groups. Conditions with aggregates belong in HAVING.
  • Logical order: WHERE → GROUP BY → aggregates → HAVING → SELECT → ORDER BY.
  • COUNT(*) counts rows; COUNT(column) counts non-NULL values — matters on real data.
  • Use AS to name derived columns — the default heading for aggregates is meaningless junk ("2").

Advanced Querying · transcript: subtitle (14).txt

Built-in Functions

Aggregate, scalar, and string functions run data operations inside the database — less data over the wire, often faster than fetching first. Examples use the PETRESCUE table.

Aggregate functions

sql · petrescue
SELECT SUM(COST) FROM PETRESCUE;                     -- total of a column
SELECT SUM(COST) AS SUM_OF_COST FROM PETRESCUE;      -- name the output column
SELECT MAX(QUANTITY) FROM PETRESCUE;                 -- highest value
SELECT MIN(ID) FROM PETRESCUE WHERE ANIMAL = 'Dog';  -- on a subset
SELECT AVG(COST/QUANTITY) FROM PETRESCUE WHERE ANIMAL = 'Dog';  -- cost per dog

Scalar & string functions

sql
SELECT ROUND(COST) FROM PETRESCUE;                   -- round each value
SELECT LENGTH(ANIMAL) FROM PETRESCUE;                -- length per value
SELECT UPPERCASE(ANIMAL) FROM PETRESCUE;             -- UPPER() in most dialects
SELECT * FROM PETRESCUE WHERE LOWERCASE(ANIMAL) = 'cat';  -- case-safe WHERE
SELECT DISTINCT(UPPERCASE(ANIMAL)) FROM PETRESCUE;   -- nested functions

Tips & tricks

  • Push computation to the database for large data; pull raw rows only when analysis can't be expressed in SQL.
  • UPPER/LOWER in WHERE = cheap insurance against inconsistent data entry.
  • Always alias aggregate columns. Function-in-where on huge tables can prevent index use.

Advanced Querying · transcript: subtitle (15).txt

Date & Time Functions

Special date/time types plus extraction functions and date arithmetic — directly in SQL. DATE = 8 digits (YYYYMMDD), TIME = 6 (HHMMSS), TIMESTAMP = 20 (with microseconds).

Syntax & code

sql · db2 flavor
SELECT DAY(RESCUE_DATE) FROM PETRESCUE WHERE ANIMAL = 'Cat'; -- day portion
SELECT COUNT(*) FROM PETRESCUE WHERE MONTH(RESCUE_DATE) = 5; -- May sales
SELECT DATE_ADD(RESCUE_DATE, INTERVAL 3 DAY) FROM PETRESCUE; -- +3 days
SELECT CURRENT DATE - RESCUE_DATE FROM PETRESCUE;            -- elapsed since rescue
                                                             -- in yrs, months, days

Tips & tricks

  • Dialect spellings differ — DB2 CURRENT DATE, MySQL CURDATE(), SQLite DATE(rescue_date,'+3 day'), standard CURRENT_DATE.
  • MONTH(date)=5 scans every row; for big tables prefer a range on the column so indexes work.
  • BETWEEN on timestamps can miss late-Dec rows — use half-open ranges (>= start AND < next start).

Best use cases

  • Seasonality / day-of-week analysis; SLA math ("process within 3 days"); ageing queries.

Advanced Querying · transcript: subtitle (16).txt

Sub-Queries & Nested SELECTs

Sub-selects are queries wrapped in parentheses and nested inside another query — powerful because they let you evaluate aggregates where you normally can't.

Three placements

sql · employees
-- ① In the WHERE clause — the classic workaround:
--    ❌ WHERE salary > AVG(salary)  → "invalid use of aggregate function"
SELECT employee_id, firstname, lastname, salary
FROM   employees
WHERE  salary > (SELECT AVG(salary) FROM employees);

-- ② In the column list — compare each row's salary with the average
--    (naive version fails for lack of GROUP BY)
SELECT employee_id, salary,
       (SELECT AVG(salary) FROM employees) AS AVG_SALARY
FROM   employees;

-- ③ In the FROM clause — a derived table (excludes sensitive fields)
SELECT * FROM
   (SELECT employee_id, firstname, lastname, department_id
    FROM   employees) AS EMPLOYEEF;

Tips & tricks

  • Rule of thumb: aggregate value needed for a comparison → sub-query.
  • Always alias derived tables (AS EMPLOYEEF — "employee for all", non-sensitive).
  • Derived tables show their real power in multi-table work and joins.

Advanced Querying · transcript: subtitle (17).txt

Multiple Tables — Sub-queries & Implicit Joins

Ways to access more than one table in the same query: sub-queries across tables, and implicit joins (comma-separated FROM). Explicit JOINs come in lectures 18–20.

Syntax & code — EMPLOYEES × DEPARTMENTS

sql
-- Sub-query: employees whose department exists / is in location L0002
SELECT * FROM employees
WHERE  DEP_ID IN (SELECT DEP_ID_DEP FROM departments);
SELECT * FROM employees
WHERE  DEP_ID IN (SELECT DEP_ID_DEP FROM departments WHERE LOC_ID = 'L0002');

-- Reversed direction: departments of employees earning > $70,000
SELECT DEP_ID_DEP, DEP_NAME FROM departments
WHERE  DEP_ID_DEP IN (SELECT DEP_ID FROM employees WHERE salary > 70000);

-- Implicit (full) join — every row × every row; MORE rows than either table
SELECT * FROM employees, departments;

-- Implicit join filtered to matching department IDs, with aliases
SELECT E.EMP_ID, D.DEP_NAME
FROM   employees E, departments D
WHERE  E.DEP_ID = D.DEP_ID_DEP;

Tips & tricks

  • Fully qualify columns (employees.DEP_ID) — different tables can share column names; aliases make it short.
  • When the condition is exactly a.col = b.col, prefer explicit INNER JOIN … ON, so the condition is never accidentally forgotten.
  • Forgot the WHERE on a two-table FROM = accidental Cartesian explosion.

Multi-Table · transcript: subtitle12.txt

JOIN Overview

The JOIN operator combines rows from two or more tables based on a relationship between certain columns — usually a primary key in one table appearing as a foreign key in the other.

How tables link

KeyRole in a join
Primary key (icon in the ERD)Uniquely identifies each row: AUTHOR_ID, BOOK_ID, BORROWER_ID, COPY_ID
[FK] Foreign keyReferences another table's PK — e.g. LOAN.BORROWER_ID → BORROWER.BORROWER_ID

Join families

INNER JOIN — intersection onlyOUTER JOIN — plus non-matches (LEFT / RIGHT / FULL)

Tips & tricks

  • Sketch the ERD before writing joins — the key to join on is already in the diagram.
  • Three-table joins = simply add tables to the chain (BORROWER↔LOAN on borrower ID, LOAN↔COPY on copy ID).
  • Join key names don't have to match (DEP_ID vs DEP_ID_DEP) — the PK→FK relationship does.

Multi-Table · transcript: subtitle15.txt

Inner Join

The most common join: only rows whose values match in the common column appear — non-matching rows are not displayed.

Syntax & code — BORROWER ⟕ LOAN

sql
-- Who is borrowing books, and the loan date
SELECT B.BORROWER_ID, B.LASTNAME, B.COUNTRY, L.BORROWER_ID, L.LOAN_DATE
FROM   BORROWER B                       -- left table (left of the JOIN clause)
INNER JOIN LOAN L                       -- right table
   ON B.BORROWER_ID = L.BORROWER_ID;    -- the join predicate

Tips & tricks

  • INNER is the default — JOIN alone means the same; write it explicitly for readability.
  • Declare aliases once (FROM BORROWER B) and prefix every column with them, in SELECT and ON alike.
  • Fan-out is expected: if the right table has several matches per left key, the left row repeats once per match — not a bug.
  • Borrowers with no loans don't appear here; they return with OUTER JOINs (next lecture).

Multi-Table · transcript: subtitle42.txt

Outer Joins — LEFT · RIGHT · FULL

Outer joins return all the matching rows of an inner join plus unmatched rows from one or both sides — filled with NULLs where data is missing.

Syntax & code

sql
-- LEFT: all borrowers; Peters, Li, Wong show NULL loan data (never borrowed)
SELECT B.BORROWER_ID, B.LASTNAME, B.COUNTRY, L.BORROWER_ID, L.LOAN_DATE
FROM   BORROWER B
LEFT OUTER JOIN LOAN L  ON B.BORROWER_ID = L.BORROWER_ID;

-- RIGHT: all loans; one row has NULL borrower columns
--        → "a book on loan to an unknown person" = a data problem to investigate
SELECT L.BORROWER_ID, L.LOAN_DATE, B.LASTNAME, B.COUNTRY
FROM   BORROWER B
RIGHT OUTER JOIN LOAN L ON L.BORROWER_ID = B.BORROWER_ID;

-- FULL: all rows from both tables (very large result set possible)
SELECT B.BORROWER_ID, B.LASTNAME, L.LOAN_DATE
FROM   BORROWER B
FULL OUTER JOIN LOAN L ON B.BORROWER_ID = L.BORROWER_ID;

Tips & tricks

  • Direction is about position in the statement, not importance — swap sides with LEFT before reaching for RIGHT.
  • Aggregate columns from the NULL side read NULL, not zero — wrap in COALESCE(L.LOAN_DATE, 'never') for reporting.
  • MySQL & SQLite have no FULL OUTER JOIN — emulate with LEFT JOIN UNION RIGHT JOIN.
  • Treat unmatched-row NULLs as signal — RIGHT-join NULLs found a data-quality issue in the lecture.

Python Access · transcript: subtitle (18).txt

Python & Databases — Overview

Python connects to databases through APIs: the application builds SQL as text and hands it to the DBMS via API calls — connect → send → check status → fetch → disconnect.

The SQL API flow

  • 1. API call(s) connect the program to the DBMS.
  • 2. Build the SQL statement as a text string in a buffer; an API call passes it to the DBMS.
  • 3. API calls check status / handle errors; others retrieve query results.
  • 4. An API call disconnects from the database.

Proprietary APIs by database

DatabaseAPI / library
MySQLC API
PostgreSQLpsycopg2
IBM DB2ibm_db
MS SQL Serverdblib
Windows DB accessODBC
OracleOCI
Java appsJDBC
MongoDBpymongo

Tips & tricks

  • Internalize: applications never parse SQL themselves — they build text and hand it over; parsing, execution, errors, results all live in the DBMS.
  • Notebooks (Jupyter) = live code + equations + visualizations + narrative, shareable and rich-output — the course's home base.

Python Access · transcript: subtitle (19).txt

DB-API — Connections & Cursors

The Python DB-API is the standard that lets one program work with any relational database. Two main objects: the Connection (connect, commit, rollback, close) and the Cursor (run queries, scan results — like a file handle).

The canonical 5-step program

python
# 1. import the database module
import ibm_db           # (or sqlite3, psycopg2, mysql.connector, ...)

# 2. open the connection: db name, username, password
conn = ibm_db.connect(dbname, username, password)

# 3. create a cursor on the connection
cursor = conn.cursor()

# 4. run queries through the cursor, then fetch results
cursor.execute('SELECT * FROM employees')
results = cursor.fetchall()

# 5. free resources — ALWAYS
conn.close()

Connection methods

cursor()commit()rollback()close()

Tips & tricks

  • Memorize the rhythm: connect → cursor → execute → fetch → close — identical for every database Python meets.
  • Use with sqlite3.connect(...) as conn: / try…finally so close() runs even on exceptions.
  • Cursors from the same connection are not isolated; from different connections it depends on transaction implementation.
  • commit()/rollback() on the connection are the ACID story (lecture 29) surfacing in Python.

Python Access · transcript: subtitle (20).txt

SQL Magic in Jupyter

Magic commands are special notebook commands — not valid Python, but they change notebook behavior. Line magics (%) run one line; cell magics (%%) run whole cells, even in other languages.

Set-up & first query

jupyter
! pip install ipython-sql        # 1. install the dependency
%load_ext sql                    # 2. load the SQL extension
%sql sqlite://HR.db              # 3. connect (SQLite example)

%%sql                            # 4. cell magic runs whole-cell SQL
SELECT * FROM employee

Popular magics

%pwd%ls%history%reset%who / %whos%matplotlib inline%timeit / %%timeit%%bash / %%js / %%html

Tips & tricks

  • Stack multiple line magics in one cell — each on its own line.
  • %timeit times one statement; %%timeit times the entire cell — the canonical line/cell pair.
  • Use %%sql cell magic for anything longer than a line: the whole cell is SQL, so line-continuation backslashes become unnecessary (next lecture).
  • List all available magics with %lsmagic.

Python Access · transcript: subtitle (22).txt

Real-World Datasets — Hints & Tips

Real datasets arrive as CSVs (values separated by commas — sometimes semicolons). Loading them, quoting messy column names, splitting long queries, and the one habit that saves hours: LIMIT while exploring.

Syntax & code

sql + python
-- Backticks for column names with spaces/special chars (MySQL)
-- ⚠ single or double quotes DO NOT WORK for column names
SELECT `Breed`, COUNT(*) AS "Count" FROM dogs GROUP BY `Breed`;

-- Restrict result server-side (don't SELECT * then df.head() — slow)
SELECT * FROM census_data LIMIT 3;
python
# pandas reads a query straight into a DataFrame
query_statement = "SELECT `Name of Dog`, `Breed` FROM dogs LIMIT 5"
df = pd.read_sql(query_statement, conn)

# %sql line magic needs backslashes to continue lines
%sql SELECT `Breed`, COUNT(*) AS "Count" \
     FROM dogs \
     GROUP BY `Breed`
# ...but %%sql cell magic makes backslashes redundant

Quote decision tree

  • Column names that are reserved / have spaces → backticks.
  • SQL string values → single quotes (escape any inside with \).
  • Python string holding SQL → wrap in single quotes, use double quotes for quoted column names inside.

Loading CSVs (phpMyAdmin)

  • Browse the file → Format: CSV → check "first line contains the table column names" → Go.
  • MySQL auto-detects headers and generates the CREATE/INSERT statements; default table name = the file's name, lowercased.

Python Access · transcript: subtitle (23).txt

Table Metadata & Catalogs

Which table is it — dog, dogs, or four_legged_mammals? Every database has catalog tables you can query for the list of tables and their properties.

Catalogs per system

SystemCatalog
DB2SYSCAT.TABLES
SQL ServerINFORMATION_SCHEMA.TABLES
SQLite3sqlite_master
MySQLSHOW TABLES
sql · sqlite + mysql
SELECT name FROM sqlite_master WHERE type = 'table';  -- list SQLite tables
PRAGMA table_info(table_name);                        -- attributes/columns
SHOW TABLES;                                          -- MySQL: list tables
DESCRIBE table_name;                                  -- MySQL: table details

Tips & tricks

  • The recon routine for any unfamiliar DB: list tables → describe → SELECT * LIMIT 5.
  • PRAGMA table_info returns name, type, notnull, pk per column — schema info without any GUI.
  • sqlite_master.sql even holds the original CREATE statements — lost-schema recovery.

Python Access · transcript: subtitle (21).txt

Analyzing Data with Python

The module-4 capstone: a real dataset (McDonald's menu nutrition, from Kaggle) into SQLite3, back through pandas, then EDA — describe(), swarmplot, jointplot, boxplot.

The pipeline

python
import pandas as pd, sqlite3, seaborn as sns

data = pd.read_csv('mcdonalds_menu_nutrition.csv')   # 1. CSV → DataFrame
conn = sqlite3.connect('mcdonalds.db')               # 2. serverless DB
data.to_sql('MCDONALDS_NUTRITION', conn)             # 3. load as table

df = pd.read_sql('SELECT * FROM MCDONALDS_NUTRITION', conn)  # 4. back out
df.head(); df.describe()                             #    260 items, 9 categories

# 5. find the max-sodium item: visualize first, confirm numerically
sns.swarmplot(x='Category', y='Sodium', data=df)     # spots ~3,600 mg
idx  = df['Sodium'].idxmax()                         # index at max → 82
item = df.at[idx, 'Item']                            # → Chicken McNuggets (40 piece)

sns.jointplot(x='Protein', y='Total Fat', data=df)   # scatter + histograms,
                                                     # Pearson r = 0.81
sns.boxplot(df['Sugars'])                            # middle 50% + outliers

Tips & tricks

  • CSV → SQLite → pandas → seaborn: the fastest way to make any local dataset SQL-queryable with zero server setup.
  • Visualize first (the swarmplot flags the outlier), then confirm numerically (idxmax + .at).
  • Correlation lives between −1 and +1; 0.81 = strongly positive.
  • .at takes (index, column) — people instinctively flip the order and get errors.
  • Box plot: the box is the middle 50% of data; lines/points show skewness and outliers (sugar outliers ≈ 128 g).

Advanced Server · transcript: subtitle (26).txt

Views

An alternative way of representing data that exists in one or more base tables — a named results table you query like one. A view stores only its definition; it is dynamic, re-running its SELECT on every use.

Syntax & code

sql
-- Omit sensitive columns (salary, birthdate)
CREATE VIEW EMPINFO
   (EMP_ID, NAME, ADDRESS, JOB_ID, MANAGER_ID, DEPT_ID)
   AS SELECT ID, NAME, ADDRESS, JOB_ID, MGR_ID, DEPT_ID
   FROM EMPLOYEES;

SELECT * FROM EMPINFO;    -- query like a table

-- Filtered view: only one manager's rows
CREATE VIEW MGR_EMPINFO AS
   SELECT NAME, ADDRESS, JOB_ID FROM EMPLOYEES
   WHERE MANAGER_ID = 30002;

DROP VIEW EMPINFO;        -- remove completely

When to use a view

  • Omit sensitive data (tax info, birth dates, salaries).
  • Combine tables in meaningful ways; show only what one process needs.
  • Grant access to the view without granting access to the underlying tables.

Tips & tricks

  • Reads/Writes through a view touch the base tables — data lives there, the view is just the definition.
  • The defining SELECT may use WHERE / GROUP BY / HAVING and other views — but no ORDER BY, no host variables; view names up to 128 characters.
  • Views being dynamic = they track schema/data changes automatically; no refresh job.

Advanced Server · transcript: subtitle (25).txt

Stored Procedures

A set of SQL statements stored and executed on the database server — the client sends one CALL instead of many statements. Parameters in, results out.

Why

↓ network traffic (1 call)↑ performance (runs at the data)code reusesecurity (validate server-side)

Syntax & code — UPDATE_SAL(emp, rating)

sql pl
DELIMITER $$                       -- stop ';' ending the definition early

CREATE PROCEDURE UPDATE_SAL(IN EMP_NUM INT, IN THE_RATING INT)
  LANGUAGE SQL
  BEGIN
      IF (THE_RATING = 1) THEN
          UPDATE EMPLOYEES SET SALARY = SALARY * 1.10   -- 10% raise
           WHERE EMPLOYEE_ID = EMP_NUM;
      ELSE
          UPDATE EMPLOYEES SET SALARY = SALARY * 1.05   -- 5% raise
           WHERE EMPLOYEE_ID = EMP_NUM;
      END IF;
  END
$$

DELIMITER ;                        -- restore the normal delimiter

CALL UPDATE_SAL('E5001', 1);       -- invoke from SQL or an application

Tips & tricks

  • Delimiter discipline: set $$ before, restore ; after — forgetting the second line breaks everything you type next.
  • Think of procedures as the database's API: thin, stable client code even when tables change.
  • ⚠ SQL is not a fully-fledged programming language — don't move all your business logic into procedures.

Advanced Server · transcript: subtitle (24).txt

ACID Transactions

A transaction is an indivisible unit of work: every statement must complete successfully (a new stable state) — or none, leaving the database exactly as before. BEGIN → … → COMMIT or ROLLBACK.

The bank-purchase example — Rose buys $200 boots

sql
BEGIN;                                 -- implicit on Db2 on Cloud

UPDATE accounts  SET balance = balance - 200  WHERE who = 'ROSE';
UPDATE accounts  SET balance = balance + 200  WHERE who = 'SHOE_SHOP';
UPDATE inventory SET stock   = stock   - 1    WHERE item = 'Boots';

COMMIT;    -- all three succeeded → persist everything
-- OR
ROLLBACK;  -- any failed (insufficient funds) → undo all, back to before

ACID

Property
AAtomic — all changes performed successfully, or not at all
CConsistent — data consistent before and after the transaction
IIsolated — no other process can change the data while it runs
DDurable — committed changes must persist

Tips & tricks

  • In application code, tie COMMIT to the success path and ROLLBACK to the exception path — that's the whole pattern; this is what conn.commit()/conn.rollback() from lecture 22 are.
  • Don't hold transactions open while waiting on user input or slow logic — Isolated blocks everyone meanwhile.
  • SQL in Java/C/R/Python goes through database-specific APIs (JDBC, ibm_db…), often with EXEC SQL-style commands.

One page · everything at a glance

The Cheat Card

The highest-frequency syntax and habits from all 29 lectures, on one page.

SELECT skeleton (evaluation order)

sql
SELECT   cols, AGG(col) AS alias
FROM     table [alias]
WHERE    row-predicate            -- before grouping
GROUP BY group-cols
HAVING   agg-predicate            -- filters groups
ORDER BY col [ASC|DESC]
LIMIT    n;                       -- dialect: TOP / FETCH FIRST

Aggregates & friends

COUNT(*)SUM / MIN / MAX / AVGDISTINCTROUND / LENGTHUPPER / LOWERDAY / MONTHDATE_ADD(x, INTERVAL 3 DAY)CURRENT DATE

WHERE toolkit

= > < >= <= !=LIKE 'R%' · '%on' · '%R%'BETWEEN a AND bIN ('x','y')AND / OR / NOTIS NULL

JOIN families

TypeReturns
INNER JOINmatching pairs only
LEFT JOINall left rows + matches (NULLs right)
RIGHT JOINall right rows + matches (NULLs left)
FULL JOINeverything, NULLs on both sides

Join on PK ↔ FK; alias tables (B, L).

DDL quick reference

StatementEffect
CREATE TABLEnew table + columns + constraints
ALTER TABLEADD / MODIFY / DROP COLUMN
TRUNCATEempty the table (DDL, fast)
DROP TABLEremove table + data
CREATE VIEWnamed, dynamic saved query

Python access

python
conn = sqlite3.connect('x.db')
cur  = conn.cursor()
cur.execute(sql); rows = cur.fetchall()
conn.commit(); conn.close()

# or, notebook style:
# %load_ext sql  →  %sql sqlite://HR.db

Top 10 habits from the whole course

  • Write the WHERE first on every UPDATE/DELETE; preview it as a SELECT.
  • LIMIT while exploring — never SELECT * a huge table just to peek.
  • WHERE filters rows, HAVING filters groups.
  • Prefer explicit JOIN … ON over comma-joins; alias tables and qualify columns.
  • Store phone numbers as strings; pick CHAR only for truly fixed lengths.
  • Name derived columns with AS — aggregates default to meaningless headings.
  • Compare against an aggregate? That's a sub-query.
  • NULLs unmatched after outer joins are signal, not noise — investigate them.
  • Sensitive column? Serve a view, grant on the view, never the table.
  • Multi-step money/inventory changes belong in ACID transactions.