← Back to Python series
🚀
Advanced
Annotations · TypeVar · Generic · dataclass · Protocol · functools.wraps · Stacking · Class decorators · __enter__ · __exit__ · __iter__ · __next__ · yield · yield from · send()

Weeks 1–3 — Type Hints & Dataclasses · Decorators & Context Managers · Iterators & Generators

Add static type information to Python code with type hints, use TypeVar for generic functions, model data with @dataclass, and define structural subtypes with Protocol. Master decorators for cross-cutting concerns like logging, caching, and timing. Then implement reusable context managers with both the class-based and contextlib.contextmanager approaches. Implement the iterator protocol from scratch, write memory-efficient generators with yield, compose pipelines with yield from, and use generator's send() for two-way communication.

typingtype hintsdataclassProtocolGenericdecoratorcontext managerfunctoolscontextlibwrapsiteratorgeneratoryieldyield fromprotocol
Duration
⏱ 2.5 hours + 2.5 hours + 2.5 hours
Level
📊 Advanced
Prerequisite
🎯 Intermediate complete
OUTCOME
Write a type-annotated data pipeline that passes mypy --strict / Write a @retry decorator and a managed_connection context manager / Build a lazy CSV reader and a coroutine-based data pipeline

What you'll learn

  • 1Annotate function signatures and variables
  • 2Use Optional, Union, List, Dict, Tuple from typing
  • 3Write generic functions with TypeVar
  • 4Model immutable records with frozen dataclasses
  • 5Define structural contracts with Protocol
  • 6Write a decorator factory that accepts arguments
  • 7Stack multiple decorators correctly
  • 8Use functools.wraps to preserve metadata
  • 9Implement __enter__ and __exit__ for a context manager class
  • 10Use @contextmanager for generator-based context managers
  • 11Implement __iter__ and __next__ in a custom class
  • 12Distinguish iterables from iterators
  • 13Write generator functions and use them in pipelines
  • 14Delegate to sub-generators with yield from
  • 15Use send() and throw() for coroutine communication

Week 1 — Type Hints & Dataclasses

Add static type information to Python code with type hints, use TypeVar for generic functions, model data with @dataclass, and define structural subtypes with Protocol.

1. Type Hints

python
from typing import Optional, Union

def greet(name: str, times: int = 1) -> str:
    return (name + "\n") * times

def find_user(user_id: int) -> Optional[dict]:
    db = {1: {"name": "Alice"}}
    return db.get(user_id)

# Union (Python 3.10+: use | directly)
def stringify(value: Union[int, float, str]) -> str:
    return str(value)
ℹ️

Type hints don't affect runtime behaviour — Python still runs without them. Use mypy or pyright for static checking.

2. Generic Functions

python
from typing import TypeVar, List

T = TypeVar("T")

def first(lst: List[T]) -> Optional[T]:
    return lst[0] if lst else None

print(first([1, 2, 3]))       # 1  — inferred as int
print(first(["a", "b"]))     # "a" — inferred as str

3. Dataclasses

python
from dataclasses import dataclass, field
from typing import List

@dataclass(frozen=True)  # immutable
class Point:
    x: float
    y: float

@dataclass
class Team:
    name: str
    members: List[str] = field(default_factory=list)

    def add(self, member: str) -> None:
        self.members.append(member)

t = Team("Alpha")
t.add("Alice")
t.add("Bob")
print(t)  # Team(name='Alpha', members=['Alice', 'Bob'])

4. Protocol

python
from typing import Protocol, runtime_checkable

@runtime_checkable
class Drawable(Protocol):
    def draw(self) -> str: ...

class Circle:
    def draw(self) -> str: return "O"

class Square:
    def draw(self) -> str: return "[]"

def render(shape: Drawable) -> None:
    print(shape.draw())

render(Circle())   # O
render(Square())   # []

Week 2 — Decorators & Context Managers

Master decorators for cross-cutting concerns like logging, caching, and timing. Then implement reusable context managers with both the class-based and contextlib.contextmanager approaches.

1. Basic Decorators

python
import functools, time

def timer(func):
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        start = time.perf_counter()
        result = func(*args, **kwargs)
        elapsed = time.perf_counter() - start
        print(f"{func.__name__} took {elapsed:.4f}s")
        return result
    return wrapper

@timer
def slow_sum(n):
    return sum(range(n))

print(slow_sum(1_000_000))

2. Decorator Factories (with arguments)

python
import functools

def retry(max_attempts=3, exceptions=(Exception,)):
    def decorator(func):
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            for attempt in range(1, max_attempts + 1):
                try:
                    return func(*args, **kwargs)
                except exceptions as e:
                    if attempt == max_attempts:
                        raise
                    print(f"  Attempt {attempt} failed: {e}. Retrying...")
        return wrapper
    return decorator

@retry(max_attempts=3, exceptions=(ValueError,))
def risky():
    import random
    if random.random() < 0.7:
        raise ValueError("unlucky")
    return "success"

3. Context Managers

python
from contextlib import contextmanager
import time

@contextmanager
def timer_ctx(label=""):
    start = time.perf_counter()
    try:
        yield
    finally:
        elapsed = time.perf_counter() - start
        print(f"{label} {elapsed:.4f}s")

with timer_ctx("matrix multiply:"):
    result = sum(i*j for i in range(1000) for j in range(100))

Week 3 — Iterators & Generators

Implement the iterator protocol from scratch, write memory-efficient generators with yield, compose pipelines with yield from, and use generator's send() for two-way communication.

1. Iterator Protocol

python
class Counter:
    def __init__(self, start, stop):
        self.current = start
        self.stop = stop

    def __iter__(self):
        return self

    def __next__(self):
        if self.current >= self.stop:
            raise StopIteration
        val = self.current
        self.current += 1
        return val

for n in Counter(1, 5):
    print(n, end=' ')   # 1 2 3 4

2. Generator Functions

python
def range_gen(start, stop, step=1):
    current = start
    while current < stop:
        yield current
        current += step

# Same as above but lazy — no list created
for n in range_gen(0, 10, 2):
    print(n, end=' ')   # 0 2 4 6 8

# yield from
def chain(*iterables):
    for it in iterables:
        yield from it

print(list(chain([1,2], [3,4], [5])))  # [1,2,3,4,5]

3. Coroutines with send()

python
def accumulator():
    total = 0
    while True:
        value = yield total
        if value is None:
            break
        total += value

acc = accumulator()
next(acc)           # prime the generator
print(acc.send(10)) # 10
print(acc.send(20)) # 30
print(acc.send(5))  # 35

💻 Examples

Run these examples and check the output yourself.

01_typed_pipeline.py— Type-annotated data processing pipeline
CODE
from dataclasses import dataclass
from typing import List, Optional

@dataclass
class Record:
    name: str
    score: float
    active: bool = True

def load(raw: List[dict]) -> List[Record]:
    return [Record(**r) for r in raw]

def top_n(records: List[Record], n: int = 3) -> List[Record]:
    active = [r for r in records if r.active]
    return sorted(active, key=lambda r: r.score, reverse=True)[:n]

raw = [
    {"name": "Alice", "score": 92.5},
    {"name": "Bob",   "score": 87.0},
    {"name": "Carol", "score": 95.1, "active": False},
    {"name": "Dave",  "score": 88.3},
]
records = load(raw)
for r in top_n(records):
    print(f"{r.name}: {r.score}")
▶ Output
Alice: 92.5
Dave: 88.3
Bob: 87.0
01_decorators.py— Composable utility decorators
CODE
import functools, time, logging
logging.basicConfig(level=logging.INFO)

def log_call(func):
    @functools.wraps(func)
    def wrapper(*a, **kw):
        logging.info(f"Calling {func.__name__}({a}, {kw})")
        result = func(*a, **kw)
        logging.info(f"{func.__name__} returned {result!r}")
        return result
    return wrapper

def memoize(func):
    cache = {}
    @functools.wraps(func)
    def wrapper(*a):
        if a not in cache:
            cache[a] = func(*a)
        return cache[a]
    return wrapper

@log_call
@memoize
def fib(n):
    if n <= 1: return n
    return fib(n-1) + fib(n-2)

print(fib(10))
01_lazy_csv.py— Memory-efficient lazy CSV reader generator
CODE
import csv
from pathlib import Path

def lazy_csv(path, batch_size=100):
    """Yield rows from a CSV file in batches."""
    with open(path, newline='', encoding='utf-8') as f:
        reader = csv.DictReader(f)
        batch = []
        for row in reader:
            batch.append(row)
            if len(batch) >= batch_size:
                yield batch
                batch = []
        if batch:
            yield batch

# Process a large file without loading it all
for batch in lazy_csv('data.csv'):
    print(f'Processing {len(batch)} rows...')
    # process batch here

📝 Exercises

Try them yourself first, then open the solution to compare.

Exercise 1

Type-safe Stack

Goal: Implement a generic Stack[T] class that passes mypy.

Requirements
  • Generic[T] base class
  • push(item: T), pop() -> T, peek() -> T, is_empty() -> bool
  • Raises IndexError on pop/peek from empty stack
  • Fully annotated
▶Toggle solution
SOLUTION
from typing import Generic, TypeVar, List

T = TypeVar("T")

class Stack(Generic[T]):
    def __init__(self) -> None:
        self._data: List[T] = []

    def push(self, item: T) -> None:
        self._data.append(item)

    def pop(self) -> T:
        if self.is_empty(): raise IndexError("Stack is empty")
        return self._data.pop()

    def peek(self) -> T:
        if self.is_empty(): raise IndexError("Stack is empty")
        return self._data[-1]

    def is_empty(self) -> bool:
        return len(self._data) == 0

s: Stack[int] = Stack()
s.push(1); s.push(2); s.push(3)
print(s.pop(), s.pop())
▶ Output
3 2
Exercise 2

@validate_types decorator

Goal: Write a decorator that checks argument types at runtime using function annotations.

Requirements
  • Reads __annotations__ from the wrapped function
  • Raises TypeError with helpful message if type doesn't match
  • Handles missing annotations gracefully (skip check)
▶Toggle solution
SOLUTION
import functools

def validate_types(func):
    hints = func.__annotations__
    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        import inspect
        sig = inspect.signature(func)
        bound = sig.bind(*args, **kwargs)
        for param, val in bound.arguments.items():
            if param in hints and param != "return":
                expected = hints[param]
                if not isinstance(val, expected):
                    raise TypeError(f"{param} must be {expected.__name__}, got {type(val).__name__}")
        return func(*args, **kwargs)
    return wrapper

@validate_types
def add(a: int, b: int) -> int:
    return a + b

print(add(3, 4))       # 7
try:
    add(3, "4")
except TypeError as e:
    print(e)           # b must be int, got str
▶ Output
7
b must be int, got str
Exercise 3

Infinite sequence generators

Goal: Write three infinite generators: naturals(), primes(), and fibonacci().

Requirements
  • Each yields values indefinitely
  • Use itertools.islice to take N values
  • All work correctly with next() and for loops
▶Toggle solution
SOLUTION
from itertools import islice

def naturals(start=1):
    n = start
    while True:
        yield n
        n += 1

def primes():
    def is_prime(n):
        if n < 2: return False
        return all(n % i for i in range(2, int(n**0.5)+1))
    for n in naturals(2):
        if is_prime(n):
            yield n

def fibonacci():
    a, b = 0, 1
    while True:
        yield a
        a, b = b, a+b

print(list(islice(naturals(), 5)))
print(list(islice(primes(), 10)))
print(list(islice(fibonacci(), 10)))
▶ Output
[1, 2, 3, 4, 5]
[2, 3, 5, 7, 11, 13, 17, 19, 23, 29]
[0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
Example code / lecture materials

All lecture materials and example code are openly available on GitHub.

View on GitHub ↗

Comments 0

No comments yet. Be the first.

Only members can comment.

Comments with ads, abuse, personal information or illegal content are hidden or removed. Comments are shown with your nickname. Privacy Policy