Week 5 — Asyncio

Write asynchronous programs with asyncio. Understand coroutines, tasks, the event loop, and use aiohttp for non-blocking HTTP requests.

asyncioasyncawaitcoroutineaiohttpevent loop

Duration

⏱ 3 hours

Level

📊 Advanced

Prerequisite

🎯 Week 4

OUTCOME

Build an async web scraper that fetches 50 URLs concurrently

What you'll learn

1Define coroutines with async def and await
2Run coroutines with asyncio.run() and create tasks
3Fetch multiple URLs concurrently with asyncio.gather
4Limit concurrency with asyncio.Semaphore
5Use asyncio.Queue for async producer-consumer

1. Async Basics

python

import asyncio

async def greet(name, delay):
    await asyncio.sleep(delay)
    print(f"Hello, {name}!")

async def main():
    # Run concurrently
    await asyncio.gather(
        greet("Alice", 2),
        greet("Bob", 1),
        greet("Carol", 0.5),
    )
    # Prints Carol, Bob, Alice (in order of completion)

asyncio.run(main())

2. Tasks & Cancellation

python

import asyncio

async def background_task():
    for i in range(100):
        print(f"  bg: step {i}")
        await asyncio.sleep(0.1)

async def main():
    task = asyncio.create_task(background_task())
    await asyncio.sleep(0.3)   # let it run for 0.3s
    task.cancel()
    try:
        await task
    except asyncio.CancelledError:
        print("Task cancelled")

asyncio.run(main())

3. aiohttp

python

import asyncio, aiohttp

async def fetch(session, url):
    async with session.get(url) as resp:
        return resp.status, len(await resp.text())

async def main(urls):
    async with aiohttp.ClientSession() as session:
        tasks = [fetch(session, u) for u in urls]
        results = await asyncio.gather(*tasks, return_exceptions=True)
    for url, result in zip(urls, results):
        if isinstance(result, Exception):
            print(f"FAIL {url}: {result}")
        else:
            status, size = result
            print(f"{status} {url} ({size} chars)")

💻 Examples

Run these examples and check the output yourself.

01_async_gather.py— Concurrent API simulation with gather

CODE

import asyncio, random

async def fetch_data(item_id: int) -> dict:
    delay = random.uniform(0.1, 0.5)
    await asyncio.sleep(delay)
    return {"id": item_id, "value": item_id * 10, "delay": round(delay, 2)}

async def main():
    import time
    start = time.perf_counter()
    results = await asyncio.gather(*[fetch_data(i) for i in range(1, 11)])
    elapsed = time.perf_counter() - start
    for r in results:
        print(f"  id={r['id']} value={r['value']} fetched_in={r['delay']}s")
    print(f"Total time: {elapsed:.2f}s (sequential would be ~{sum(r['delay'] for r in results):.2f}s)")

asyncio.run(main())

📝 Exercises

Try them yourself first, then open the solution to compare.

Exercise 1

Rate-limited async downloader

Goal: Fetch N URLs concurrently but limit to max 5 simultaneous requests using asyncio.Semaphore.

Requirements

Use asyncio.Semaphore(5) to limit concurrency
Collect results in order (use tasks, not gather directly)
Print status code and response size for each URL

▶Toggle solution

SOLUTION

import asyncio
import aiohttp

async def fetch(session, url, sem):
    async with sem:
        try:
            async with session.get(url, timeout=aiohttp.ClientTimeout(total=10)) as r:
                body = await r.text()
                return url, r.status, len(body)
        except Exception as e:
            return url, 0, 0

async def main(urls):
    sem = asyncio.Semaphore(5)
    async with aiohttp.ClientSession() as session:
        tasks = [fetch(session, u, sem) for u in urls]
        for coro in asyncio.as_completed(tasks):
            url, status, size = await coro
            print(f"[{status}] {url} ({size} chars)")

urls = [f"https://httpbin.org/delay/{i%3}" for i in range(10)]
asyncio.run(main(urls))

Example code / lecture materials

All lecture materials and example code are openly available on GitHub.

View on GitHub ↗