Python’s concurrency story is a bit of a Rube Goldberg machine, and understanding where to pull which lever requires knowing which gears are actually moving.
Let’s see asyncio in action. Imagine a web scraper that needs to fetch content from 100 different URLs. Doing this sequentially would take ages. Here’s how asyncio makes it fly:
import asyncio
import aiohttp
import time
async def fetch_url(session, url):
async with session.get(url) as response:
return await response.text()
async def main():
urls = [f"http://example.com/page/{i}" for i in range(100)]
async with aiohttp.ClientSession() as session:
tasks = [fetch_url(session, url) for url in urls]
results = await asyncio.gather(*tasks)
# Process results here
print(f"Fetched {len(results)} URLs.")
if __name__ == "__main__":
start_time = time.time()
asyncio.run(main())
end_time = time.time()
print(f"Total time taken: {end_time - start_time:.2f} seconds")
This code defines an asynchronous function fetch_url that fetches a single URL. The main function creates a list of these tasks and then uses asyncio.gather to run them concurrently. When one fetch_url task is waiting for a network response, the asyncio event loop switches to another task that’s ready to run. This is cooperative multitasking: tasks explicitly yield control back to the event loop.
The core problem asyncio solves is I/O-bound operations. When your Python program spends most of its time waiting for external resources (like network requests, disk reads/writes, or database queries), asyncio allows you to do other useful work during those waiting periods instead of just blocking. It’s not about making a single CPU core run faster; it’s about keeping that core busy with other tasks while one task is idle. The event loop is the conductor, orchestrating when each "musician" (coroutine) gets its turn.
The surprising thing most people miss is that asyncio doesn’t magically make your code run faster on multi-core processors by default. It’s designed to maximize the utilization of a single core by efficiently managing I/O waits. If your task is CPU-bound (heavy computation), asyncio alone won’t help much; in fact, it can add overhead. For CPU-bound work, you’d typically pair asyncio with multiprocessing or threading.
When you need to scale beyond a single CPU core, you start looking at threading and multiprocessing.
The next step in understanding Python concurrency is exploring how threading and multiprocessing differ and when to choose one over the other.