Collections and Itertools
Express common counting, queueing, grouping, and streaming tasks with tools that match the data flow.
By the end, you’ll be able to…
- Use Counter, defaultdict, and deque for suitable workloads.
- Recognize lazy iterators and one-pass consumption.
- Apply grouping only after establishing the required ordering.
01Choose a collection from its operations
A plain dictionary and list solve many tasks, but specialized containers can state your intent more directly. Counter tallies hashable values, defaultdict creates a missing value through a factory, and deque efficiently adds and removes items at either end. Choose by the operations your workload repeatedly performs rather than by which import looks sophisticated.
A list is appropriate for indexed access and appending at the right. Removing its first item repeatedly shifts the remaining items. A deque fits a first-in, first-out work queue: append new work and popleft the oldest. A bounded deque is also convenient for retaining only the most recent events.
02Factories remove repetitive initialization
defaultdict(list) is useful when accumulating values under a key: accessing an absent key creates and stores a new list. Pass the factory itself, not list(). A normal dict with setdefault or an explicit conditional may be clearer when creating a missing key is unusual or should be tightly controlled.
Be aware that reading an absent key through brackets on a defaultdict changes the dictionary. get does not invoke its factory. That difference matters when inspection should not create empty groups. Convert to a regular dictionary at an interface boundary when callers should not inherit implicit creation behavior.
03Itertools builds lazy pipelines
Many itertools functions return iterators, producing values only when requested. islice can take a finite window without copying an entire source, chain joins input streams, and pairwise provides adjacent pairs. This is useful for large or even unbounded inputs, but it changes how you reason about reuse.
An iterator is generally consumed as you traverse it. If you print list(iterator) to inspect it, a later loop may see nothing. Materialize a small result when you truly need multiple passes; otherwise build a fresh pipeline for each pass. Avoid list() on an unbounded source such as count().
04Grouping depends on adjacency
itertools.groupby groups consecutive equal keys, not every matching key anywhere in the input. If you want one group per category, sort by the same key first or accumulate into a dictionary. Sorting costs memory and time; dictionary accumulation may be a better fit when order does not matter.
Each group iterator shares its underlying source with groupby. Consume a group before advancing the outer iterator, or materialize it if it must be retained. When designing a pipeline, ask whether the input is already ordered, whether it can fit in memory, and whether the result must be revisited. Those constraints determine which concise expression is actually correct.
The ideas worth keeping.
- Select containers by their repeated operations.
- Treat lazy iterators as consumable streams.
- Remember that groupby groups adjacent equal keys.
Make it yours.
Type and test your code right here in the interactive workspace. Run your solution, check the output, and compare with the worked guide when ready.
Retain a bounded event history
Keep only the last three events from an iterable using deque. Return a list in chronological order and test five incoming events.
Explore a worked solution
from collections import dequedef recent_events(events): return list(deque(events, maxlen=3))assert recent_events(["a", "b", "c", "d", "e"]) == ["c", "d", "e"]assert recent_events([]) == []A bounded deque discards the oldest entries as new ones arrive. Its storage stays bounded even if the input stream is much larger.
Find adjacent rises
Use pairwise to return each increase between consecutive temperatures. Do not compare the last reading with the first.
Explore a worked solution
from itertools import pairwisedef rises(readings): return [current - previous for previous, current in pairwise(readings) if current > previous]assert rises([20, 23, 22, 26]) == [3, 4]assert rises([20]) == []assert rises([]) == []pairwise yields only real adjacent pairs, naturally producing no pairs for zero or one reading.
What stayed with you?
Check the ideas while they’re fresh. Your answers stay in this page and feedback appears as you go.
Retain a bounded event history
Keep only the last three events from an iterable using deque. Return a list in chronological order and test five incoming events.
[Process exited with code 0 - No output]
Keep exploring.
The official references behind this chapter, when you want a second angle or the full details.
One chapter closer.
Reading is a start. Trying it yourself makes the connection.