{"version":"https:\/\/jsonfeed.org\/version\/1","title":"mathspp.com feed","home_page_url":"https:\/\/mathspp.com\/blog\/tags\/itertools","feed_url":"https:\/\/mathspp.com\/blog\/tags\/itertools.json","description":"Stay up-to-date with the articles on mathematics and programming that get published to mathspp.com.","author":{"name":"Rodrigo Gir\u00e3o Serr\u00e3o"},"items":[{"title":"itertools cheatsheet","date_published":"2026-07-13T16:28:00+02:00","id":"https:\/\/mathspp.com\/blog\/itertools-cheatsheet","url":"https:\/\/mathspp.com\/blog\/itertools-cheatsheet","content_html":"

Cheatsheet with visual diagrams that explain how the iterables from itertools<\/code> work.<\/p>\n\n

This cheatsheet contains diagrams that explain how the iterables from the module itertools<\/code> work in a visual way.<\/p>\n

Download this cheatsheet<\/a><\/p>\n

\"A4<\/p>\n

\"A4<\/p>\n

Download this cheatsheet<\/a><\/p>","summary":"Cheatsheet with visual diagrams that explain how the iterables from itertools work.","date_modified":"2026-07-13T16:43:09+02:00","tags":["python","programming","itertools","modules"],"image":"\/user\/pages\/02.blog\/itertools-cheatsheet\/thumbnail.webp"},{"title":"Generalising itertools.pairwise","date_published":"2025-11-24T19:36:00+01:00","id":"https:\/\/mathspp.com\/blog\/generalising-itertools-pairwise","url":"https:\/\/mathspp.com\/blog\/generalising-itertools-pairwise","content_html":"

In this article you will learn about itertools.pairwise, how to use it, and how to generalise it.<\/p>\n\n

In this tutorial you will learn to use and generalise itertools.pairwise<\/code>.\nYou will understand what itertools.pairwise<\/code> does, how to use it, and how to implement a generalised version for when itertools.pairwise<\/code> isn't enough.<\/p>\n

itertools.pairwise<\/code><\/a><\/h2>\n

itertools.pairwise<\/code> is an iterable from the standard module itertools<\/code> that lets you access overlapping pairs of consecutive elements of the input iterable.\nThat's quite a mouthful, so let me translate:<\/p>\n

\n

You give pairwise<\/code> an iterable, like \"ABCD\"<\/code>, and pairwise<\/code> gives you pairs back, like (\"A\", \"B\")<\/code>, (\"B\", \"C\")<\/code>, and (\"C\", \"D\")<\/code>.<\/p>\n<\/blockquote>\n

In loops, it is common to unpack the pairs directly to perform some operation on both values.\nThe example below uses pairwise<\/code> to determine how the balance of a bank account changed based on the balance history:<\/p>\n

from itertools import pairwise  # Python 3.10+\n\nbalance_history = [700, 1000, 800, 750]\n\nfor before, after in pairwise(balance_history):\n    change = after - before\n    print(f\"Balance changed by {change:+}.\")<\/code><\/pre>\n
Balance changed by +300.\nBalance changed by -200.\nBalance changed by -50.<\/code><\/pre>\n
How to implement pairwise<\/code><\/a><\/h2>\n
If you had to implement pairwise<\/code>, you might think of something like the code below:<\/p>\n
def my_pairwise(iterable):\n    for prev_, next_ in zip(iterable, iterable[1:]):\n        yield (prev_, next_)<\/code><\/pre>\n
Which directly translates to<\/p>\n
def my_pairwise(iterable):\n    yield from zip(iterable, iterable[1:])<\/code><\/pre>\n
But there is a problem with this implementation, and that is the slicing operation.\npairwise<\/code> is supposed to work with any iterable and not all iterables are sliceable.\nFor example, files are iterables but are not sliceable.<\/p>\n
There are a couple of different ways to fix this but my favourite uses collections.deque<\/code> with its parameter maxlen<\/code><\/a>:<\/p>\n
from collections import deque\nfrom itertools import islice\n\ndef my_pairwise(data):\n    data = iter(data)\n    window = deque(islice(data, 1), maxlen=2)\n    for value in data:\n        window.append(value)\n        yield tuple(window)<\/code><\/pre>\n
Generalising itertools.pairwise<\/code><\/a><\/h2>\n
pairwise<\/code> will always produce pairs<\/strong> of consecutive elements, but sometimes you might want tuples of different sizes.\nFor example, you might want something like “triplewise<\/code>”, to get triples of consecutive elements, but pairwise<\/code> can't be used for that.\nSo, how do you implement that generalisation?<\/p>\n
In the upcoming subsections I will present different ways of implementing the function nwise(iterable, n)<\/code> that accepts an iterable and a positive integer n<\/code> and produces overlapping tuples of n<\/code> elements taken from the given iterable.<\/p>\n
Some example applications:<\/p>\n
nwise(\"ABCD\", 2) -> (\"A\", \"B\"), (\"B\", \"C\"), (\"C\", \"D\")\nnwise(\"ABCD\", 3) -> (\"A\", \"B\", \"C\"), (\"B\", \"C\", \"D\")\nnwise(\"ABCD\", 4) -> (\"A\", \"B\", \"C\", \"D\")<\/code><\/pre>\n
Using deque<\/code><\/a><\/h3>\n
The implementation of pairwise<\/code> that I showed above can be adapted for nwise<\/code>:<\/p>\n
from collections import deque\nfrom itertools import islice\n\ndef nwise(iterable, n):\n    iterable = iter(iterable)\n    window = deque(islice(iterable, n - 1), maxlen=n)\n    for value in iterable:\n        window.append(value)\n        yield tuple(window)<\/code><\/pre>\n
Note that you have to change maxlen=2<\/code> to maxlen=n<\/code>, but also islice(iterable, 1)<\/code> to islice(iterable, n - 1)<\/code>.<\/p>\n
Using tee<\/code><\/a><\/h3>\n
Another fundamentally different way of implement nwise<\/code> is by using itertools.tee<\/code> to split the input...<\/p>","summary":"In this article you will learn about itertools.pairwise, how to use it, and how to generalise it.","date_modified":"2026-07-13T16:40:24+02:00","tags":["programming","python","algorithms","itertools"],"image":"\/user\/pages\/02.blog\/generalising-itertools-pairwise\/thumbnail.webp"},{"title":"Module itertools overview","date_published":"2024-07-23T00:00:00+02:00","id":"https:\/\/mathspp.com\/blog\/module-itertools-overview","url":"https:\/\/mathspp.com\/blog\/module-itertools-overview","content_html":"
This article briefly describes the iterators available in the Python module itertools and how to use them.<\/p>\n\n
The Python module itertools<\/code> contains 20 tools that every Python developer should be aware of.\nWe divide the iterators from the module itertools<\/code> in 5 categories to make it easier to learn them and we also present a short list of the generally most useful ones.<\/p>\n
All the iterators from itertools<\/code><\/a><\/h2>\n
Category<\/th>\nIterators<\/th>\n<\/tr><\/thead>
Reshaping iterators<\/a><\/td>\nbatched<\/code>, chain<\/code>*, groupby<\/code>, islice<\/code>, pairwise<\/code>*<\/td>\n<\/tr>
Filtering iterators<\/a><\/td>\ncompress<\/code>, dropwhile<\/code>, filterfalse<\/code>, takewhile<\/code><\/td>\n<\/tr>
Combinatorial iterators<\/a><\/td>\ncombinations<\/code>, combinations_with_replacement<\/code>, permutations<\/code>, product<\/code>*<\/td>\n<\/tr>
Infinite iterators<\/a><\/td>\ncount<\/code>, cycle<\/code>, repeat<\/code><\/td>\n<\/tr>
Iterators that complement other tools<\/a><\/td>\naccumulate<\/code>, starmap<\/code>, zip_longest<\/code><\/td>\n<\/tr><\/tbody><\/table>
On top of the 19 iterators listed in the table above, the module itertools<\/code> also provides the function tee<\/code>, which is very powerful but often not necessary.\nIn “The little book of itertools<\/code>”<\/a> I devote a small chapter to it because understanding and reimplementing tee<\/code> is an excellent learning and coding exercise.<\/p>\n
The 3 most useful iterators from itertools<\/code><\/a><\/h2>\n
In my experience, the 3 more commonly useful tools in the module itertools<\/code> are product<\/code>, chain<\/code>, and pairwise<\/code>.<\/p>\n
product<\/code> flattens nested loops<\/a><\/h3>\n
The iterator product<\/code> is a combinatorial iterator<\/a> that is very useful when you want to flatten a series of nested for<\/code> loops.\nAs the prototypical example, a nested loop that traverses a two-dimensional grid can be rewritten as a single loop with product<\/code>.<\/p>\n
So, whenever we have two or more independent, nested for<\/code> loops, like below:<\/p>\n
for x in range(width):\n    for y in range(height):\n        # Do stuff...<\/code><\/pre>\n
We can reshape them into a single loop if we use product<\/code>:<\/p>\n
from itertools import product\n\nfor x, y in product(range(width), range(height)):\n    # Do stuff...<\/code><\/pre>\n
The flat structure gives you more horizontal space to write your code and makes it easier to manage breaking out of your loop.<\/p>\n
This is a very common use case for product<\/code>.\nIf you go back to old code of yours, I am sure you will be able to find places where you could rewrite a loop like this.<\/p>\n
chain<\/code> creates a single iterable out of many<\/a><\/h3>\n
The iterator chain<\/code> lets you chain two or more iterables together, so that you can traverse them in sequence without having to add them explicitly.<\/p>\n
When you are dealing with iterables like lists or strings, you might argue that you would rather spend the time doing the addition instead of having to import chain<\/code>, but this doesn't always work.<\/p>\n
Consider this snippet of code that concatenates two lists so that we can traverse them:<\/p>\n
# Typical pattern:\nfirst_list = [...]\nsecond_list = [...]\nfull_list = first_list + second_list  # + third_list + ...\nfor element in full_list:\n    # Do stuff<\/code><\/pre>\n
Using chain<\/code>, we wouldn't need the addition:<\/p>\n
from itertools import chain\n\nfirst_list = [...]\nsecond_list = [...]\nfor element in chain(first_list, second_list):  # Also works with 3+ iterables.\n    # Do stuff<\/code><\/pre>\n
This...<\/p>","summary":"This article briefly describes the iterators available in the Python module itertools and how to use them.","date_modified":"2026-07-13T16:40:24+02:00","tags":["modules","programming","python","itertools"],"image":"\/user\/pages\/02.blog\/module-itertools-overview\/thumbnail.webp"},{"title":"itertools.batched","date_published":"2023-10-17T00:00:00+02:00","id":"https:\/\/mathspp.com\/blog\/itertools-batched","url":"https:\/\/mathspp.com\/blog\/itertools-batched","content_html":"
Learn how batched<\/code> from the module itertools<\/code> works, example use cases, and how to implement it.<\/p>\n\n
The module itertools<\/code> introduced a new tool called batched<\/code> in Python 3.12.\nitertools.batched<\/code> lets you iterate over an iterable by going over portions of that iterable – or batches – that all have the same size, except possibly for the last one.\nSome example use cases<\/a> include batching API requests or batching data processing.<\/p>\n
As a dummy example, consider the snippet below:<\/p>\n
>>> from itertools import batched\n>>> for batch in batched(\"Hello, world!\", 3):\n...     print(batch)\n...\n('H', 'e', 'l')\n('l', 'o', ',')\n(' ', 'w', 'o')\n('r', 'l', 'd')\n('!',)<\/code><\/pre>\n
Notice how the last batch is a tuple with the single character, \"!\"<\/code>.\nThat's because the original input string had length 13 and I asked for batches of size 3, so batched<\/code> served me with four batches of size 3 and the final batch contained the remaining character.<\/p>\n
Example use cases<\/a><\/h2>\n
In this section I will show you a couple of example use cases for itertools.batched<\/code>.\nIf you know of any other good use cases, feel free to leave a comment below or to reach out to me<\/a> and I'll include them here.<\/p>\n
Go over a file in chunks<\/a><\/h3>\n
If you have a huge file that you need to go over, but you don't need to read the whole file at once, you can use batched<\/code> to go over a set number of lines at a time.\nThe code would look like this:<\/p>\n
from itertools import batched\n\nwith open(some_file, \"r\") as file:\n    for chunk in batched(file, 25):\n        process_chunk_of_lines(chunk)<\/code><\/pre>\n
This could work well if you were looking for some specific line in the file, for example, and you couldn't hold the whole file in memory.\nOf course, in that case, you could probably increase the batch size to something bigger than 25<\/code>.<\/p>\n
Chunking a response over a socket<\/a><\/h3>\n
In contexts like socket communications, it is common to have to chunk your response to a maximum size, so if your message is bigger than some limit, you have to send it in chunks.\nThe code would look something like this:<\/p>\n
from itertools import batched\n\nfor chunk in batched(raw_data, 1024):\n    sent = socket.send(b\"\".join(chunk))\n    if sent < len(chunk):\n        # Handle the fact that not all data was sent.<\/code><\/pre>\n
In this case, you may need to send strings (or bytes) over the socket, and that is why we do b\"\".join(...)<\/code>, because batched<\/code> returns tuples with the elements.<\/p>\n
Iterating over substrings<\/a><\/h3>\n
If you need to use batched<\/code> to split a string, but the final thing you need is substrings, you can use batched<\/code> together with \"\".join<\/code> and map<\/code> to create an iterator that produces substrings of a given length:<\/p>\n
map(\"\".join, batched(string, length))<\/code><\/pre>\n
Here is an example:<\/p>\n
>>> from itertools import batched\n>>> hello_world_substrings = map(\"\".join, batched(\"Hello, world!\", 3))\n>>> for substring in hello_world_substrings:\n...     print(substring)\n...\nHel\nlo,\n wo\nrld\n!<\/code><\/pre>\n
This could also work well with the socket example<\/a> from above.<\/p>\n
Another example where...<\/p>","summary":"Learn how batched from the module itertools works, example use cases, and how to implement it.","date_modified":"2026-07-13T16:40:24+02:00","tags":["dunder methods","generators","modules","programming","python","itertools"],"image":"\/user\/pages\/02.blog\/itertools-batched\/thumbnail.webp"}]}