This article proposes an implementation of an ergonomic binary search algorithm implemented as a bidirectional generator.<\/p>\n\n
Python generators can be bidirectional and in this article I will use this feature to try to implement the binary search<\/a> algorithm in an ergonomic way.\nTo this end, I'll write a bidirectional generator that implements the binary search algorithm independent of any use cases.\nThis means the resulting generator can be used ergonomically whenever a binary search needs to be employed.<\/p>\n Python generators support the method The snippet below shows how the method Note that the Because of the way the method If you abstract the algorithm away from its concrete applications, the binary search algorithm produces a sequence of integer guesses that are candidates to satisfy some criteria.\nSince binary search produces these values consecutively, and it doesn't need to generate every possible value ahead of time, it makes sense to consider a generator as the tool to implement binary search.<\/p>\n Additionally, depending on the result of an evaluation that is performed on a guess produced by the algorithm, the next guess might be above or below the previous guess.\nThis is why the generator needs to be bidirectional, since the user will inform the generator on whether guesses are currently too high or too low.<\/p>\n To inform the binary search generator of whether the guesses need to be increased or decreased, we will send the following integers to the binary search:<\/p>\n With this in mind, here is an example usage of the generator I am proposing:<\/p>\n The...<\/p>","summary":"This article proposes an implementation of an ergonomic binary search algorithm implemented as a bidirectional generator.","date_modified":"2025-07-23T16:49:02+02:00","tags":["algorithms","generators","programming","python"],"image":"\/user\/pages\/02.blog\/binary-search-as-a-bidirectional-generator\/thumbnail.webp"},{"title":"itertools.batched","date_published":"2023-10-17T00:00:00+02:00","id":"https:\/\/mathspp.com\/blog\/itertools-batched","url":"https:\/\/mathspp.com\/blog\/itertools-batched","content_html":" Learn how The module As a dummy example, consider the snippet below:<\/p>\n Notice how the last batch is a tuple with the single character, In this section I will show you a couple of example use cases for If you have a huge file that you need to go over, but you don't need to read the whole file at once, you can use This could work well if you were looking for some specific line in the file, for example, and you couldn't hold the whole file in memory.\nOf course, in that case, you could probably increase the batch size to something bigger than In contexts like socket communications, it is common to have to chunk your response to a maximum size, so if your message is bigger than some limit, you have to send it in chunks.\nThe code would look something like this:<\/p>\n In this case, you may need to send strings (or bytes) over the socket, and that is why we do If you need to use Here is an example:<\/p>\n This could also work well with the socket example<\/a> from above.<\/p>\n Another example where...<\/p>","summary":"Learn how batched from the module itertools works, example use cases, and how to implement it.","date_modified":"2025-07-23T16:49:02+02:00","tags":["dunder methods","generators","modules","programming","python"],"image":"\/user\/pages\/02.blog\/itertools-batched\/thumbnail.webp"},{"title":"TIL #071 \u2013 generator method close","date_published":"2023-07-21T23:00:00+02:00","id":"https:\/\/mathspp.com\/blog\/til\/generator-method-close","url":"https:\/\/mathspp.com\/blog\/til\/generator-method-close","content_html":" Today I learned about the generator method Python generators have a method called Here is a trivial example that shows this method in use:<\/p>\n I learned this from a poster session I attended at EuroPython 2023<\/a> by Maxim Danilov.<\/p>\n In his poster session, Maxim proposed a really interesting pattern with an infinite loop Essentially, the pattern uses the infinite iterator If you run this piece of code, you will get output like this:<\/p>\n If I understood correctly, Maxim claimed that there was a common pattern with some infinite loops implemented in terms of This is relevant because calling This was a very specific use case for That's it for now! Stay tuned<\/a> and I'll see you around!<\/p>","summary":"Today I learned about the generator method `close`.","date_modified":"2025-07-23T16:49:02+02:00","tags":["generators","programming","python"],"image":"\/user\/pages\/02.blog\/04.til\/071.generator-method-close\/thumbnail.webp"},{"title":"TIL #069 \u2013 generator membership testing","date_published":"2023-07-17T22:00:00+02:00","id":"https:\/\/mathspp.com\/blog\/til\/generator-membership-testing","url":"https:\/\/mathspp.com\/blog\/til\/generator-membership-testing","content_html":" Today I learned that generators support membership testing with the operator Membership testing in Python boils down to using the operator What I hadn't realised yet is that generators also support membership testing!\nIt is very easy to verify that they do<\/em> support membership testing, though:<\/p>\n What I immediately realised afterwards is that you need to be very careful about membership testing with generators.\nFor example, consider the generator function from before:<\/p>\n We can check if the number 999 is in there and we will get the correct answer:<\/p>\n But now, think about it.\nHow can Python tell that 999 is not inside the generator In practice, if a generator reports that a given value is not<\/em> in the generator, then it is probably because of this issue; because you had already iterated past<\/em> the value that you were looking for.<\/p>\n Let me show you another example of this:<\/p>\n This example might look obvious to you because I am using a non-existing value to exhaust the generator.\nHowever, this issue can also arise after successful membership tests:<\/p>\n To make sure you understand what's going on, try to guess the output of the three membership checks below.\nOther than that small challenge, that's it for now! Stay tuned<\/a> and I'll see you around!<\/p>\n That's it for now! Stay tuned<\/a> and I'll see you around!<\/p>","summary":"Today I learned that generators support membership testing with the operator `in`.","date_modified":"2025-07-23T16:49:02+02:00","tags":["generators","programming","python"],"image":"\/user\/pages\/02.blog\/04.til\/069.generator-membership-testing\/thumbnail.webp"},{"title":"TIL #058 \u2013 do not cache generators","date_published":"2023-04-12T10:30:00+02:00","id":"https:\/\/mathspp.com\/blog\/til\/do-not-cache-generators","url":"https:\/\/mathspp.com\/blog\/til\/do-not-cache-generators","content_html":" Today I learned not to cache generators.<\/p>\n\nBidirectional generators<\/a><\/h2>\n
send<\/code> which allows sending data into a generator.\nTo receive data inside a generator, you can assign a yield<\/code> statement to a variable and when the user calls the method send<\/code>, the argument passed in will be stored in the variable used.\nIt's also important to note that the method send<\/code> will send data into the generator and it will also execute the generator up to its next yield<\/code> statement.<\/p>\nsend<\/code> interacts with a generator:<\/p>\ndef my_gen():\n yield 1\n yield 2\n value = yield 3\n print(value)\n yield 5\n yield 6\n\nvalues = my_gen()\nprint(next(values)) # 1\nprint(next(values)) # 2\nprint(next(values)) # 3\nvalues.send(4) # 4\nprint(next(values)) # 6<\/code><\/pre>\n5<\/code> was skipped because it was yielded when you sent the 4<\/code> into the generator and we didn't catch it outside the generator.<\/p>\nsend<\/code> interacts with generators, a generator that has been instantiated but hasn't been started yet can't be sent any data but it can be started with a call to send(None)<\/code>, which will run the generator up to its first yield<\/code>.<\/p>\nBinary search<\/a><\/h2>\n
-1<\/code>: the guess is too high;<\/li>\n0<\/code>: the guess is correct; and<\/li>\n1<\/code>: the guess is too low.<\/li>\n<\/ul>import random\n\nsn = random.randint(0, 100) # Secret number to be guessed.\n\nsearcher = binary_search(0, 100)\nfor guess in searcher:\n feedback = 1 if guess < sn else -1 if guess > sn else 0\n searcher.send(feedback)\n\nprint(f\"Secret number was {guess}.\")\n# Secret number was 42.<\/code><\/pre>\nbatched<\/code> from the module itertools<\/code> works, example use cases, and how to implement it.<\/p>\n\nitertools<\/code> introduced a new tool called batched<\/code> in Python 3.12.\nitertools.batched<\/code> lets you iterate over an iterable by going over portions of that iterable – or batches – that all have the same size, except possibly for the last one.\nSome example use cases<\/a> include batching API requests or batching data processing.<\/p>\n>>> from itertools import batched\n>>> for batch in batched(\"Hello, world!\", 3):\n... print(batch)\n...\n('H', 'e', 'l')\n('l', 'o', ',')\n(' ', 'w', 'o')\n('r', 'l', 'd')\n('!',)<\/code><\/pre>\n\"!\"<\/code>.\nThat's because the original input string had length 13 and I asked for batches of size 3, so batched<\/code> served me with four batches of size 3 and the final batch contained the remaining character.<\/p>\nExample use cases<\/a><\/h2>\n
itertools.batched<\/code>.\nIf you know of any other good use cases, feel free to leave a comment below or to reach out to me<\/a> and I'll include them here.<\/p>\nGo over a file in chunks<\/a><\/h3>\n
batched<\/code> to go over a set number of lines at a time.\nThe code would look like this:<\/p>\nfrom itertools import batched\n\nwith open(some_file, \"r\") as file:\n for chunk in batched(file, 25):\n process_chunk_of_lines(chunk)<\/code><\/pre>\n25<\/code>.<\/p>\nChunking a response over a socket<\/a><\/h3>\n
from itertools import batched\n\nfor chunk in batched(raw_data, 1024):\n sent = socket.send(b\"\".join(chunk))\n if sent < len(chunk):\n # Handle the fact that not all data was sent.<\/code><\/pre>\nb\"\".join(...)<\/code>, because batched<\/code> returns tuples with the elements.<\/p>\nIterating over substrings<\/a><\/h3>\n
batched<\/code> to split a string, but the final thing you need is substrings, you can use batched<\/code> together with \"\".join<\/code> and map<\/code> to create an iterator that produces substrings of a given length:<\/p>\nmap(\"\".join, batched(string, length))<\/code><\/pre>\n>>> from itertools import batched\n>>> hello_world_substrings = map(\"\".join, batched(\"Hello, world!\", 3))\n>>> for substring in hello_world_substrings:\n... print(substring)\n...\nHel\nlo,\n wo\nrld\n!<\/code><\/pre>\nclose<\/code>.<\/p>\n\nThe generator method
close<\/code><\/a><\/h2>\nclose<\/code> that \u201ccloses\u201d the generator.\nClosing the generator means that if you call next<\/code> on it again, it will raise StopIteration<\/code>, which is essentially the same as if the generator had been exhausted completely.<\/p>\n>>> squares = (num ** 2 for num in range(10))\n>>> for square in squares:\n... print(square)\n... if square > 20:\n... squares.close()\n...\n0\n1\n4\n16\n25<\/code><\/pre>\nAn infinite loop
for<\/code> that is cancellable<\/a><\/h2>\nfor<\/code> that could easily get \u201ccancelled\u201d\u00a0by calling the method close<\/code> on the infinite generator.\nMaxim said there were examples in the standard library where this example could be interesting, but sadly I don't remember exactly where he said we could do it.<\/p>\niter(int, 1)<\/code> to bootstrap a generator and then it uses the method close<\/code>:<\/p>\nimport random\n\ninfinity = (_ for _ in iter(int, 1))\nfor _ in infinity:\n print(\"v\")\n if random.random() < 0.05:\n infinity.close()\n print(\"Loop `for` has been cancelled.\")<\/code><\/pre>\nv\nv\nv\nv\nv\nv\nv\nLoop `for` has been cancelled.<\/code><\/pre>\nwhile True:<\/code>.\nIn certain situations, those loops can get messy because of the logic used inside the loop to determine when to use the keyword break<\/code>!<\/p>\nclose<\/code> on the generator that we are iterating over will prevent the next iteration<\/em> but it will finish running the code of the current iteration.\nOn the other hand, the keyword break<\/code> will exit the loop altogether as soon as it is encountered, skipping the remainder of the code in the loop.<\/p>\nclose<\/code>, but close<\/code> can be used with any generator; it doesn't have to be an infinite one.<\/p>\nin<\/code>.<\/p>\n\nGenerator membership testing<\/a><\/h2>\n
in<\/code>, as the REPL session excerpt shows:<\/p>\n>>> my_list = [42, 73, 16, 0, 10]\n>>> 42 in my_list\nTrue\n>>> 43 in my_list\nFalse\n>>> 10 in my_list\nTrue<\/code><\/pre>\ndef generator():\n yield 42\n yield 73\n yield 16\n yield 0\n yield 10\n\ngen = generator()\nprint(42 in gen) # True<\/code><\/pre>\nGotchas of generator membership testing<\/a><\/h2>\n
def generator():\n yield 42\n yield 73\n yield 16\n yield 0\n yield 10<\/code><\/pre>\n>>> gen = generator()\n>>> 999 in gen\nFalse<\/code><\/pre>\ngen<\/code>?\nIt had to iterate over it, which means it had to go through all the yield<\/code> expressions already!\nIn turn, this means that the generator was already exhausted.\nIt is already empty:<\/p>\n>>> next(gen)\nTraceback (most recent call last):\n File \"<stdin>\", line 1, in <module>\nStopIteration<\/code><\/pre>\n>>> def generator():\n... yield 1\n... yield 2\n... yield 3\n...\n>>> gen = generator()\n>>> 999 in gen\nFalse\n>>> 1 in gen\nFalse\n>>> 2 in gen\nFalse\n>>> 3 in gen\nFalse<\/code><\/pre>\n>>> def generator():\n... yield 1\n... yield 2\n... yield 3\n...\n>>> gen = generator()\n>>> 2 in gen\nTrue\n>>> 1 in gen\nFalse<\/code><\/pre>\n>>> def generator():\n... yield 1\n... yield 2\n... yield 3\n...\n>>> gen = generator()\n>>> 3 in gen\n## ???\n>>> 1 in gen\n## ???\n>>> 2 in gen\n## ???<\/code><\/pre>\nDo not cache generators<\/a><\/h2>\n