My new ebook “Comprehending Comprehensions” is on pre-sale and 40% off!

Today I learned how to do custom JSON decoding in Python with the module json.

A solid background with an arrow going from a JSON logo into the Python logo.

How to decode Python objects as JSON

The module json is your go-to tool whenever you need to work with JSON data in Python. Much like you have the methods json.dump and json.dumps to turn Python objects into JSON, you can use the methods json.load and json.loads to turn JSON into Python.

By using json.loads, you can take a JSON string and turn it into a valid Python object:

import json

string = '[42, {"key": null, "other_key": false}]'

[42, {'key': None, 'other_key': False}]

The method json.load comes in handy if you have a file (or something similar) to read the data from.

You can read more about json, json.load, and json.loads in the json documentation.

If you want to take a look at how to encode Python as JSON, take a look at the TIL about custom JSON encoding of Python objects.

Because we cannot encode arbitrary Python objects as JSON by default, we implement custom JSON encoders to work around that.

The reverse side of that coin is that we must be able to decode those objects again. However, the module json cannot guess what custom encodings we may have done, so we need to specify our custom decodings when appropriate.

But first, let us take a look at what the module json does in its default decoding.

Default JSON decoding into Python objects

According to the json docs, this is what the default decoding does:

JSON Python
object dict
array list
string str
number (integer) int
number (real) float
true True
false False
null None

Thus, if you want to obtain, as a Python object, anything that is not on the right column, you need to implement a custom JSON decoder.

Custom JSON decoding into arbitrary Python objects

We have two mechanisms that let us do custom JSON decoding. By taking a look at the json.load documentation, we see that we care about the keyword-arguments object_hook and object_pairs_hook. Let us play with object_hook a bit.

The function you provide as the object_hook argument is a function that:

  • will be called whenever the JSON decoding process builds a dictionary (a Python dict); and
  • must return the Python object to be used in place of the dictionary that was built.

In essence, the function you provide as the object_hook will be “intercepting” the decoding process, and you can use it to build custom objects!

Going back to the example where we used a custom JSON encoder for complex numbers, now we may want to let our code know how to decode complex numbers. In other words, we might want to do the reverse process.

So, we want to go from:

    "real": 2,
    "imag": 3

to the complex number

2+3j  # also represented as complex(2, 3)

We start by defining our object hook function, and use it to recognise dictionaries that look like complex numbers:

def object_hook(dct):
        return complex(dct["real"], dct["imag"])
    except KeyError:
        return dct

What our function above does is:

  1. try to build a complex from a dictionary with keys "real" and "imag";
  2. return the dictionary unchanged if the step above fails.

Maybe you expected the function object_hook to use an if to determine if we had the keys necessary. The distinction between that approach and what I did is discussed in my Pydon't about the LBYL and EAFP coding styles.

Now that we have implemented an object hook, we just have to plug it in when we are doing some JSON decoding:

import json

def object_hook(dct):
        return complex(dct["real"], dct["imag"])
    except KeyError:
        return dct

json_string = """[
        "key": true,
        "complex": {
            "real": 2,
            "imag": 3

# Use the object hook when calling `json.loads`.
# --------------------------------vvvvvvvvvvvvvvvvvvvvvvv
my_list = json.loads(json_string, object_hook=object_hook)
[42, None, False, {'key': True, 'complex': (2+3j)}]

Therefore, implementing a custom function object_hook is the easiest way to do custom decoding of JSON into arbitrary Python objects. Of course, this specific approach assumes your arbitrary Python objects are represented as dictionaries in their JSON form.

Read the follow-up article on defining a custom JSON decoder class or read up on how to implement a system that lets you extend JSON.

That's it for now! Stay tuned and I'll see you around!

Espero que tenhas aprendido algo novo! Se sim, considera seguir as pisadas dos leitores que me pagaram uma fatia de pizza 🍕. O teu pequeno contributo ajuda-me a manter este projeto grátis e livre de anúncios aborrecidos.


Artigo anterior Próximo artigo

Blog Comments powered by Disqus.