Parse JSON in Python
Turn JSON text into dicts and lists, and deal with text that is not JSON after all.
To parse JSON in Python, pass the text to json.loads. It is part of the standard library and hands back ordinary dicts and lists.
The short answer
import json
text = '{"name": "Nitro", "stars": 120, "tags": ["python", "static"], "archived": false}'
data = json.loads(text)
print(data["name"]) # Nitro
print(data["tags"][0]) # python
print(data["archived"]) # False
A JSON object becomes a dict, an array becomes a list, and true, false and null become True, False and None. The Python JSON overview has the full type table.
What json.loads accepts
json.loads takes a str, bytes or bytearray. Bytes are decoded for you, as UTF-8, UTF-16 or UTF-32. The top level does not have to be an object either: any JSON value parses.
json.loads(b'{"name": "Nitro"}') # {'name': 'Nitro'}
json.loads('[1, 2.5, "x", true, null]') # [1, 2.5, 'x', True, None]
json.loads('"hello"') # 'hello'
Two things it does not accept catch people out:
- A filename.
json.loads("config.json")tries to parse the name itself and fails withExpecting value: line 1 column 1 (char 0). To read a JSON file, usejson.loadon the open file. - A dict. If you get
the JSON object must be str, bytes or bytearray, not dict, the data has already been parsed, usually by your HTTP library. There is nothing left to do.
Parse a JSON API response
A response body arrives as bytes, and json.loads takes it as it is:
body = b'{"user": {"name": "Ada", "emails": ["[email protected]"]}, "meta": {"page": 1}}'
payload = json.loads(body)
payload["user"]["emails"][0] # '[email protected]'
With urllib from the standard library you can hand the response object straight to json.load, because it has a read method like a file:
from urllib.request import urlopen
with urlopen("https://api.example.com/users/1") as response:
payload = json.load(response)
If you use requests or httpx, response.json() does the same job.
When the text is not valid JSON
json.loads raises json.JSONDecodeError when it cannot parse the input. The error carries a message plus the line and column where parsing stopped.
def parse(text):
try:
return json.loads(text)
except json.JSONDecodeError as e:
print(f"{e.msg}: line {e.lineno}, column {e.colno}")
return None
parse('{"name": "Nitro",}') # Expecting property name enclosed in double quotes: line 1, column 18
parse("{'name': 'Nitro'}") # Expecting property name enclosed in double quotes: line 1, column 2
parse("") # Expecting value: line 1, column 1
parse('{"name": "Nitro"} extra') # Extra data: line 1, column 19
Those four cover most of what you will meet:
- Trailing comma. Python allows one, JSON does not.
- Single quotes. That is the
reprof a Python dict, not JSON. Fix the code that produced it so it callsjson.dumps. - Empty or non-JSON body.
Expecting valueat line 1, column 1 usually means an empty response or an HTML error page. Check the status code before parsing. - Extra data. The first value parsed fine and something follows it. Often it is two documents stuck together, or a file with one JSON object per line that should be parsed line by line.
JSONDecodeError is a subclass of ValueError, so an existing except ValueError already catches it.
Decimals and dates
JSON numbers with a fractional part become float. For money, ask for Decimal instead:
from decimal import Decimal
json.loads('{"price": 19.99}', parse_float=Decimal) # {'price': Decimal('19.99')}
JSON has no date type. A date arrives as a string, and you convert it yourself with datetime.fromisoformat once the data is parsed.
The shorter way with Nitro DataStore
Parsing needs nothing beyond json.loads. The awkward part comes afterwards, when the payload is nested and half its keys are optional. Nitro DataStore wraps the parsed dict so you can read it with a dotted path and a default:
from nitro_datastore import NitroDataStore
reply = NitroDataStore(json.loads(body))
reply.user.name # 'Ada'
reply.get("user.address.city", "unknown") # 'unknown', no KeyError
reply.has("meta.page") # True
It only wraps objects, so a payload whose top level is an array has to stay a list. If you read two or three keys from a response, plain indexing is fine and the extra dependency is not worth it. It pays off when the same deep payload is picked apart in many places.
Common questions
Is json.loads safe on untrusted input? It never executes code, which is the reason to use it rather than eval. It will still read whatever size you give it, and extremely deep nesting raises RecursionError, so cap the size of anything that comes from outside.
What happens with duplicate keys? The last one wins. json.loads('{"a": 1, "a": 2}') returns {'a': 2} without complaint.
What is the difference between json.load and json.loads? loads parses text you already hold in memory. load reads from an open file or anything else with a read method.
Related
Read a JSON file in Python
Open a JSON file, get a dict back, and handle the three ways it goes wrong.
Read ->json.dumps in Python
Turn a dict into JSON text, and teach it about dates, decimals and sets.
Read ->Working with JSON in Python
The four functions, the type table, and where each job is covered in detail.
Read ->This page uses Nitro DataStore.
Read JSON with dot notation and a chainable query builder. No schema to define, no ORM to learn.
$ pip install nitro-datastore