JSONPath in Python
Pull values out of nested JSON, with or without a query language.
JSONPath is a query language for JSON, in the way XPath is for XML: $.store.books[*].title means "the title of every book in the store". Python has no JSONPath support in the standard library, so you either write the query as ordinary Python or install a package.
The short answer
For most queries, a comprehension over the parsed data is all you need:
import json
with open("store.json", encoding="utf-8") as f:
data = json.load(f)
titles = [book["title"] for book in data["store"]["books"]]
print(titles) # ['Dune', 'Emma', 'Neuromancer']
That is $.store.books[*].title written out by hand. Once JSON is parsed it is only dicts and lists, and Python is already good at those.
JSONPath expressions in plain Python
| JSONPath | Python |
|---|---|
$.store.name | data["store"]["name"] |
$.store.books[0].title | data["store"]["books"][0]["title"] |
$.store.books[*].title | [b["title"] for b in data["store"]["books"]] |
$.store.books[?(@.price < 9)].title | the same comprehension with an if |
$..name | a recursive search, shown below |
Query JSON with a filter
A JSONPath filter is the if clause of a comprehension:
books = data["store"]["books"]
[b["title"] for b in books if b["price"] < 9] # ['Dune', 'Emma']
[b["title"] for b in books if "sci-fi" in b["tags"]] # ['Dune', 'Neuromancer']
When you want the first match only, use next with a default so an empty result does not raise:
next((b for b in books if b["author"] == "Jane Austen"), None)["title"] # 'Emma'
next((b for b in books if b["author"] == "Nobody"), None) # None
Search JSON for a key at any depth
The one thing a comprehension cannot do is the .. operator, which finds a key wherever it sits in the document. A short recursive generator covers it:
def find_key(node, key):
if isinstance(node, dict):
for k, value in node.items():
if k == key:
yield value
yield from find_key(value, key)
elif isinstance(node, list):
for item in node:
yield from find_key(item, key)
list(find_key(data, "name")) # ['Corner Books', 'Priya']
list(find_key(data, "price")) # [8.99, 5.5, 9.25]
It walks every dict and list once, so it is fine for documents that fit in memory.
Real JSONPath with jsonpath-ng
If you want actual JSONPath expressions, the most widely used package is jsonpath-ng (pip install jsonpath-ng):
from jsonpath_ng.ext import parse
expr = parse("$.store.books[?(@.price < 9)].title")
[match.value for match in expr.find(data)] # ['Dune', 'Emma']
[match.value for match in parse("$..name").find(data)] # ['Corner Books', 'Priya']
Import parse from jsonpath_ng.ext rather than jsonpath_ng if you need filters. Each match also has a full_path, which tells you where the value was found.
It is worth the dependency when the expression is data rather than code: it comes from a config file, a user, or a system written in another language that already speaks JSONPath. When the query is fixed and lives in your own source, the comprehension is shorter, faster and easier to debug.
A lighter alternative: Nitro DataStore
Nitro DataStore is not a JSONPath implementation. There is no $, no slices, and no filter expressions inside strings. What it has is dotted paths, simple wildcards and a small query builder, which between them cover the common cases:
from nitro_datastore import NitroDataStore
store = NitroDataStore.from_file("store.json")
store.get("store.books.0.title") # 'Dune'
store.get("store.books.9.title", "n/a") # 'n/a', no IndexError
store.find_paths("store.books.*.title")
store.find_all_keys("name")
# {'store.name': 'Corner Books', 'store.staff.manager.name': 'Priya'}
Filters are Python functions rather than a string syntax:
(store.query("store.books")
.where(lambda b: "sci-fi" in b["tags"])
.sort(key=lambda b: b["price"])
.pluck("title"))
# ['Dune', 'Neuromancer']
Pick it when you want safe nested lookups with defaults and the odd filter over a list. If you need to accept real JSONPath expressions, use jsonpath-ng. If you run one query in one script, use the comprehension.
Common questions
Is JSONPath a standard? It is now. It began as a 2007 blog post, and was published as RFC 9535 in 2024. Most Python packages predate the RFC and differ in the details, filters especially, so test an expression against the library you actually use.
Is there a JSONPath module built into Python? No. The json module only parses and writes. Querying is left to you or to a third-party package.
What about JMESPath or jq? Both are separate query languages with Python packages of their own. JMESPath is the one the AWS CLI uses for --query. They solve the same problem with different syntax, and the same advice applies: only add one if the query has to be a string.
Related
Read a JSON file in Python
Open a JSON file, get a dict back, and handle the three ways it goes wrong.
Read ->Parse JSON in Python
Turn JSON text into dicts and lists, and deal with text that is not JSON after all.
Read ->Working with JSON in Python
The four functions, the type table, and where each job is covered in detail.
Read ->This page uses Nitro DataStore.
Read JSON with dot notation and a chainable query builder. No schema to define, no ORM to learn.
$ pip install nitro-datastore