We take the kind of JSON body an API returns, read it into R, write some of it back out, and see what happens when a body is broken. By the end you will have met the seven functions that do most of the work, and the rule that decides what each JSON value becomes in R. The full type mapping, the limits and the differences from jsonlite are left to the help pages and the articles listed at the end.
Parse a response body
Here is a small response: a count, a cursor that is
null, and two records.
body <- '{
"total": 2,
"cursor": null,
"items": [
{"id": 1, "name": "Ada", "tags": ["admin", "dev"], "score": 9.5},
{"id": 2, "name": "Grace", "tags": [], "score": null}
]
}'json_parse() turns it into R. Look at what each kind of
JSON value has become:
res <- json_parse(body)
str(res)
#> List of 3
#> $ total : int 2
#> $ cursor: NULL
#> $ items :List of 2
#> ..$ :List of 4
#> .. ..$ id : int 1
#> .. ..$ name : chr "Ada"
#> .. ..$ tags : chr [1:2] "admin" "dev"
#> .. ..$ score: num 9.5
#> ..$ :List of 4
#> .. ..$ id : int 2
#> .. ..$ name : chr "Grace"
#> .. ..$ tags : logi(0)
#> .. ..$ score: NULLA JSON object is a named list, and null
is NULL. A JSON array is an atomic vector
when its elements are all strings, all numbers or all logical values,
which is why Ada’s tags is a character vector. Any other
array is a list, which is why items, an array of objects,
is a list of two records. Grace’s empty tags has no element
to take a type from, and is an empty logical vector. A whole number is
an integer, and any other number a double.
The type of a value is never changed to make a vector. An array that mixes numbers and strings stays a list, with the number still a number:
str(json_parse('[1, "Ada"]'))
#> List of 2
#> $ : int 1
#> $ : chr "Ada"?json_parse has the complete mapping, and the
simplify argument that changes this rule.
Get records as a data frame
An array of objects is how an API sends a table. With
data_frame = TRUE, json_parse() builds a data
frame from every such array, wherever it sits in the document. Here it
is items:
json_parse(body, data_frame = TRUE)$items
#> id name tags score
#> 1 1 Ada admin, dev 9.5
#> 2 2 Grace NAEach key is a column. score was null for
Grace and is NA here, and tags, which holds a
vector per record, is a list column.
Write a request body
json_write() goes the other way and returns the JSON as
one string, which cat() prints without R’s escapes. We
build the body of a request that adds a third record:
new_item <- list(name = "Edsger", tags = "dev", active = TRUE)
cat(json_write(new_item))
#> {"name":"Edsger","tags":"dev","active":true}A list whose elements are all named is written as an object; a list
or vector without names is written as an array, except that a vector of
length one is written as a bare value. So tags came out as
a string. The API sent tags as an array, so we wrap it in
I() to keep it one:
new_item$tags <- I("dev")
cat(json_write(new_item))
#> {"name":"Edsger","tags":["dev"],"active":true}Writing res, the response we parsed, and parsing the
result gives back the same R object:
identical(json_parse(json_write(res)), res)
#> [1] TRUESend and receive bytes
An HTTP client hands you a response body as a raw vector and wants a
request body as one. json_write_raw() returns UTF-8 bytes,
and json_parse() accepts them as it accepts a string:
bytes <- json_write_raw(new_item)
head(bytes)
#> [1] 7b 22 6e 61 6d 65
str(json_parse(bytes))
#> List of 3
#> $ name : chr "Edsger"
#> $ tags : chr "dev"
#> $ active: logi TRUEThe one-element array we wrote for tags reads back as a
character vector of length one.
When the JSON is in a file, json_parse_file() reads it
without the detour through a string:
path <- tempfile(fileext = ".json")
writeBin(bytes, path)
json_parse_file(path)$name
#> [1] "Edsger"Handle a broken body
A connection that drops leaves you with part of a body. We cut ours
off after 40 characters. json_validate() says whether the
input is JSON, without building anything and without raising an
error:
truncated <- substr(body, 1, 40)
json_validate(truncated)
#> [1] FALSEjson_parse() stops, and the message says where:
json_parse(truncated)
#> Error:
#> ! invalid JSON at byte 40: unexpected end of dataEvery error zujson raises has the class zujson_error,
and a more specific one before it, so a single handler can catch them
all and still tell them apart:
tryCatch(
json_parse(truncated),
zujson_error = function(e) class(e)[1]
)
#> [1] "zujson_parse_error"Read and write NDJSON
NDJSON is one JSON value per line, the format of log
streams and bulk uploads. json_write_ndjson() writes one
line for each element of a list, or each row of a data frame:
lines <- json_write_ndjson(res$items)
cat(lines)
#> {"id":1,"name":"Ada","tags":["admin","dev"],"score":9.5}
#> {"id":2,"name":"Grace","tags":[],"score":null}json_parse_ndjson() reads such a body back as a list
with one element per line:
records <- json_parse_ndjson(lines)
identical(records, res$items)
#> [1] TRUEWhere next
zujson stops there on purpose: it has no matrices, no streaming
parser and no custom serializers, and ?zujson lists what is
out of scope.
- Handle malformed or untrusted input: the condition classes, the depth and size limits, and when validating first is not enough.
- Read a large API document: the same functions on the 3.3 MB Kubernetes API specification.
- Coming from jsonlite: where the two packages give different results for the same input.
- Testing and conformance: how the package is checked, and what it does with the JSON that parsers disagree on.
- The reference:
every function, with the full type mapping under
?json_parseand?json_write.