std.json

std.json is a JSON parser and emitter over a recursive Json value, added in 1.4.3, hardened in 1.4.4, and given a deep free in 1.8.1. It is pure Dusk but for a single number formatting shim: json_parse reads the full grammar into a heap Json tree and hands back an error value on any malformed input rather than a partial or invalid tree, json_emit writes a tree back out in compact form, and json_free reclaims a tree node by node. The module lives at lib/std/json.dusk.

@import std.json

Imported names are flat: after @import std.json you call json_parse and json_emit and name the Json enum with no prefix. See stdlib overview for how imports work in general.

The Json value

Json is a tagged union over the six JSON value kinds, and it is recursive: an array holds a vector of child pointers and an object holds a map of them, so a nested document is a tree of alloc’d nodes.

export enum Json {
    JNull,
    JBool(b: bool),
    JNum(n: float64),
    JStr(s: string),
    JArr(items: *Vector<*Json>),
    JObj(fields: *Map<string, *Json>),
}

Arm	Payload
`JNull`	Nothing. JSON’s `null`.
`JBool(b: bool)`	`true` or `false`.
`JNum(n: float64)`	Every JSON number, integer or not. There is no separate integer arm.
`JStr(s: string)`	The decoded contents, escapes already resolved.
`JArr(items: Vector<Json>)`	The elements, in order. The arm owns the vector.
`JObj(fields: Map<string, Json>)`	The members, keyed by name. The arm owns the map.

The map went generic over its key in 1.5.2, so an object’s fields are a Map<string, *Json>. Older material that spells it Map<*Json> predates that change and is wrong today.

You build a leaf with alloc(Json.JNum(3.5)) and a branch with alloc(Json.JArr(items)). The tree is heap allocated throughout, so a Json a function builds outlives the frame and the caller gets it back intact.

Parsing and emitting

Function	Description
`json_parse(s: string) -> (*Json, error)`	Parse `s` as a single JSON document. On success the error does not exist and the pointer roots a fully built tree.
`json_emit(j: *Json) -> string`	Render `j` as compact JSON. The result is a fresh heap string you own.
`json_free(root: *Json) -> void`	Deep free a parsed tree, every node, string payload, object key, and buffer. Consumes `root`.

json_parse reads the whole grammar. Every value kind is accepted, arrays and objects nest arbitrarily, and insignificant whitespace (space, tab, newline, carriage return) is skipped anywhere the grammar allows it. A number may carry a sign, a fraction, and an exponent, and the scan enforces JSON’s own number rules rather than C’s: no leading zero on the integer part, at least one digit after the decimal point, and at least one digit in the exponent, so 01, 1., and 1e are all refused. A string decodes the simple escapes \", \\, \/, \b, \f, \n, \r, and \t, and it decodes \uXXXX, where a high surrogate in D800..DBFF followed by a low surrogate in DC00..DFFF combines into one scalar above the BMP and gets encoded to UTF-8 through std.unicode. A lone or ill-formed surrogate is not a fault; it appends U+FFFD, the same replacement std.unicode’s encoder writes for an unencodable scalar.

Anything malformed comes back as an error carrying a descriptive message: an unexpected byte, a bad escape, an unterminated string, an out of grammar number, an unbalanced bracket, a missing colon, a non-string object key, or trailing content after the value. When the error exists, the returned pointer is a throwaway JNull you must not read. Destructure the pair and check the error before you touch the tree.

json_emit writes compact JSON with no insignificant whitespace at all. Strings are re-escaped on the way out: the quote, the backslash, and the C0 control bytes get escaped, the five with short forms take them, and every other byte below 0x20 takes the \u00XX long form. A byte at or above 0x20 passes through verbatim, so a multibyte UTF-8 sequence survives the round trip intact. Object keys emit in the order they were inserted, which for a parsed tree is source order, so emitting a document you just parsed is deterministic.

A number is formatted through cool_f64_str, a %.17g runtime shim. Seventeen significant digits round trips a float64 through decimal, so a number you parse emits to text that parses back to the identical value, bit for bit. %g strips trailing zeros, so 3.5 emits as 3.5 rather than as a padded decimal.

The depth bound

json_parse recurses once per nesting level, so before 1.4.4 an input of many thousand open brackets ran the stack out and crashed the process. The parser now bounds nesting at a fixed depth, far above any nesting a real document carries, and an input deeper than that returns a nesting is too deep error instead. A pathological document is a clean error you handle like any other, not a crash you cannot catch.

The count rises when the parser enters a value and falls when it leaves, so siblings at the same level share a budget rather than accumulating. Only true nesting counts against the limit, and a long flat array of a hundred thousand elements is fine.

The number range

A number can be inside JSON’s grammar and still outside a float64. 1e400 is spelled correctly and overflows anyway, and before 1.4.4 it parsed to an infinity that json_emit then wrote as inf, text that is not JSON and does not reparse. json_parse now refuses a number whose magnitude overflows with a number out of range error. The accepted set stays inside real, round-trippable JSON: anything the parser hands you emits to text the parser reads back.

Freeing a tree with json_free

json_free, added in 1.8.1, reclaims a parsed tree. It consumes root and frees every node, string payload, object key, and backing buffer, so after the call no pointer into the tree is valid and a later dereference faults named through the generation check rather than reading reclaimed memory. Through 1.7.x there was no such function, because a value read out of a match arm is a borrow and a recursive walk cannot free a tree it does not own; 1.8.0’s owning takes are what make the deep free expressible, and json_free is the standard library’s first caller of them, walking the tree with a worklist and taking each child out of its vector or map as an owner before freeing it.

root, e := json_parse(doc)
if e.exists() { return 1 }
// ... work with the tree ...
json_free(root)   // every node, key, and payload reclaimed; root is now dead

Two conditions come with it. json_free requires a fully heap-allocated tree, which every json_parse result is, so a tree you got from the parser is always safe to free. A tree you built by hand is safe only if every payload is a heap allocation too: a JStr carrying a string literal frees that literal into undefined behavior, since a literal is not a heap block free may reclaim. And a tree is not a graph. A subtree reachable from two parents is freed twice, and the second free faults named rather than corrupting anything, so keep a parsed tree a tree if you intend to free it.

One related improvement rides along. json_parse now reclaims a repeated object key’s shadowed value at parse time: {"a":1,"a":2} keeps the later value as it always did, and the earlier subtree and the repeat key’s bytes are freed as the document parses instead of leaking past every later json_free.

For a program that parses a document, works with it, and exits, calling json_free or leaning on process exit both cost you nothing. For a long running program that parses many documents, json_free is the difference between a flat memory profile and one that climbs with every document the program has ever seen.

A tour

Parse a small document, read a field out of the root object with a match, and emit the tree back.

@paradigm procedural

@import std.json
@import std.map
@import std.functional.maybe

// The string a JStr carries, or fallback for any other kind. A match over a
// Json has to name every arm, so the other five hand back the fallback.
func str_of(j: *Json, fallback: string) -> string {
    match *j {
        JStr(s) => return s,
        JNull => return fallback,
        JBool(b) => return fallback,
        JNum(n) => return fallback,
        JArr(items) => return fallback,
        JObj(fields) => return fallback,
    }
}

func main() -> int32 {
    doc: string = "{ \"name\": \"dusk\", \"stars\": 1.5, \"tags\": [\"c\", \"llvm\"] }"
    root, e := json_parse(doc)
    if e.exists() {
        println(e.message)
        return 1
    }

    // Read the "name" field out of the root object. The fields map is a borrow
    // from the match arm, and map_get hands back a Maybe you match in turn.
    match *root {
        JObj(fields) => {
            found: Maybe<*Json> = map_get(fields, "name")
            match found {
                Some(v) => println(str_of(v, "?")),   // dusk
                None => println("no name field"),
            }
        }
        JNull => println("not an object"),
        JBool(b) => println("not an object"),
        JNum(n) => println("not an object"),
        JStr(s) => println("not an object"),
        JArr(items) => println("not an object"),
    }

    // Emit the tree back to compact text: no insignificant whitespace, keys in
    // the order they were parsed.
    out: string = json_emit(root)
    println(out)   // {"name":"dusk","stars":1.5,"tags":["c","llvm"]}
    free(out)

    // Reclaim the whole tree: every node, key, and payload. root is dead after.
    json_free(root)
    return 0
}

A match over a Json names every arm, which is why str_of spells out the five kinds it does not want. That exhaustiveness is the point: a document whose shape you assumed wrong is a branch you were made to write, not a surprise at runtime.