This page is the authority on what Jsonnet programs should do. It defines Jsonnet lexing and syntax. It describes which programs should be rejected statically (i.e. before execution). Finally, it specifies the manner in which the program is executed, i.e. the JSON that is output, or the runtime error if there is one.
The specification is intended to be terse, precise, and illuminate all the subtleties and edge cases in order to allow fully-compatible language reimplementations and tools. The specification employs some standard theoretical computer science techniques, namely type systems and big step operational semantics. If you just want to write Jsonnet code (not build a Jsonnet interpreter or tool), you don't need to read this. You should read the tutorial and reference .
A Jsonnet program is UTF-8 encoded text. The file is a sequence of tokens, separated by
optional whitespace and comments. Whitespace consists of space, tab, newline and carriage
return. Tokens are lexed greedily. Comments are either single line comments, beginning
with a # or a //, or block comments beginning with /*
and terminating at the first */ encountered within the comment.
id: Matched by [_a-zA-Z][_a-zA-Z0-9]*.
Some identifiers are reserved as keywords, thus are not in the set id:
assert else error false
for function if import
importstr importbin in local null
tailstrict then self super
true.
number: As defined by JSON but without the leading minus.
string: Which can have five quoting forms:
" and ending with the first subsequent
non-quoted "
' and ending with the first subsequent
non-quoted '
@" and ending with the first
subsequent non-quoted "
@' and ending with the first
subsequent non-quoted '
|||, followed by optional whitespace and a
new-line. The next non-empty line must be prefixed with some non-zero length
whitespace W. The block ends at the first subsequent line that is non-empty
and does not begin with W, and it is an error if this line does not contain
some optional whitespace followed by |||. The content of the string is
the concatenation of all the lines between the two |||, which either
begin with W (in which case that prefix is stripped) or they are empty lines
(in which case they remain as empty lines). The line ending style in the file is
preserved in the string. This form cannot be used in import statements.
Double- and single-quoted strings are allowed to span multiple lines, in which case
whatever dos/unix end-of-line character your editor inserts will be put in the string.
They both understand the following escape characters: "'\/bfnrt which have
their standard meanings, as well as \uXXXX for hexadecimal unicode escapes.
Verbatim strings eschew all of the normal string escaping, including hexadecimal unicode
escapes. Every character in a verbatim string is processed literally, with the
exception of doubled end-quotes. Within a verbatim single-quoted string,
'' is processed as ', and a verbatim double-quoted string,
"" is processed as ".
In the rest of this specification, the string is assumed to be canonicalized into a sequence of unicode codepoints with no record of the original quoting form as well and any escape characters removed.
symbol: The following single-character symbols:
{}[],.();
operator: A sequence of at least one of the following single-character symbols:
!$:~+-&|^=<>*/%.
Additionally it is subject to the following rules, which may cause the lexing to terminate with a shorter token:
// is not allowed in an operator.
/* is not allowed in an operator.
||| is not allowed in an operator.
+, -, ~, !, $.
The notation used here is as follows: { } denotes zero or more repetitions of a sequence of
tokens, and [ ] represents an optional sequence of tokens. This is not to be confused with
{ } and [ ] which represent tokens in Jsonnet itself.
Note that although the lexer will generate tokens for a wide range of operators, only a finite set are currently parseable, the rest being reserved for possible future use.
| expr ∈ Expr | ::= |
null |
true |
false |
self |
$ |
string |
number
|
| | |
{
objinside
}
|
|
| | |
[
[ expr { , expr } [ , ] ]
]
|
|
| | |
[
expr
[ , ]
forspec compspec
]
|
|
| | |
expr
.
id
|
|
| | |
expr
[
[ expr ]
[ : [ expr ]
[ : [ expr ] ] ]
]
|
|
| | |
super
.
id
|
|
| | |
super
[
expr
]
|
|
| | |
expr
(
[ args ]
)
|
|
| | | id | |
| | |
local
bind
{
,
bind
}
;
expr
|
|
| | |
if
expr
then
expr
[ else
expr ]
|
|
| | | expr binaryop expr | |
| | | unaryop expr | |
| | |
expr
{
objinside
}
|
|
| | |
function
(
[ params ]
)
expr
|
|
| | |
assert
;
expr
|
|
| | |
import
string
|
|
| | |
importstr
string
|
|
| | |
importbin
string
|
|
| | |
error
expr
|
|
| | |
expr in super
|
| objinside | ::= |
member { , member } [ , ]
|
| | |
{ objlocal , }
[
expr
]
:
expr
[ { , objlocal } ] [ , ]
forspec compspec
|
|
| member | ::= | objlocal | assert | field |
| field ∈ Field | ::= |
fieldname
[ + ] h
expr
|
| | |
fieldname
(
[ params ]
)
h
expr
|
|
| h ∈ Hidden | ::= |
: | :: | :::
|
| objlocal | ::= |
local bind
|
| compspec ∈ CompSpec | ::= | { forspec | ifspec } |
| forspec | ::= |
for
id
in
expr
|
| ifspec | ::= |
if
expr
|
| fieldname | ::= |
id |
string |
[
expr
]
|
| assert | ::= |
assert expr [ : expr ]
|
| bind ∈ Bind | ::= |
id
=
expr
|
| | |
id
(
[ params ]
)
=
expr
|
| args | ::= |
expr { , expr } { , id =
expr } [ , ]
|
| | |
id = expr { , id =
expr } [ , ]
|
| params | ::= |
param { , param } [ , ]
|
| param | ::= |
id [ = expr ]
|
| binaryop | ::= |
* |
/ |
% |
+ |
- |
<< |
>> |
< |
<= |
> |
>= |
== |
!= |
in |
& |
^ |
| |
&& |
||
|
| unaryop | ::= | - | + | ! | ~ |
The abstract syntax by itself cannot unambiguously parse a sequence of tokens. Ambiguities
are resolved according to the following rules, which can also be overridden by adding
parenthesis symbols ().
Everything is left associative. In the case of assert, error,
function, if, import, importstr, importbin, and
local, ambiguity is resolved by consuming as many tokens as possible on the
right hand side. For example the parentheses are redundant in local x = 1; (x +
x). All remaining ambiguities are resolved according to the following decreasing
order of precedence:
e(...) e[...] e.f
(application and indexing)+ - ! ~
(the unary operators)* / %
(these, and the remainder below, are binary operators)+ -<< >>< > <= >= in
== !=&^|&&||To make the specification of Jsonnet as simple as possible, many of the language features are represented as syntax sugar. Below is defined the core syntax and the desugaring function from the abstract syntax to the core syntax. Both the static checking rules and the operational semantics are defined at the level of the core language, so it is possible to desugar immediately after parsing.
The core language has the following simplifications:
$, which is no-longer a special keyword.
!= == %
in
[::] are removed.+:, +::, and +::: sugars are removed.e[e].super can exist on its own.Commas are no-longer part of this abstract syntax but we may still write them in our notation to make the presentation more clear.
Also removed in the core language are import, importstr, and importbin. The
semantics of these constructs is that they are replaced with either the contents of the
file, or an error construct if importing failed (e.g. due to I/O errors). In the first
case, the file is parsed, desugared, and subject to static checking before it can be
substituted. In the case of importstr, the file is substituted in the form of a string, so it
merely needs to contain valid UTF-8. For importbin, the file is substituted as an array of integer numbers between 0 and 255 inclusive.
A given Jsonnet file can be recursively imported via import. Thus, the
implementation loads files lazily (i.e. during execution) as opposed to via static
desugaring. The imported Jsonnet file is parsed and statically checked in isolation.
Therefore, the behavior of the import is not affected by the environment into which it is
imported. The files are cached by filename, so that even if the file changes on disk during
Jsonnet execution, referential transparency is maintained.
| e ∈ Core | ::= |
null |
true |
false |
self |
super |
string |
number
|
| | |
{
{
assert e
}
{
[ e ] h e
}
}
|
|
| | |
{
[ e ] : e
for id in e
}
|
|
| | |
[
{ e }
]
|
|
| | |
e
[
e
]
|
|
| | |
e
(
{ e }
{ id = e }
)
|
|
| | | id | |
| | |
local
id = e
{
id = e
}
;
e
|
|
| | |
if
e
then
e
else
e
|
|
| | | e binaryop e | |
| | | unaryop e | |
| | |
function
(
{ id = e }
)
e
|
|
| | |
error
e
|
Desugaring removes constructs that are not in the core language by replacing them with constructs that are. It is defined via the following functions, which proceed by syntax-directed recursion. If a function is not defined on a construct then it simply recurses into the sub-expressions of that construct. Note that we import the standard library at the top of every file, and some of the desugarings call functions defined in the standard library. Their behavior is specified by implementation. However not all standard library functions are written in Jsonnet. The ones that are built into the interpreter (e.g. reflection) will be given special operational semantics rules with the rest of the core language constructs.
desugar: Expr → Core. This desugars a Jsonnet file. Let \(e_{std}\) be the parsed content of std.jsonnet.
desugarexpr: (Expr × Boolean) → Core: This desugars an expression. The second parameter of the function tracks whether we are within an object.
desugarassert: (Field × [Bind]) → Field. This desugars object assertions.
desugarfield: (Field × [Bind] × Boolean) → Field. This desugars object fields.
Recall that h ranges over :, ::, :::. The
boolean records whether the object containing this field is itself in another object. The
notation string(id) means converting the identifier token to a string literal.
desugarbind: (Bind × Boolean) → Field. This desugars local bindings.
desugarparam: (Param × Boolean) → Param. This desugars function parameters.
desugararrcomp: (Expr × CompSpec × Boolean) → Field. This desugars array comprehensions.
After the Jsonnet program is parsed and desugared, a syntax-directed algorithm is employed
to reject programs that contain certain classes of errors. This is presented like a static
type system, except that there are no static types. Programs are only rejected if they use
undefined variables, or if self, super or $ are used
outside the bounds of an object. In the core language, $ has been desugared to
a variable, so its checking is implicit in the checking of bound variables.
The static checking is described below as a judgement \(Γ ⊢ e\), where \(Γ\) is the set of
variables in scope of \(e\). The set \(Γ\) initially contains only std, the
implicit standard library. In the case of imported files, each jsonnet file is checked
independently of the other files.
We present two sets of operational semantics rules. The first defines the judgement \(e ↓ v\) which represents the execution of Jsonnet expressions into Jsonnet values. The other defines the judgement \(v ⇓ j\) which represents manifestation, the process by which Jsonnet values are converted into JSON values.
We model both explicit runtime errors (raised by the error construct) and implicit runtime errors (e.g. array bounds errors) as stuck execution. Errors can occur both in the \(e ↓ v\) judgement and in the \(v ⇓ j\) judgement (because it is defined in terms of \(e ↓ v\)).
When executed, Jsonnet expressions yield Jsonnet values. These need to be manifested, an additional step, to get JSON values. The differences between Jsonnet values and JSON values are: 1) Jsonnet values contain functions (which are not representable in JSON). 2) Due to the lazy semantics, both object fields and array elements have yet to be executed to yield values. 3) Object assertions still need to be checked.
Execution of a statically-checked expression will never yield an object with duplicate field names. By abuse of notation, we consider two objects to be equivalent even if their fields and assertions are re-ordered. However this is not true of array elements or function parameters.
| v | ∈ | Value | = | Primitive ∪ Object ∪ Function ∪ Array |
| Primitive | ::= |
null | true | false | string
| double
|
||
| o | ∈ | Object | ::= |
{
{ assert e } { string h e }
}
|
| Function | ::= |
function ( { id=e } ) e
|
||
| a | ∈ | Array | ::= |
[ { e } ]
|
The hidden status of fields is preserved over inheritance if the right hand side uses the
: form. This is codified with the following function:
The rules for capture-avoiding variable substitution [e/id] are an extension of those in the lambda calculus.
Let y ≠ x.
self[e/x] = self
|
|
super[e/x] = super
|
|
| x[e/x] = e | |
| y[e/x] = y | |
{
...
assert e'
...
[e''] h e'''
...
}[e/x] =
{
...
assert e'[e/x]
...
[e'[e/x]] h
e''[e/x]
...
}
|
|
{
[e']: e''
for x in e'''
}[e/x] =
{
[e']: e''
for x in e'''[e/x]
}
|
|
{
[e']: e''
for y in e'''
}[e/x] =
{
[e'[e/x]]:
e''[e/x]
for y in e''' [e/x]
}
|
|
(local ... y=e' ... ; e'')
[e/x] =
local ... y=e' ... ; e''
|
(If any variable matches.) |
(local ... y=e' ... ; e'')
[e/x] =
local ... y=e'[e/x] ...
;e''[e/x]
|
(If no variable matches.) |
(function (
... y=e' ...
) e'')[e/x] =
function (
... y=e' ...
)e''
|
(If any param matches.) |
(function (
... y=e' ...
) e'')[e/x] =
function (
... y=e'[e/x] ...
) e''[e/x]
|
(If no param matches.) |
| Otherwise, e' [e/x] proceeds via syntax-directed recursion into subterms of e'. | |
The rules for keyword substitution ⟦e/kw⟧ for kw ∈ { self,
super } avoid substituting keywords that are captured by nested objects:
self
⟦e/self⟧ = e
|
super
⟦e/super⟧ = e
|
self
⟦e/super⟧ = self
|
super
⟦e/self⟧ = super
|
{
...
assert e'
...
[e'']h e'''
...
}
⟦e/kw⟧ =
{
...
assert e'
...
[e''⟦e/kw⟧]h e'''
...
}
|
{
[e']: e''
for x in e'''
}⟦e/kw⟧ =
{
[e'⟦e'/kw⟧]: e''
for x in e'''⟦e/kw⟧
}
|
| Otherwise, e'⟦e'/kw⟧ proceeds via syntax-directed recursion into the subterms of e'. |
The following big step operational semantics rules define the execution of Jsonnet programs, i.e. the reduction of a Jsonnet program e into its Jsonnet value v via the judgement \(e ↓ v\).
Let f range over strings, as used in object field names.
String concatenation will implicitly convert one of the values to a string if necessary. This is similar to Java. The referred function \(tostring\) returns its argument unchanged if it is a string. Otherwise it will manifest its argument as a JSON value \(j\) and unparse it as a single line of text. The referred function \(strlen\) returns the number of unicode characters in the string.
The numeric semantics are as follows:
*, /, +, -,
<, <=, >, >=, and unary
+ and - operate on numbers and have IEEE double precision
floating point semantics, except that the special states NaN, Infinity raise errors. Note
that + is also overloaded on objects, arrays, and when either argument is a
string. Also, <, <=, >, >=
are overloaded on strings and on arrays. In both cases the comparison is performed
lexicographically (in case of strings, by unicode codepoint).
<<,
>>,
&,
^,
| and
~
first convert their operands to signed 64 bit integers, then perform the operations in a
standard way, then convert back to IEEE double precision floating point. In shift
operations <<, >>, the right hand value modulo 64 is
interpreted as the shift count. Shifting with a negative shift count raises an error.
std.pow(a, b),
std.floor(x),
std.ceil(x),
std.sqrt(x),
std.sin(x),
std.cos(x),
std.tan(x),
std.asin(x),
std.acos(x),
std.atan(x),
std.log(x),
std.exp(x),
std.mantissa(x),
std.exponent(x) and
std.modulo(a, b).
Also, std.codepoint(x) take a single character string,
returning the unicode codepoint as a number, and std.char(x) is its inverse.
The error operator has no rule because we model errors (both from the language
and user-defined) as stuck execution. The semantics of error are that its
subterm is evaluated to a Jsonnet value. If this is a string, then that is the error that
is raised. Otherwise, it is converted to a string using \(tostring\) like during string
concatenation. The specification does not specify how the error is presented to the user,
and whether or not there is a stack trace. Error messages are meant for human inspection,
and there is therefore no need to standardize them.
Finally, the function std.native(x) takes a string and returns a function
configured by the user in a custom execution environment, thus its semantics cannot be
formally described here. The function std.extVar(x) also takes a string and
returns the value bound to that external variable at the time the Jsonnet
environment was created.
After execution, the resulting Jsonnet value is manifested into a JSON value whose serialized form is the ultimate output. The Manifestation process removes hidden fields, checks assertions, and forces array elements and non-hidden object fields. Attempting to manifest a function raises an error since they do not exist in JSON. JSON values are formalized below.
By abuse of notation, we consider two objects to be equivalent even if their fields are re-ordered. However this is not true of array elements whose ordering is strict.
| j | ∈ | JValue | = | Primitive ∪ JObject ∪ JArray |
| Primitive | ::= |
null | true | false | string |
double
|
||
| o | ∈ | JObject | ::= |
{ { string : j } }
|
| a | ∈ | Array | ::= |
[ { j } ]
|
Note that JValue ⊂ Value.
Manifestation is the conversion of a Jsonnet value into a JSON value. It is represented with the judgement \(v⇓j\). The process requires executing arbitrary Jsonnet code fragments, so the two semantic judgements represented by \(↓\) and \(⇓\) are mutually recursive. Hidden fields are ignored during manifestation. Functions cannot be manifested, so an error is raised in that case (formalized as stuck execution).