JSONPath query is applied to. tbl-types defines the available types in terms of abstract instances, <del> where "n" denotes a node, "v" denotes a value, and "nl" denotes a nodelist. </del>  Notes: 2.6.2. <del> The following type conversions may occur: Where an expression with a declared type of "NodesType" needs to be converted to a "ValueType", the conversion proceeds as follows: If the nodelist contains a single node, the conversion result is the value of the node. If the nodelist is empty or contains multiple nodes, the conversion result is "Nothing". </del>  Where a member of "NodesType" needs to be converted to a "LogicalType", the conversion proceeds as follows: If the nodelist is empty, the conversion result is "LogicalFalse".  The well-typedness of function expressions can now be defined in terms of this type system. The argument is a Singular Path or "filter-path" (which includes Singular Paths), or a function expression with <del> declared result type "NodesType".  Where the declared type of the parameter is not "NodesType", a conversion applies. 2.6.3.1. While functions returning "NodesType" can appear in both comparisons and test expressions, path authors must be aware of the conversions and the potential outcomes.  Specifically, nodelists convert to values and logicals according to tbl-typeconv: For example, given a function "myfunc()" of declared result type "NodesType", the following filter expressions are both valid but can have different and unexpected results: Note that a multiple-node nodelist result is deemed "LogicalTrue" in the test expression, but cannot be used for conversion to a single value and thus feeds a comparison with "Nothing". </del>  2.6.4. 2.6.8.  2.7. For each node in an input nodelist, segments apply one or more

This implicit conversion can be surprising in some situations, so it is made explicit via the value() function. Calling the value() function is a conscious choice, so should not be a surprise to the user. Fixes URL
I specifically do not like conversion functions. They are unnecessary cruft. For the conversion from to , I'm happy forbidding it, as we have in . I see no reason why a multiple nodelist should ever be converted to a value, even if it happens to have zero or one node.
Along with URL, I'm going to say that I'm okay with this as long as it's not trying to address the issues that is trying to address. My only argument against is that it enables behavior that can be unintuitive, even if it's explicit. I don't see any reason for a multiple-nodelist to be converted into a value. No one has provided a use case for this. This needs to be merged after .
Merged. See URL for the rationale behind this.
These questions/hurdles/nits came up while reading PR and related issues/PR's carefully.
In (L-1473) a nodelist. That '' for node seems to be used nowhere else.
Example (L-1552) What, if function returns ? Is , is it a comparable?
Example (L-1581) What is the return type, if the argument is ?
Nit (L-1650) ... 'an instance' ...
Consider What is the result, if the return value of is ? Shouldn't I always guard this by I think, I always would.
If I, as a non-implementer of JSONPath, would like to register a function, say ... taking a as single argument and returning an array of normalized pathes or ... I would need to know the implementation specific shape of and . As a consequence of this only implementers can register (or implement already registered) functions, as long as they didn't document internal aspects of and . If we do not want such a scenario, how could we achieve an implementation-independent interface (types) for functions?
Yep, this is a remnant. Fixed in e6dc704
Is declared as a ValueType, so it is well-typed as a . Section says: Then it goes on to say: So is false (5 is not Nothing), and also is false. Finally, it says: Since neither is the case, the answer is that the comparison is false.
NodesType does not have Nothing as a special value; this would be an empty nodelist. Count so it returns 0 on this case, as there is no (0) node in the nodelist.
Fixed in ff5f2a3
That is a matter of style, and I don't disagree with being explicit. value() still has to have a defined value as we don't have short-circuiting in JSONPath.
You can register it right away, but then you need to get the new function extension into implementations. JSONPath does not provide a way to define your own functions, so indeed, you'll have to wait for implementers. An expression language that allows to define functions in-language would be an interesting project, but probably going beyond what we can do here. No/Yes, see above. This type and this value are well-defined, so describing them in a registration should pose no problems. I'm having a hard time imagining such an interface. It would need to be different for different platforms, and for different strategies for implementing JSONPath on such a platform. So it would, for all intents and purposes, be implementation dependent. Unless/Until we define the abovementioned powerful expression language.
Oh, and remember that we are still in the filter selector expression mechanism, so it is not clear that such a function extension would be particularly useful. Adding Projection/Transformations to JSONPath is another interesting project that goes beyond base.
NAME : thanks for clarification of all ...
I believe the required changes were covered by URL Please re-open if this is not the case.
JSON Path may not provide this, but the more user-friendly implementations likely will (mine does). So you can register your function, then go on to implement the function and plug it in.
Indeed, it should not be too hard to do this for any single implementation (my PoC implementation currently doesn't, because I didn't need to prove that particular concept :-), but it will). Defining a way to do this across implementations, in a "standard" way, is much harder.
I don't think we need to define it. We can just state something like, or some such. The wording isn't great. We would need to be careful to not open the door for arbitrary custom functions (although implementations can still do that if they want; it just wouldn't be spec-sanctioned).
URL Followup issue from
If we want to know whether a function will return a singular nodelist or potentially a full one, we need to add to the declared types. We could define explicit conversion functions from nodelists and/or the new singular nodelist type to JSON values. So would become , making it explicit that we are not comparing the nodelists.
I think (type conversions) needs to be explored (and probably settled) before we can rightfully address this.
I had considered this as well, but with the conversions, I think they just muddy up the syntax.
To me could be taken a number of different ways. Either the node lists are the same or each element has an and a that must be equal. It could also mean some exists that equals some . Wouldn't it be better to use a nested selector (sorry if my terminology is incorrect) such as ?
I just noticed the double-dot. I'm not sure what you would mean by . This isn't valid (enforced by ABNF), and I don't think anyone is arguing that it should be. For , I think the type conversions are obvious. For something like , I'm not sure what to do. With the current conversions, you have three cases: (please forgive the incorrect syntax) both independently give no nodes or multiple nodes, implying returning true both give one node, meaning returning whatever the result of that is (could be true or false) one gives either no nodes or multiple nodes, and the other gives one node, meaning returning false I think the one we're hung up on here is the first case because and also in most cases . Moreover, we don't have a mechanism to compare multiple-node nodelists at all.
I see your line of thinking is to compare the node lists for a boolean result. There are case where a comparison is clear for lists of one or less. If that option was to be allowed and not just return false in all cases then I would think that the lists could only consistently be compared as to the contained nodes in any order since the ancestor () results could be in any order if one of the ancestor nodes was an object which is generally considered to be unordered.
is merely editorial, so it shouldn't block this issue.
PR is my preferred solution to this issue. PR 420 encapsulates the surprising behavior, which is the subject of this issue, in the function. The use of the function in queries becomes an explicit and conscious choice. The following alternative solutions have been suggested: PR proposes that all type conversions be explicit. This adds some precision to the spec and makes the well-typedness rules very compact, but at the cost of usability. The "Conversion example" section in PR 410 highlights the potential surprises remaining in the approach of PR 410. PR builds on PR 410 by adding a . This further impacts usability, as highlighted by the example query . PR 414 provides a type equivalent of a singular nodelist (the result of evaluating a Singular Path). is used either as a variant of in test expressions or as a way of obtaining a value in a comparison. Unlike Singular Paths, which are a syntactic device for avoiding multi-nodelist comparisons, does not appear to pull its weight in terms of specification precision vs the extra complexity/verbosity involved. PR also adds the , but without the usability downsides of the type conversion functions of PRs 410/414. The above critique of the applies to PR 412 too. PR 412 co-opts some implicit conversion rules in the rest of the spec into the function type system, making well-typedness more complex to define and understand.
After clarification of some points everything is ok.

JSONPath query is applied to. tbl-types defines the available types in terms of abstract instances,  <ins> where "v" denotes a value, and "nl" denotes a nodelist. </ins> Notes: 2.6.2.  <ins> The following implicit type conversion may occur: </ins> Where a member of "NodesType" needs to be converted to a "LogicalType", the conversion proceeds as follows: If the nodelist is empty, the conversion result is "LogicalFalse". <ins> Given an expression with a declared type of "NodesType", a "ValueType" can be obtained using a function such as "value()" (see value). </ins> The well-typedness of function expressions can now be defined in terms of this type system. The argument is a Singular Path or "filter-path" (which includes Singular Paths), or a function expression with  <ins> declared result type "NodesType" and the defined type of the parameter is "NodesType".  Where the declared type of the parameter is not "NodesType", a conversion applies. </ins> 2.6.4. 2.6.8. <ins> The "value" function extension provides a way to convert an instance of "NodesType" to a value and make that available for further processing in the filter expression: Its only argument is an instance of "NodesType" (possibly taken from a "filter-path" as in the example above).  The result is an instance of "ValueType". If the argument contains a single node, the result is the value of the node. If the argument is "Nothing" or contains multiple nodes, the result is "Nothing". 2.6.9. </ins> 2.7. For each node in an input nodelist, segments apply one or more


they work with paths that select structured values. <del> To test the value of a node selected by a path, an explicit </del>  comparison is necessary.  For example, to test whether the node selected by the path "@.foo" has the value "null", use "@.foo == null" (see null-semantics) rather than the negated existence test The comparison operators "==" and "<" are defined first and then these are used to define "!=", "<=", ">", and ">=". <del> When a path resulting in an empty nodelist appears on either side of a comparison: </del>  <del> a comparison using the operator "==" yields true if and only if the comparison is between two paths each of which result in an empty nodelist. </del>  a comparison using the operator "<" yields false. <del> When any path on either side of a comparison results in a nodelist consisting of a single node, each such path is replaced by the value of its node and then: </del>  a comparison using the operator "==" yields true if and only if the comparison is between: 2.6.1. <del> Each parameter and the result of a function extension must have a declared type. </del>  A type is a set of instances.  Declared types enable checking a JSONPath query for well-typedness independent of any argument the function is declared to have a result type of "ValueType", or (conversion applies) "NodesType". <del> Otherwise, it is occurring as an argument to a further function </del>  expression, and the following rules for function arguments apply to its declared result type.

These editorial changes from PR are needed in the base.
I confess I tend to write tedious text when precision is at stake. Hopefully, the second commit avoids the tedium.
Why was this merged so quickly?!
We merged this editorial change quickly so we could rebase the other substantive PRs on it and put us in a better position to compare their pro's and con's.
Looks good now! Should merge this quickly so we can rebase the others.

they work with paths that select structured values.  <ins> To examine the value of a node selected by a path, an explicit </ins> comparison is necessary.  For example, to test whether the node selected by the path "@.foo" has the value "null", use "@.foo == null" (see null-semantics) rather than the negated existence test The comparison operators "==" and "<" are defined first and then these are used to define "!=", "<=", ">", and ">=".  <ins> When either side of a comparison results in an empty nodelist or "Nothing": </ins>  <ins> a comparison using the operator "==" yields true if and only the other side also results in an empty nodelist or "Nothing". </ins> a comparison using the operator "<" yields false.  <ins> When any path or function expression on either side of a comparison results in a nodelist consisting of a single node, that side is replaced by the value of its node and then: </ins> a comparison using the operator "==" yields true if and only if the comparison is between: 2.6.1.  <ins> Each parameter as well as the result of a function extension must have a declared type. </ins> A type is a set of instances.  Declared types enable checking a JSONPath query for well-typedness independent of any argument the function is declared to have a result type of "ValueType", or (conversion applies) "NodesType".  <ins> Otherwise, it occurs as an argument in another function </ins> expression, and the following rules for function arguments apply to its declared result type.


even if it is rough, as long as that does not jeopardize the objective of obtaining a usable, stable JSON query language.  1.3. The JSON value a JSONPath query is applied to is, by definition, a

PR changed the spec to use "path" only for paths, and "query" for queries. The term "JSONPath" is then left in need of some motivation. This is provided in the History section so as not to interfere with the normative text.
... fine ...

even if it is rough, as long as that does not jeopardize the objective of obtaining a usable, stable JSON query language. <ins> The term was chosen because of the XPath inspiration and also because the outcome of a query consists of identifying nodes in the JSON argument. </ins> 1.3. The JSON value a JSONPath query is applied to is, by definition, a


Each parameter as well as the result of a function extension must have a declared type. <del> A type is a set of instances.  Declared types enable checking a JSONPath query for well-typedness independent of any query argument the JSONPath query is applied to. </del>  <del> tbl-types defines the available types in terms of abstract instances, where "v" denotes a value, and "nl" denotes a nodelist. </del>  Notes: <del> "ValueType" is an abstraction of a JSON value or "Nothing". </del>  <del> "LogicalType" is an abstraction of the result of a "logical-expr". Its two instances, "LogicalTrue" and "LogicalFalse", are not related to the JSON literals "true" and "false" and have no direct syntactical representation in JSONPath. </del>  <del> "NodesType" is an abstraction of a "filter-query" (which appears in a test expression or as a function argument).  Members of "NodesType" have no direct syntactical representation in JSONPath. The abstract instances above can be obtained from the concrete representations in tbl-typerep. </del>  2.6.2. The following implicit type conversion may occur: <del> Where a member of "NodesType" needs to be converted to a </del>  "LogicalType", the conversion proceeds as follows: If the nodelist contains one or more nodes, the conversion The argument is a function expression with declared result type that is the same as the declared type of the parameter. <del> The argument is a literal primitive value and the defined type of the parameter is "ValueType". </del>  <del> The argument is a Singular Query or "filter-query" (which includes Singular Queries), or a function expression with declared result type "NodesType" and the defined type of the parameter is "NodesType".  Where the declared type of the parameter is not "NodesType", a conversion applies. </del>  2.6.4.

Also tidy up the well-typedness rules and add an example. Fixes URL
NAME I'm going to merge this as I'm sure it will satisfy your concern - it follows pretty closely your preferred solution.
I am working my way through 2.6.1 and my progress is being hampered by the use of the undefined terms "instance" and "abstract". I'm trying to understand this: "A type is a set of instances." The difficulty is increased when I run into undefined notations such as . I think you're trying to say something like "A type is construct that constrains what can appear as an argument to a function and the result of evaluating a function." I'm not sure Table 13 is helpful at all, the bullet list seems to contain all the useful information without relying on mysterious notations. And I'm not sure you need to use the words "abstract" or "abstraction" at all. For example, is the following clearer? "ValueType" means either a JSON value or Nothing. "LogicalType" means the result of a logical-expr. Its possible values, LogicalTrue and LogicalFalse, are not related to the JSON literals true and false and have no direct syntactical representation in JSONPath. "NodesType" means the results of evaluating a filter-query (which appears in a test expression or as a function argument). Members of NodesType have no direct syntactical representation in JSONPath.
As you point out, this is tricky language for readers unfamiliar with abstract data types. It would be good to find wording which presumes less background knowledge. I have three concerns over the rewording you propose: the loss of abstraction, whether the resultant types are well defined, and the impact on the spec. The types are intended to be abstract: each JSONPath implementation is free to choose its own concrete representation of each type. Let's take as an example. As you observe, it's meant to cover two possibilities. Either it corresponds to a JSON value or to the abstract value Nothing. The value doesn't correspond to any JSON value. So an implementation in C could represent a as a pointer to a JSON value (possibly a struct) with the null pointer representing . On the other hand, an implementation in Haskell could use a data declaration: Note that in both these implementations, is distinct from JSON values: a JSON value is not valid as a . If we removed the "abstract" language, we'd need to ensure that these kinds of implementations, and others, are still acceptable. The spec doesn't define a type system for all JSONPath subexpressions. In particular, does not have a type, so it's not clear how we define as a type in terms of the results of a . Similarly doesn't have a type, so the same concern applies to defining as the result of a . If we were to define as the result of a , then we'd presumably need to allow any as an argument to a parameter of type - something the current well-typedness rules disallow. We may decide this is a desirable spec change, but it is a change.
For example, currently isn't allowed.
I think this is not just good, but essential. I really don't think it's OK to introduce this terminology without defining what it means. In table 14, you have an excellent explanation of what Nothing means. I think it'd be valuable to hoist that language up near the top of the Type System section, and then when you say "JSON value or Nothing" I think that's all an implementer needs to be told. I can think, in Java or C or Go, of a variety of ways I might represent Nothing, and I can't imagine that spec language getting in my way. Actually, I went and looked again and I'm increasingly convinced that your useful definition of Nothing needs to appear in that section. before you start using it. Well, their acceptability is surely to be judged by whether their behaviour conforms to the spec. So the spec's job is to define behaviour with sufficient clarity to establish that. It's not obvious to me that the (unspecified) formalisms you're using are required to do that. Fair enough; I suggest that all you need to say is that LogicalType can be either LogicalTrue or LogicalFalse, which are interpreted in the usual way for booleans but are distinct from the JSON true / false values. What is "abstract" about this? Draft currently says "NodesType is an abstraction of a filter-query". I'm sorry, I don't understand that. What does "abstraction" mean? My guess was that you meant "what a filter-query returns" and that made sense when I ran some examples in my head but I guess I got it wrong. Hmm, evaluating a filter-query generates a NodeList, no? Is the constraint that the argument/result must be a NodeList (something that a developer who's read this far into the spec is certainly going to know how to deal with)?
NAME I agree that it's essential to find wording which presumes less background knowledge, particularly of (you may be interested in following this link if you'd like to understand my usage of the term abstraction). I'll have a shot at a PR tomorrow.
FWIW, in JSON Schema, "instance" is the term we use for the subject data (the data being validated). For us, that's the data being searched.
It's hard to avoid overlap with other specs! (I'd prefer to talk about "values", rather than instances, of types, but that's a loaded term in the JSONPath spec.)
I'm not worried about overlap. I was just offering another usage of the term.
Yep, the JSON vobaculary is often getting in the way. We used some of this experience when defining the vocabulary for CBOR; e.g. "data item" for what is "value" in JSON, and "map" for what is "object" in JSON.

Each parameter as well as the result of a function extension must have a declared type.  <ins> Declared types enable checking a JSONPath query for well-typedness independent of any query argument the JSONPath query is applied to. </ins>  <ins> tbl-types defines the available types in terms of the instances they contain. </ins> Notes:  <ins> The only instances that can be directly represented in JSONPath syntax are certain JSON values in "ValueType" expressed as literals (which, in JSONPath, are limited to primitive values). </ins>  <ins> "Nothing" represents the absence of a JSON value and is distinct from any JSON value, including "null". </ins>  <ins> "LogicalTrue" and "LogicalFalse" are unrelated to the JSON values expressed by the literals "true" and "false". </ins> 2.6.2. The following implicit type conversion may occur:  <ins> Where an instance of "NodesType" needs to be converted to a </ins> "LogicalType", the conversion proceeds as follows: If the nodelist contains one or more nodes, the conversion The argument is a function expression with declared result type that is the same as the declared type of the parameter.  <ins> The argument is a function expression with declared result type "NodesType" and the declared type of the parameter is "LogicalType".  In this case the argument is converted to "LogicalType". The argument is a value expressed as a literal and the defined type of the parameter is "ValueType". The argument is a query and the declared type of the parameter is "NodesType". </ins>  <ins> The argument is a query and the declared type of the parameter is "LogicalType".  In this case the argument is converted to "LogicalType". </ins> 2.6.4.


subsections, JSONPath defines an extension point that can be used to add filter expression functionality: "Function Extensions". <del> This section defines the extension point as well as four function </del>  extensions that use this extension point.  While these mechanisms are designed to use the extension point, they are an integral part of the JSONPath specification and are expected to be implemented like any implementation level only -- they do not influence the result of the evaluation.) <del> A function argument is a "filter-query" or a "comparable". According to filter-selector, a "function-expr" is valid as a "filter-query" or a "comparable". </del> Any function expressions in a query must be well-formed (by conforming to the above ABNF) and well-typed, otherwise the JSONPath implementation MUST raise an error (see synsem-overview).  To define 2.6.2. <del> The following implicit type conversion may occur: </del>  <del> Where an instance of "NodesType" needs to be converted to a "LogicalType", the conversion proceeds as follows: </del>  <del> If the nodelist contains one or more nodes, the conversion result is "LogicalTrue". </del>  <del> If the nodelist is empty, the conversion result is "LogicalFalse". Given an expression with a declared type of "NodesType", a "ValueType" can be obtained using a function such as "value()" (see value). </del>  The well-typedness of function expressions can now be defined in terms of this type system. A function expression is well-typed if all of the following are true: <del> If it occurs directly in a test expression, the function is declared to have a result type of "LogicalType", or (conversion applies) "NodesType". </del>  <del> If it occurs directly as a "comparable" in a comparison, the function is declared to have a result type of "ValueType". </del>  <del> Otherwise, it occurs as an argument in another function expression, and the following rules for function arguments apply to its declared result type. </del>  <del> Each argument of the function can be used for the declared type of the corresponding declared parameter according to one of the following rules: </del>  <del> The argument is a function expression with declared result type that is the same as the declared type of the parameter. </del>  <del> The argument is a function expression with declared result type "NodesType" and the declared type of the parameter is "LogicalType".  In this case the argument is converted to "LogicalType". </del>  <del> The argument is a value expressed as a literal and the defined type of the parameter is "ValueType". </del>  <del> The argument is a query and the declared type of the parameter is "NodesType". </del>  <del> The argument is a query and the declared type of the parameter is "LogicalType".  In this case the argument is converted to "LogicalType". </del>  2.6.4. filter expression: Its only argument is an instance of "ValueType" (possibly taken from <del> a singular query as in the example above).  The result also is an </del>  instance of "ValueType": an unsigned integer or "Nothing". If the argument value is a string, the result is the number of entirety of, see search below) a given string matches a given regular expression, which is in I-D.draft-ietf-jsonpath-iregexp form. <del> Its arguments are instances of "ValueType".  If the first argument is not a string or the second argument is not a string conforming to I- D.draft-ietf-jsonpath-iregexp, the result is "LogicalFalse". Otherwise, the string that is the first argument is matched against the iregexp contained in the string that is the second argument; the result is "LogicalTrue" if the string matches the iregexp and "LogicalFalse" otherwise. </del>  2.6.7. given string contains a substring that matches a given regular expression, which is in I-D.draft-ietf-jsonpath-iregexp form. <del> Its arguments are instances of "ValueType".  If the first argument is not a string or the second argument is not a string conforming to I- D.draft-ietf-jsonpath-iregexp, the result is "LogicalFalse". Otherwise, the string that is the first argument is searched for at least one substring that matches the iregexp contained in the string that is the second argument; the result is "LogicalTrue" if such a substring exists and "LogicalFalse" otherwise. </del>  2.6.8. processing in the filter expression: Its only argument is an instance of "NodesType" (possibly taken from <del> a "filter-query" as in the example above).  The result is an instance of "ValueType". </del>  If the argument contains a single node, the result is the value of the node. If the argument is "Nothing" or contains multiple nodes, the result is "Nothing".  2.6.9. 2.7.

Any parameter type works with
Indeed. We could make the example more general by not talking about the parameter type there.
Or by adding a bit variety. Done.
How does this work with a parameter?
It doesn't. Example needs to be fixed. Thanks.
Ref: URL Currently is not allowed. The argument is that we've not defined as resulting in a instance (an artifact of not extending the type system to the whole expression), but I don't really see a problem with allowing it. It's pretty intuitive. I can see applications where you'd want an argument to a function to be whether a location in a path is a certain value, e.g. . An function could work this way.
If we extend the use of the type system in this way (any would be of type , possibly after implicit conversion from ), we should also roll out the other types across the spec. Then any would have type and any would have type . This fixes a bug in the current spec which I've only just noticed. The and functions are assumed to be able to take Singular Queries, but as the spec is currently written, these would need converting to using the function.
Thanks. Updating jpt to the new semantics and running it over the spec to find inconsistencies was what I wanted to do over the weekend...
I can next work on implementing this issue on Saturday.
PR attempts to implement this issue using a point solution rather than attempting to roll out the type system across the whole spec.
Approving with several suggestions that I believe will complete the remaining editorial work. (I raised some editorial nits unrelated to this PR in issue URL)

subsections, JSONPath defines an extension point that can be used to add filter expression functionality: "Function Extensions".  <ins> This section defines the extension point as well as some function </ins> extensions that use this extension point.  While these mechanisms are designed to use the extension point, they are an integral part of the JSONPath specification and are expected to be implemented like any implementation level only -- they do not influence the result of the evaluation.)  Any function expressions in a query must be well-formed (by conforming to the above ABNF) and well-typed, otherwise the JSONPath implementation MUST raise an error (see synsem-overview).  To define 2.6.2.  <ins> Just as queries can be used in logical expressions by testing for the existence of at least one node (extest), a function expression of declared type "NodesType" can be used as a function argument for a parameter of declared type "LogicalType", with the equivalent conversion rule: </ins>  <ins> If the nodelist contains one or more nodes, the conversion result is "LogicalTrue". </ins>  <ins> If the nodelist is empty, the conversion result is "LogicalFalse". </ins>  <ins> Extraction of a value from a nodelist can be performed in several ways, so an implicit conversion from "NodesType" to "ValueType" may be surprising and has therefore not been defined.  A function expression with a declared type of "NodesType" can indirectly be used as an argument for a parameter of declared type "ValueType" by wrapping the expression in a call to a function extension such as "value" (see value). </ins> The well-typedness of function expressions can now be defined in terms of this type system. A function expression is well-typed if all of the following are true:  <ins> If the function expression occurs as a "test-expr" in a logical expression, the function is declared to have a result type of "LogicalType", or (by conversion as per type-conv) "NodesType". If the function expression occurs directly as a "comparable" in a comparison, the function is declared to have a result type of "ValueType". </ins>  <ins> Otherwise (the function expression occurs as an argument in another function expression), the following rules for function arguments apply to its declared result type. </ins>  <ins> The arguments of the function expression are well-typed, as follows. </ins>  <ins> Each argument of the function can be used for the declared type of the corresponding declared parameter according to one of the following rules: </ins>  <ins> The argument is a function expression with declared result type that is the same as the declared type of the parameter. </ins>  <ins> The argument is a function expression with declared result type "NodesType" and the declared type of the parameter is "LogicalType".  In this case the argument is converted to "LogicalType" as per type-conv. </ins>  <ins> The argument is a value expressed as a literal and the declared type of the parameter is "ValueType". </ins>  <ins> The argument is a singular query and the declared type of the parameter is "ValueType".  In this case: </ins>  <ins> If the query results in a nodelist consisting of a single node, the argument is the value of the node. If the query results in an empty nodelist, the argument is "Nothing". The argument is a query (including singular query) and the declared type of the parameter is "NodesType". The argument is a "logical-expr" and the declared type of the parameter is "LogicalType". Note that the last bullet item includes the case that the argument is a query (including singular query) and the declared type of the parameter is "LogicalType".  In this case the nodelist resulting from the query is interpreted as a logical-expr in the same way (extest) it would be converted to "LogicalType" as per type-conv. </ins> 2.6.4. filter expression: Its only argument is an instance of "ValueType" (possibly taken from  <ins> a singular query, as in the example above).  The result also is an </ins> instance of "ValueType": an unsigned integer or "Nothing". If the argument value is a string, the result is the number of entirety of, see search below) a given string matches a given regular expression, which is in I-D.draft-ietf-jsonpath-iregexp form.  <ins> Its arguments are instances of "ValueType" (possibly taken from a singular query, as for the first argument in the example above).  If the first argument is not a string or the second argument is not a string conforming to I-D.draft-ietf-jsonpath-iregexp, the result is "LogicalFalse".  Otherwise, the string that is the first argument is matched against the iregexp contained in the string that is the second argument; the result is "LogicalTrue" if the string matches the iregexp and "LogicalFalse" otherwise. </ins> 2.6.7. given string contains a substring that matches a given regular expression, which is in I-D.draft-ietf-jsonpath-iregexp form.  <ins> Its arguments are instances of "ValueType" (possibly taken from a singular query, as for the first argument in the example above).  If the first argument is not a string or the second argument is not a string conforming to I-D.draft-ietf-jsonpath-iregexp, the result is "LogicalFalse".  Otherwise, the string that is the first argument is searched for at least one substring that matches the iregexp contained in the string that is the second argument; the result is "LogicalTrue" if such a substring exists and "LogicalFalse" otherwise. </ins> 2.6.8. processing in the filter expression: Its only argument is an instance of "NodesType" (possibly taken from  <ins> a "filter-query", as in the example above).  The result is an instance of "ValueType". </ins> If the argument contains a single node, the result is the value of the node. If the argument is "Nothing" or contains multiple nodes, the result is "Nothing". <ins> Note: a Singular Query may be used anywhere where a ValueType is expected, so there is no need to use the "value" function extension with a Singular Query. </ins> 2.6.9. 2.7.


A test expression either tests the existence of a node designated by an embedded query (see extest) or tests the result of a function <del> expression (see fnex).  In the latter case, if the function result type is declared as "LogicalType" (see typesys), it tests whether the result is "LogicalTrue"; if the function result type is declared as "NodesType", it tests whether the result is non-empty.  If the declared function result type is "ValueType", its use in a test </del>  expression is not well-typed. Comparison expressions are available for comparisons between 2.6.3. <del> A function expression is well-typed if all of the following are true: </del>  <del> If the function expression occurs as a "test-expr" in a logical expression, the function is declared to have a result type of "LogicalType", or (by conversion as per type-conv) "NodesType". </del>  <del> If the function expression occurs directly as a "comparable" in a comparison, the function is declared to have a result type of "ValueType". </del>  <del> Otherwise (the function expression occurs as an argument in another function expression), the following rules for function arguments apply to its declared result type. </del>  <del> The arguments of the function expression are well-typed, as follows. Each argument of the function can be used for the declared type of the corresponding declared parameter according to one of the following rules: The argument is a function expression with declared result type that is the same as the declared type of the parameter. The argument is a function expression with declared result type "NodesType" and the declared type of the parameter is "LogicalType".  In this case the argument is converted to "LogicalType" as per type-conv. The argument is a value expressed as a literal and the declared type of the parameter is "ValueType". The argument is a singular query and the declared type of the parameter is "ValueType".  In this case: If the query results in a nodelist consisting of a single node, the argument is the value of the node. If the query results in an empty nodelist, the argument is "Nothing". The argument is a query (including singular query) and the declared type of the parameter is "NodesType". The argument is a "logical-expr" and the declared type of the parameter is "LogicalType". Note that the last bullet item includes the case that the argument is a query (including singular query) and the declared type of the parameter is "LogicalType".  In this case the nodelist resulting from the query is interpreted as a logical-expr in the same way (extest) it would be converted to "LogicalType" as per type-conv. </del>  2.6.4.

I actually ordered the two definition lists in the order the production appear in the grammar, so this is now easy to check (and I have checked it).
The section in question (and subject to some change soon) is URL The part that feels clunky to me is the first line: While not wrong, when you get into the bullet points, it feels odd that we're saying "all of the following are true" when only one things ever applies due to the conditions each item defines. We're saying we have three options, A, B, and C and then we define rules: If A, then ... If B, then ... Otherwise C, then ... (I know I made a about changing "rules" to "scenarios" in a previous PR, but each of these statements conforms to the definition I gave for a "rule.") The thing that feels weird is that we're saying they all_ must be true, which... is "correct" because the requirements are conditionally applied, but also it seems that (because of the conditions) only one ever actually applies, meaning that the other two are more "not applicable" than they are "true." Maybe if we instead said: Previous to the current text (before the type system was introduced), we had:
I was also thinking of turning this into a table.
A table could be nice, too, but the "all of following must be true" language still needs adjustment.
I agree this section is rather tedious to read.
Oh, and regardless of whether a table could be crammed into the document, it would be good to somehow check that all the permutations have been covered correctly. It's too hard to hold the possibilities in my small head while reading the current text...

A test expression either tests the existence of a node designated by an embedded query (see extest) or tests the result of a function  <ins> expression (see fnex).  In the latter case, if the function's declared result type is "LogicalType" (see typesys), it tests whether the result is "LogicalTrue"; if the function's declared result type is "NodesType", it tests whether the result is non-empty.  If the function's declared result type is "ValueType", its use in a test </ins> expression is not well-typed. Comparison expressions are available for comparisons between 2.6.3.  <ins> For a function expression to be well typed: </ins>  <ins> its declared type must be well typed in the context in which it occurs, and </ins>  <ins> its arguments must be well typed for the declared type of the corresponding parameters. </ins>  <ins> (1) As per the grammar, a function expression can occur in three different immediate contexts, which lead to the following conditions for well-typedness: </ins>  <ins> (2) The arguments of the function expression are well-typed when each argument of the function can be used for the declared type of the corresponding parameter, each according to one of the following conditions depending on the way the argument is supplied: </ins> 2.6.4.


the result is "LogicalTrue"; if the function's declared result type is "NodesType", it tests whether the result is non-empty.  If the function's declared result type is "ValueType", its use in a test <del> expression is not well typed. </del>  Comparison expressions are available for comparisons between primitive values (that is, numbers, strings, "true", "false", and (2) The arguments of the function expression are well typed when each argument of the function can be used for the declared type of the <del> corresponding parameter, according to one of the following conditions depending on the way the argument is supplied: </del>  2.4.4.

Fixes URL
You're welcome. The issue needed a bit of "show, don't tell". I currently agree, but I'd like to let this PR go "cold" for a while and then compare how this section reads relative to the main branch. Meanwhile, others may like to give their opinions.
I now ran the tests (and updated a few details with changing external tests)
The organization of the expansion of point 2 in "Well-Typedness of Function Expressions" can be improved. Currently, it's grouped by the kind of argument, but it's much easier to consume if you group by function parameter type. (Also, I think we're missing some cases.) I'm going to put it in a table so it's a bit easier to see. The above answers the question I don't think that's going to be a very common question JSON Path authors will have. I think the more common question will be To that end, we need to invert the grouping by using parameter type as the key. Doing this, we get This is a lot easier to consume and mentally digest. It also better serves as a reference for JSON Path authors. (We lose the "function type matches parameter type" thing, but I don't think we need to explicitly say that.)
Questions: (1) A is defined in the draft as a list of location-value pairs. Is there a distinction between and ? (2) If a function has a parameter that accepts arguments of , does it receive a list of location-value pairs?
Abstractly, yes. A nodelist is an instance (or member) of . Yes.
NAME thanks. Are the following statements correct? The JSONPath query in its entirety always evaluates to a nodelist (a list of location-value pairs) Within a filter, a JSONPath query evaluates to either a value of type or a nodelist of type . If the query satisfies the requirements of a Singular Query, it evaluates to a value of type , otherwise it evaluates to a nodelist of type .
Yes A query always evaluates to a nodelist (). In the context of a filter expression, if it's a singular query, the resulting nodelist can be implicitly converted to a value () by extracting the single node's value (or for an empty nodelist).
Yes. Types only exist in the function interface. The declared type of the function you use a query for determines what the function gets.
NAME thanks. What about the operator interface? Operators are much like functions. Would it be correct to say that the comparison operators take arguments of type ValueType?
The spec doesn't say that. It could, be we decided against rolling out the (function) type system across the whole spec as it didn't seem to add much value, it would impact readability, and it is not without its own complexity. If you'd like to explore that option, please could we do it on the mailing list so it gets appropriate visibility? I don't think that topic is so relevant to this particular issue.
Back to the issue as proposed. We definitely need to say that. We also have to transfer all the other information from what is there now to the transposed form. I don't think the result would be an improvement.
We do say it, just not in those words. It's explicitly listed in the (new) table. What other information is there? I've stated all of the possible arguments for each parameter type. The improvement is the question that is answered. "Where can I put this argument?" vs "What kinds of things can I use for arguments to this function?" Users are going to ask the second question, not the first.
I think part of the difficulty in comparing the approach in this issue with the draft is that it's not obvious (at least to me) how much of section 2.4.3 the table would be replacing. I think it would be just section (2). Right? Even replacing just section (2), the following information is missing: What to do when a singular query is used as an argument for a parameter of type What to do when a query (including a singular query) is used as an argument for a parameter of type What to do when a function expression with result type is used as an argument for a parameter of type
I reordered (2) of 2.4.3 so related items are closed to each other (and the declared types also happen to be in alphabetical order).

the result is "LogicalTrue"; if the function's declared result type is "NodesType", it tests whether the result is non-empty.  If the function's declared result type is "ValueType", its use in a test  <ins> expression is not well typed (see well-typedness). </ins> Comparison expressions are available for comparisons between primitive values (that is, numbers, strings, "true", "false", and (2) The arguments of the function expression are well typed when each argument of the function can be used for the declared type of the  <ins> corresponding parameter, according to one of the following conditions: When the argument is a function expression with declared result type the same as the declared type of the parameter. When the declared type of the parameter is "LogicalType" and the argument is one of the following: A function expression with declared result type "NodesType". In this case the argument is converted to LogicalType as per type-conv. A "logical-expr" that is not a function expression. When the declared type of the parameter is "NodesType" and the argument is a query (which includes singular query). When the declared type of the parameter is "ValueType" and the argument is one of the following: A value expressed as a literal. A singular query.  In this case: If the query results in a nodelist consisting of a single node, the argument is the value of the node. If the query results in an empty nodelist, the argument is Nothing. </ins> 2.4.4.


A structure complying to the generic data model of JSON, i.e., composed of containers, namely JSON objects and arrays, and of atomic data, namely null, true, false, numbers, and text strings. <del> Also called a JSON value. </del>  A JSON object as defined in RFC8259.  Never used in its generic sense, e.g., for programming language objects. The name in a name/value pair constituting a member.  (Also known as "key", "tag", or "label".)  An item in an array.  (Also used with a distinct meaning in XML context for XML elements.)  Short name for JSONPath expression. Short name for the JSON data item a JSONPath expression is applied to. <del> A simple form of JSONPath expression that identifies a Position by providing a query that results in exactly that position.  Similar to, but syntactically different from, a JSON Pointer RFC6901. A JSON data item identical to or nested within the JSON data item to which the query is applied to, expressed either by the value of that data item or by providing a Normalized Path Expression as a JSONPath Output Path. A query in a normalized form that identifies exactly one Position in an Argument; see overview. </del>  1.2. for paths input to a JSONPath processor.  [1] Where a JSONPath processor uses JSONPath expressions as output paths, these will <del> always be converted to normalized JSONPath expressions which employ the more general _bracket-notation_.  [2] Bracket notation is more general than dot notation and can serve as a canonical form when a JSONPath processor uses JSONPath expressions as output paths. </del>  JSONPath allows the wildcard symbol "*" for member names and array indices.  It borrows the descendant operator ".." from E4X and the and "end" depend on the sign of "step", as follows: Slice expression parameters "start" and "end" are not directly usable <del> as slice bounds and must first be normalized.  Normalization is defined as: </del>  The result of the array indexing expression "[i]" is defined to be the result of the array slicing expression "[i:Normalize(i)+1:1]".

I created a pull request so we have something concrete to discuss to
(This was original raised in and now being split out) 8259, and that the word "object" should not be used for items that are not JSON objects in the sense of RFC 8259. (Daniel P) terms per RFC8259 is pretty small: JSON text, value, object, array, number, string. Having spent quite a bit of time specifying JSON DSLs, I find that using just those terms doesn't seem to get in the way or cause problems, so I'd argue that we should stick to them (and build up to higher-level constructs as required for JSONPath). (Tim Bray) (Carsten Bormann) (Michael Kay) "member" to mean (Douglas Crockford uses "member"). (Daniel P) (Glyn Normington) I understand, that within RFC8259 we have JSON values of different types. They are structured somehow, which is not so much of interest here. But while querying that structure with JSONPath it is vitally important to identify that hierarchical structure as a tree. So in fact we build up a higher-level construct here. We also need to call "the things" in the tree somehow. I was able to identify "node" or "item" of a tree "member" of an object "name/value" or "key/value" pair alias "member" "element" of an array but could not see an agreement here. I agree to Glyn calling the term "union" poor (s. below).
Also relates to .
Until the early discussions on the jsonpath list, I thought more about value selection, I hadn't really thought much about nodes, or appreciated the importance of paths for identity (duplicate removal) and sorting, if the user wanted that. But having changed my thinking based on those conversations, I'd like to understand better what exactly a node represents in JSONPath. In XML, we have the DOM, and DOM nodes. But in JSON we don't really have anything like that. I don't find it natural to think about a "tree of nodes" in the context of JSONPath. The implementations I know about don't have a tree, they don't identify location by the position in a tree, rather, they operate on path/value pairs. For example, looking at the slice operator code in there is no tree, just a selected value and an updated path. That's typical I think of many implementations. Would it be better to talk about a node as a path-value pair, rather than a node in a tree?
For me, regarding a JSON value as a tree isn't an implementation consideration - it's an observation about the abstract structure of the data. Similarly, the notion of the position (or perhaps location) of a node in the tree is just an abstract way of identifying where the node is in the tree. A path is one implementation of the abstract notion of position, but not the only one. I think in the spec we should try to avoid implementation bias and speak of positions rather than paths, references, or pointers, all of which bring to mind implementations. It's unfortunate that (general) trees are also common data structures and maybe that's part of the problem here. But in reality, we also have binary trees, AVL trees, splay trees, B-trees, etc., so I try to think of a general tree data structure as just one way of implementing an abstract tree. Does that help at all?
NAME wrote: Okay. So we want to talk about abstract input, abstract output. Tree of nodes in, tree of nodes out. It would be really helpful if you could answer two questions. Question 1: What is a node in JSONPath? What are its properties? "node" appears 60 times in the draft, but I only found two mentions of what a node is. In section 3.2, it says "Each node holds a JSON value" (emphasis added). And then there is "root node which is the input document" (emphasis added.) I'm assuming (please correct me if I'm wrong), that a JSON value as defined in RFC8259 cannot itself be considered a "tree of nodes". It's a hierarchy of values, not nodes that contain values. What are the properties of a node? We're told it holds a JSON value. What else does it hold? Does a node provide access to its parent node, like a DOM Node does? Does a node provide access to its associated key, or index? Question 2: How do we interpret "root node which is the input document" The term "input document", is that a real thing? Actual JSON data? No nodes? Does the draft really mean that the root node is the input document? Or that it represents the input document? Suppose the input document is What value does the root node contain? ? Apologies for questions that must seem self evident to you, but I don't have any experience with nodes in the context of JSON. Thanks, Daniel
The draft defines Position, In section 1.1, as follows: "A JSON data item identical to or nested within the JSON data item to which the query is applied to, expressed either by the value of that data item or by providing a Normalized Path Expression as a JSONPath Output Path." I think this needs to be clearer. It would be easier to read if the phrase "the JSON data item to which the query is applied to" was replaced by a single term defined earlier, perhaps root, or whatever term the authors choose to use consistently that means the same thing. The statement that position can be expressed "by the value of that data item" is not clear. What is its intended meaning? A "data item" is said earlier to be the the same as a "JSON value". I don't understand how "position" can be expressed as the value of that item. Is it its location within the data item? Particularly when the alternative is "a Normalized Path Expression as a JSONPath Output Path". I think it would be easier to read the sentence if "Normalized Path Expression as a JSONPath Output Path" was replaced by "Output Path", which is defined earlier.
I agree, "node" is an important term to define clearly. Pragmatically quoting as not being a computer scientist gives: With this, for me "node" is either a primitive JSON value or a JSON container – array or object – as a possibly empty subtree. Nodes are either named or indexed; thus have a unique tree location. Leaf nodes are either primitive values or empty containers. There is always a root node. What do you think – I'm not an expert with "abstract data types".
A node is an item (a term we have defined), with an emphasis on this item possibly being a specific part of a bigger item. (Add same instance vs. equal value discussion here.)
Oh well: "(Data) item" == "JSON value" == "node". Maybe we should make that triple equivalence explicit.
Data item and JSON value, as defined in the draft, are crystal clear. But it isn't stated in the text of the draft that "node" == "JSON value". It's stated that "Each node holds a JSON value", and "the JSON value held by a node " (emphasis added). My reading of that text is that a node boxes a JSON value ( and possibly has additional attributes, such as location information if the JSON value is a descendent of the root.) Also consider that a reader seeing the term "node" may have a priori understanding of the term by analogy to DOM nodes, where it's possible to navigate back to the root from any child node. Note that my concern is with what the text says, what the authors intended, and the alignment thereof. I'm less concerned with any particular resolution.
So we now have (bold terms added): Living table ... Please edit.
Why not simply Normalized Path instead of Normalized Path Expression, since an Output Path is always a Normalized Path ?
NAME wrote: What my question really comes down to, considering both "JSON value" and "node" as abstract entities, is this: Is this abstract operation defined in one or both cases: For (1), my expectation is "no", since JSON values are generally regarded in JSON specifications as self-contained. Using RFC 6901 terminology, a "specific value" extracted from a "JSON document" has no connection anymore to the original "JSON document". For (2), my expectation is "yes", by analogy to the DOM and DOM nodes. More generally, are there any abstract operations defined on "node" that are not also defined on "JSON value"? If the answer is no, than introducing the term "node" serves no purpose.
I don't think we should use "JSON value"; we have the term "item" for that. "node" means exactly the same, so we don't need it. In the same way that you don't need both the term "flophouse" and "hotel". I believe "node" is a good term when the emphasis is on the position that item has on a tree.
For the whole selector, the input is a JSON value which can be thought of abstractly as a tree of nodes, but the output is a sequence of JSON values each of which can be thought of as a subtree of the input tree. If you're not familiar with trees and nodes, Wikipedia gives a reasonable definition of - see the terminology section for "node". However, that page is about data structures, so is not particularly abstract. The page on may be more helpful. A JSON value can be considered as a tree of nodes. A JSON literal corresponds to a single node with value equal to the literal. A sequence corresponds to a node whose children (indexed by position in the sequence) correspond to the values of the sequence. An object corresponds to a node whose children (indexed by name in the object) correspond to the values of the object. I think it's probably better to talk of a JSON value corresponding to the tree rooted in a node. For literals, the node holds the type and value of the literal. For sequences or objects, the node holds just the number of children or their names, respectively. Every node except the root node has_ a parent. I don't think it "provides access" to anything, because this is an abstraction rather than a programming construct. No, a node doesn't know its key or index - that's the concern of its parent. The input document is JSON data and is also a JSON value. It corresponds to a tree of nodes as outlined above. Probably "represents" is closest. I used the word "corresponds" above because I have in mind a mapping from the JSON value to a tree of nodes. The tree in this case is something like this:
Is this a hint to rename "data item" to "item"? ok ... "node" is self-explaining in the tree context, so I will delete it from the terms list.
Data item is just the long form. I still think we should have a terminology entry (or add it to the “item” entry). Grüße, Carsten
I don't think its necessary to talk about a "tree of nodes", which is a less abstract concept; it is enough to define node, and identify its properties. "tree of nodes" suggests an arrangement of data that is never found in actual implementations, "node" by itself doesn't have that connotation. It's properties are all that matter.
I don't think it's necessary for a node to "know" about its parent. I personally think it's easier to talk about a tree of nodes rather than a node in isolation. The tree structure is implicit in the way JSON is structured. Anyway, I feel we are talking somewhat at cross-purposes, but do you think we are converging on a common understanding yet?
This is precisely how every .net solution I've seen/created works. Also, seconding everything that NAME . (I was building such a response, but I upon reading it, I figured the horse was dead enough.) Regarding access to parents, I'd like to call out that the Newtonsoft parser results in a tree of items that do have such access, while the newer in-built System.Text.Json parser results in a tree of items that do not have access. NOTE These are represented by .Net classes internally, so naturally they have some extended properties besides the JSON value. The point here is that one points back to the parent while the other doesn't. Therefore it's completely up to the implementation/parser to determine how it wants to model the JSON internally. JSON doesn't define how or whether such access should be supported. It's just a data structure. Additionally, (he's on sabbatical, so I don't want to tag him) has released as an extension to the base JSON Pointers spec where the first action is to navigate up the tree before dereferencing the rest of the pointer. This spec assumes nothing of "nodes" that contain any metadata or their navigability. It's just considering the structure as a tree of values.
I just read the current draft carefully and was left unconvinced that the term "node" needs to be used. In particular, consider the nice example in section 3.5. I went through that and it seems to me that every instance of "node" could be replaced by "JSON Value" and be accurate, because I think every bit of the instance discussed there is a JSON value. (If we were going to do this, we should probably have a terminology note that "Value" means "JSON Value", just for brevity and readability.) Are there other places in the spec where the use of the term "node" is necessary for accuracy or clarity? To test my hypothesis, I'll create a PR that updates section 3.5 to (hopefully) support my point. But if someone has a strong argument why we need to invent new terminology, I'd be happy to avoid the effort.
There is no doubt that the draft could be made to work by calling the things "mugs". One can adapt to anything. Value is a particularly bad term to use here because it focuses on the valueness, i.e., comparison by value. Nodes focus on the place, the position, in the tree, i.e., comparison by identity. Again, the draft can be written in a way that makes the reader make that distinction on their own, but that is not a good way to do tech doc. Why don't the terms from 8259 suffice? 8295 defines a JSON text, not its processing. It is rather natural that once processing is added, new terms are added, too.
(Of course, I don't have a strong argument for why we need to "invent" "new" terminology; I think the term "node" has been used before. Jeez.)
NAME wrote: JSON = jsonPath(JSON, JSONPath-expression, value-option) This is the most common case, and the entirety of the cburgmer JSONPath Comparisons is devoted to it. I believe an implementation could match all of the consensus results, and where no consensus exists, the largest minority result, with doing nothing more than starting with a JSON value and working down through its children, without accumulating paths, and without concerning itself with identity. I note that the consensus is (32 out of 41 implementations); with duplicates, there are issues with identity, but without, there are not. Now consider the evaluation of a JSON document that produces a JSON array of Normalized Paths, JSON = jsonPath(JSON, JSONPath-expression, path-option) In terms of the processing model, in the case of duplicates allowed, not much is different, except that in addition to selecting a value at each step, a corresponding path needs to be accumulated. The paths are simply accumulated, we don't do anything with them, we compute them along with the values, and we return a JSON array of them. At every step, the "current value" corresponds to an accumulated path, these two exist as a pair, so I think it makes sense to refer to that pair as a "node". In the output, a path will be represented as a "string". In the processing model, it need not be, we can talk about a "context position" if you like, but conceptually it's a path. For either the value or path option, it's in the minority position of "no duplicates" that things become interesting, where identity matters. But identity can be determined entirely by paths. Duplicate path-value pairs are straight forward to exclude. Of course, other processing models that result in no duplicates are interesting to look at also. Yes, but it seems to me that the items upon which the processing model operate should be from 8259. New terms shouldn't be introduced gratuitously. Daniel
So why are you talking about JSON documents, paths, children, etc.? I completely agree on this one, but I think you made a great demonstration that a few new terms do help.
Well, you could call them mugs or nodes, but what they actually are is JSON Values, as defined in 8259. I'm having trouble parsing the phrase "the place, the position, in the tree, i.e. comparison by identity". I'll go ahead and do a PR to update 3.5 without introducing a new term and let's see if we like how it reads.
NAME wrote: Point taken :-) One does feel the need to call that first thing provided something other than a JSON value, JSONPointer uses "JSON document", JSON Schema uses "JSON instance", Goessner uses "root". I think paths are required. "Children" can be dispensed with.
uses "argument" for the argument of the query.
NAME wrote: Incidentally, specific language issues aside, if you're thinking about processing models for preserving identity and removing duplicates, which is the impression I got from watching the last meeting video, I'm actually quite interested in that. Daniel
In 8259, the top-level production is " ws value ws ". Anyone who's read that RFC is not going to have any trouble with the phrase "root value". 8259 does not define or use the term "child", so if we do need it (not obvious to me) we can define it as meaning "elements, if the value is an array, member values if it's an object". I note that the current draft does not actually define the term "node". I think that if you did, the definition would be something like "synonym of JSON value". So…
Please read at some point.
NAME wrote: Right Still, when talking about "NAME I prefer to talk about it as a "current node" rather than a "current value". "current value" isn't enough for fulling describing the JSONPath processing model, when talking about producing Normalized Paths, the path associated with the current value matters too. It matters even more when we talk about no duplicates. I think NAME is right to emphasize that aspect. Talking about @ as representing both the location in the root value as well as the value itself, and using node to mean that, seems reasonable to me.
I think it makes more sense to talk about the location of a node rather than the location of a JSON value because a JSON value could be present at multiple nodes whereas a node has a unique location.
NAME thanks for bringing it to the point pragmatically. In we have five JSON values, from which two values are 5, i.e. identical. They belong to two different name/value pairs (members) according to 8259. Now consider with three JSON values, from which again two values are 5, i.e. identical. They belong to two different ... hmm ... 8259 does not tell. But JSONPath needs to identify them uniquely by their location. Looking at JSON Pointer 6901 we read So JSON Pointer also needs to identify things uniquely by their location and uses ... ... for addressing members in objects and elements in arrays by a single term, since alone simply does not work. For us in fact two additional – location specifying – terms would suffice , or or skipping , then only one.
JSON pointer is about references, so they might have that word in their terminology; I don't think we need that. node and item are in . root is interesting, because that is an argument to the query — I called that argument in . We can always change these terms more if we want to, but at some point we need to enable ourselves to work on the content of the draft.
NAME wrote: But it only makes sense to talk about the location of a "node" if the term "node" is defined, which I don't believe it is in the current draft. is what "node" means in XPath 3.1, but what does it mean in JSONPath? It is meaningless to talk about "node" unless "node" is defined, and its properties enumerated, as they are in the XPath 3.1 reference. I think terms are being introduced into the draft that are suggestive of terms in XPath, such as "item" and "node", but surely the one thing that we can all agree on is that we do not want the JSONPath spec to look anything like XPath, with its horrendously complicated data model. It's also unnecessary to introduce "node" to fully describe JSONPath, including selection of values or references from a root value with duplicates, selection of Normalized Paths from a root value with duplicates, and both of these operations without duplicates (when identity matters.) It is enough to have the notion of path/value pairs, where "path" describes the location of the value in the root value. Nothing else is needed. Daniel
NAME ... this was merely an answer for Tim's wish, restricting to 8259 terms exclusively. NAME ... so you are proposing to use and ?
NAME — yes, I explained why limiting ourselves to 8259 terminology is misguided. We can certainly invent more and more complex terminology, or we can accept that "node" is the term of art here.
NAME wrote: NAME proposed , so I'll stick to that term in my commentary, unless another term is agreed to. I'm broadly in favour of Tim's wish to adhere to 8259 terms to the extent possible. I'm happy with "value location", since it seems to be nicely symmetrical with the other terms. I believe all operations in the JSONPath processing model can be fully described with just the notions of "root value", "value" and "value location". Daniel
A query language for JSON Comparison table update from . Thanks NAME for adding JMESPath column.
NAME Regrading the table with terminology compared, not sure where I can comment on this, as the issue has been closed, and I don't see it in a pull request. "XPath" in column heading should specify the version. If "what is it" references "XDM 3.0", it should be "XPath 3.0". You could also specify , and change "what is it" to reference If XPath is 3 or 3.1, "expression evaluates to" shouldn't have "node-set", node-set disappeared with XPath 1.0. It should have sequences. Quoting from XPath 3.1, "Sequences replace node-sets from XPath 1.0. In XPath 1.0, node-sets do not contain duplicates. In generalizing node-sets to sequences in XPath 3.0 and XPath 3.1, duplicate removal is provided by functions on node sequences."
Hi Daniel, This way of reporting things is fine, but even better is raising an issue -- yes, even small editorial fixes are best handled as issues (you may want to label them "editorial"). Re XPath versions, I'm not sure how much we need to dwell on this in this document. I think I will try to come up with a pull request that moves some of this material into an appendix as we said, but I think I'll focus on for a couple of days.
Does applying a JSONPath query to a JSON data item always generate the same result? Some selectors do not have a natural order of the nodelists they return, e.g., $. applied to a JSON object We could prescribe an order (e.g., sorting keys lexicographically or in JavaScript order) if determinism is important We could disallow . on JSON objects
hmm .. depends, how deterministic JSON parsers are when iterating over objects ... they should work in ascending chronological order ... URL JavaScript 'URL' also guaranties the same order URL So ... order should be always the same, as long the implementation is the same. How different implementations behave is uncertain at best. But why should we care and heal deficiencies of JSON parsers in that case? I can't see a benefit of always guaranteeing the same -whatever- order. And disallowing iterating over objects by '$.*' might break existing implementations.
A minor benefit would be that the Compliance Test Suite would be simpler if the output always had a defined implementation-independent ordering. Agreed.
(Determinism in interop testing is a major benefit for me. But that can be added by reprocessing after the JSONpath processing, so it is not completely a killer argument.)
Another selector which requires careful definition for ordering is , particularly when this recurses over an object or objects.
hmm ... compliance testing is a point, sure. Simple first tests of and regarding ordering should be possible with current test suite ... ?
Anything is "possible". ;-) Tests involving objects which have only one name/value pair are trivially easy to test because any non-determinism of object enumeration is irrelevant. But tests involving objects with two or more name/value pairs need to capture the ordering possibilities somehow. Note that the is a JSON document so there would need to be a declarative way of specifying the ordering possibilities.
We covered this topic over in the JSON Schema repo somewhere with quite a lengthy discussion. I'll see if I can dig it up. In short, parsers in some languages just can't be made to be deterministic. Reading the same object from a file (where the keys are certainly in a defined sequence) will yield differing orderings when printing or even debugging the value. This topic, therefore, falls into the "language agnosticity" category. I don't think it's something we can enforce.
is accepted (and correctly evalueted) by ~93% of the implementations on the . Each of them has a number of users, it is really important to not disrupt their existing expressions and their habits. I don't think determinism is that important, on a object evals to an array but semantically it is an unordered set (which is not a JSON type, therefore a different type is used ¹) which is still a meaningful result and developers should remember that. Hence we shouldn't discard this feature. Anyway what I found confusing when implementing was the lack of any official recommendation about on a object behaviour. I think we should clearly state that the result has to be treated as unordered but we might suggest using lexicographic order for validation purposes when possibile (I think a huge number of implementations are using ordered maps for JSON objects storage so it shouldn't be hard). Removing an implemented fature used from a number of users for testing purposes is something that I wish to not discuss. I think a number of JSONPath implementations are completely decoupled from the JSON parser, and they just accepted maps/lists/atomic-values as input ( is one of them, it doesn't depend on any JSON parser and it doesn't care about JSON parsing). I'm not even sure if JSONPath is only used for JSON inputs, it works so nicely with any JSON-like structure. ¹ I'm not 100% sure that all implementations are using JSON types, there might be implementations using unordered sets for real as containers for results.
A noble goal, to be sure. But I imagine that sudden conformance to a spec would natural engender breaking changes, especially in a landscape of Implementation as large as we're dealing with. I don't think it'll be avoided. As such, my feeling is that we get the spec right rather than make concessions just because the majority does it some way. Absolutely correct.
We have some of this in our DNA, but we also have a clear mandate not to disregard the installed base (this is JSONpath, not a new protocol based on JSONpath). So any argument on this spectrum will need to be carefully weighed, because both extremes are wrong. Grüße, Carsten
I think the summary so far is We recognize that there are operations in JSONPath whose result is ordered but that order needs to be created based on unordered input. We are not going to cut off these noses to spite our face. We have a choice between imposing a language-defined order (e.g., sorting) or leaving the result non-deterministic. I gather that there is a general direction toward non-determinism, as imposing an order would be too expensive, but I'm not sure we have a conclusion here. Also:
I agree with Carsten's general direction in support of nondeterminism, simply because 8259 explicitly discourages assigning semantics to the ordering of object members. If we want determinism, we probably have to specify RFC8785 (JCS) conformance, which doesn't feel reasonable. As for testing, anyone who's doing this seriously in the real world has to have an URL() method already, right?
I also agree with supporting non-determinism, but please note that URL() won't necessarily help because the non-deterministic portions of the output won't necessarily be part of a JSONObject. For example, applying to an input consisting of a JSON object will produce a node list in a non-deterministic order. I think we'd either need to decorate such tests with an indication of where non-determinism can occur so that each test framework can compare the output to the expected value and take into account non-determinism or we'd need to have a way of capturing a set of possible alternative outputs. The latter may be preferable as it would complicate the generation of tests, which is a one-off process, but would simplify implementing test frameworks, of which there are likely to be many. (There may be other solutions too - haven't spent very long on this.)
FWIW, in implementing JSON Schema, I've found that I need both contents-equality and sequence-equality on arrays depending on the circumstance. Given that JSON Path outputs an array, and the contents of that array are likely to be generated non-deterministically due to evaluating objects, I think it makes sense for implementations to verify evaluation results (e.g. when processing a test suite) in a "contents-equality" sense. It's not per the JSON definition of array equality (which mandates sequence-equality), but I think the circumstance fits. The fact that the result array is not guaranteed to be deterministic would need to be declared in the specification, probably associated with a remark about it deviating from JSON array equalilty and perhaps even an explanation why.
Nodelists are not arrays. (They can be represented in a JSONPath API as arrays. Or as lists. Or as tuples. ...)
I do agree with common non-determinism here. Naming results as nodelists is clever. Shouldn't we define that term 'nodelist' then ?
Yes, we should define nodelist. I made a first attempt in PR now.
We need to tread carefully here. Are we saying we're comfortable outputting something that's not JSON but acts really similar? All implementations (including NAME output an array (or optionally ) as JSON. If we suddenly call the output a "nodelist" are we breaking that?
No. I agree that many APIs will output platform arrays here (which may or may not feel similar to JSON arrays). Saying that the output is a nodelist is gives them freedom to define their API and how these present the nodelist. More importantly, we make processing for embedded queries less finicky.
Daniel, nobody can (or will try to) stop you thinking about a nodelist as a JSON array, if that suits your mental model. Giving a separate name to that concept does have its advantages. It, for instance, helps finding answers to the role of nested queries in filter expressions. It also helps reflect the fact that some implementations handle nodelists different from others. So I don't know why we are having this discussion. (And why is it under "Determinism?".)
This is irrelevant because of many of the points that NAME mentioned: essentially, implementations don't use nodelists from what we can tell, they use arrays. And isn't our charter to break existing implementations as little as possible? If all implementations use arrays, changing that is not something we should consider lightly. NAME We're discussing it here because of . Following some discussion about expressions, you spammed "nodelist" comments anywhere it might be related in order to promote the idea before anyone had a chance to discuss it. So now we're discussing it. All of that said, I think we should consider the output format ( & ), determinism (this issue), and paths embedded in expressions () together. There is some subtle interplay going on. If the output is an array of objects that specify location and value, then both that format and the sequence of the items within it need to be considered when returning from an embedded path so that the remainder of the expression can handle it properly. If embedded paths are to follow some different output that operators can then use, we need to specify what that is as well. Maybe that is the idea of a nodelist. But we need to discuss it first and come to some level of agreement.
It is a pretty common technique in model building to identify important concepts, and once one has identified them, to give them (tentative) names. That allows the participants to refer to the concept-in-formation, and to gradually increase certainty about it. Once there is enough certainty, it may be possible to identify some concepts with each other and turn them into one, but you don't do that before achieving that certainty. I think much of the confusion comes from a perception that whatever is in the draft at one time is cast in concrete. That is not a healthy way to run this, and maybe I was making an unwarranted assumption that this would be obvious. So, indeed, feedback is good, but not so much about reasons wanting to eliminate/equate concepts prematurely and end the discussion about them before understanding them, but more about the properties of the concept itself. The fact that XPath arrived at a specific form of nodelist after trying nodesets is certainly interesting feedback. We may not need that complexity, but it is much easier to eliminate complexity once we understand the concepts.
I'm not sure this is sufficient. Let me explain why with an example. Take the input argument: and the selector . I think the valid outputs should be: But I think the following output, for example, should be invalid: The spec should allow for this level of non-determinism, but not more. Similarly, if the Compliance Test Suite covers this example, it should allow precisely the valid outputs to pass. I don't see how a "contents-equality" comparison of arrays would help. This makes me favour encoding sets of possible outputs in the CTS as described in URL
Actually, [2, 3, 0, 1] etc. as well (the outer container also is a map).
Whoops! Yes. Edited the list so it is now complete.
I'm curious why you think this should be invalid. I understand how you got the result, and if you include (or replace with) the paths to those values, it's just as (or more) clear. Partial determinism is going to create a bunch of weird edge cases that will be impossible to completely define.
That of course depends on the semantics of the nodelist. If, as I'm assuming, it is sequence-preserving, you cannot arrive at [0, 2, 1, 3], which would be (JSON pointer syntax, I'm lazy) /a/c, /b/e, /a/d, /b/f.
This is quite a complicated discussion. I'm trying to sum things up: JSON objects as maps are unordered by nature. Due to that, JSONPath query results are non-deterministic regarding their order – which is common sense here. But determinism i.e. reproducability of JSONPath query results is required for compliance tests. Now implementations usually return query results as an array – pragmatically JSON in ... JSON out. JSON arrays are ordered by nature, which might falsy signal determinism of results. So calling the result a "node-list" might help here out, but introduces another level of complexity in terms and their usage. Having understood, that determinism is solely required for compliance testing, I am not sure, how to interprete Glyn's example above. Should future compliance tests really tell about every possible valid and invalid result ordering? I am sure someone of us would be able to implement something, exactly reproducing as result (noone would of course). Thus it's implementation dependent. But if we only need to insure, to always generate a single reproducable result, we can achieve this by something like ... Generate a list of output pathes. Sort it lexically or however. Generate the list of values from that path list – preserving order. ... which we can even communicate in the draft as a recipe. Regarding "node-list", there is indeed in JavaScript a concept of "array-like objects". Those objects only need to be able to tell the list length and provide an iterator to visit all entries. Defacto dom node-lists are exactly such things. This is what above discussion also reminds me on. Maybe I still miss another important point.
Neither determinism of the spec nor determinism for a given JSONPath implementation are necessary for compliance tests, but non-determinism does add some complexity to testing. That said, I am in favour of non-determinism in the spec because it gives the most freedom to implementations.
Because I would expect each "stage" of a selector to be evaluated in turn. In other words, the intermediate results which are output from one stage to the next can be in a non-deterministic order, but that order won't be modified by other stages as those stages are operating on the individual values from earlier stages. Sorry, but what is more clear? I don't think so. I think once we have addressed the non-determinism of specific selector "stages" applied to objects, that's pretty much it. What you describes as "partial determinism" is simply a result of how the spec is constructed from small pieces in a uniform way.
To be clear, non-determinism in the spec doesn't force implementations to be non-deterministic. But it allows implementations to trade off efficiency against non-determinism. It also allows for variations between (deterministic) implementations.
Also related to (maybe duplicates) .
I think in an ideal world, it would be good to use only and exactly the 8259 terminology. "Value", "JSON Text", "Array", "Object". The latest draft isn't fresh enough in my mind to have an opinion as to whether this is possible. I think the summary so far is We recognize that there are operations in JSONPath whose result is ordered but that order needs to be created based on unordered input. We are not going to cut off these noses to spite our face. We have a choice between imposing a language-defined order (e.g., sorting) or leaving the result non-deterministic. I gather that there is a general direction toward non-determinism, as imposing an order would be too expensive, but I'm not sure we have a conclusion here. Grüße, Carsten

A structure complying to the generic data model of JSON, i.e., composed of containers, namely JSON objects and arrays, and of atomic data, namely null, true, false, numbers, and text strings.  <ins> Also called a JSON value.  In this document, often abbreviated to _item_.  The term _node_ has the same meaning, but is used to emphasize the location of an item within the Argument, i.e., an item identical to or nested within the JSON data item to which the query is applied. </ins> A JSON object as defined in RFC8259.  Never used in its generic sense, e.g., for programming language objects. The name in a name/value pair constituting a member.  (Also known as "key", "tag", or "label".) <ins> A JSON array as defined in RFC8259.  Never used in its generic sense, e.g., for programming language objects. </ins> An item in an array.  (Also used with a distinct meaning in XML context for XML elements.) <ins> A non-negative integer that identifies a specific element in an array.  The term, in particular in the form indexing, is also sometimes used for either an index or a member name, when both arrays and objects can be fed to the indexing operation. </ins> Short name for JSONPath expression. Short name for the JSON data item a JSONPath expression is applied to.  <ins> A simple form of JSONPath expression that identifies a node by providing a query that results in exactly that node.  Similar to, but syntactically different from, a JSON Pointer RFC6901. </ins> 1.2. for paths input to a JSONPath processor.  [1] Where a JSONPath processor uses JSONPath expressions as output paths, these will  <ins> always be converted to Output Paths which employ the more general _bracket-notation_.  [2] Bracket notation is more general than dot notation and can serve as a canonical form when a JSONPath processor uses JSONPath expressions as output paths. </ins> JSONPath allows the wildcard symbol "*" for member names and array indices.  It borrows the descendant operator ".." from E4X and the and "end" depend on the sign of "step", as follows: Slice expression parameters "start" and "end" are not directly usable  <ins> as slice bounds and must first be normalized.  Normalization for this purpose is defined as: </ins> The result of the array indexing expression "[i]" is defined to be the result of the array slicing expression "[i:Normalize(i)+1:1]".


<del> JavaScript Object Notation (JSON) Path </del>  draft-ietf-jsonpath-base-latest Abstract

(This was original raised in and now being split out) At IETF 110 we reached rough agreement for “JSONPath: Query expressions for JSON” to be the title of the document.
Created PR
Closed in

<ins> JSONPath: Query expressions for JSON </ins> draft-ietf-jsonpath-base-latest Abstract