Parent operator proposal

[Allam76]

A while ago I said that I would describe the way a parent operator works in legacy programming languages.
Since then I discovered the power of Symbols.

In legacy we would work with classes so to add behaviour to data we would create an access class as a layer between the framework(jsonata) and the data. In there pointers to the data could be stored to indicate, for instance, parent of current data.

Drawback of this solution is that every call to the evaluation part has to be redirected to this class. That is quite a bit of interfacing.

Now with javascript we have symbols as a metadata annotation that can be added directly to the data without pollution the look and feel. We could add either a reference, a copy or a path so that we can easily retrieve the parent.
In the jsonata common scope add:

var parentSymbol = Symbol('parent');

and for instance use a standard name in the evaluateName method:

            if(expr.value === "_parent" && input[parentSymbol]){
                result = input[parentSymbol];
            } else {
                result = input[expr.value];
            }

Then whenever a name is traversed, the symbol must be added, with the parent:

            if(typeof result === "object" && result !== null && !result[parentSymbol]){
                result[parentSymbol] = input;
            }

In the debugger we can see the symbol but it cannot be accessed outside of the jsonata scope.

This above example is not complete since it does not deal with arrays, but you get the point.
A similar approach can be used to store paths. This to do even more complicated accessing.

Here is an idea of path + access class in java
Let me know what you think.
Cheers
Martin

[Allam76]

When I think of it, the access class can for paths be nicely implemented in javascript with the proxy object.

In this way we could add custom behaviour without polluting jsonata. This is an alternative to the mixin: #87.

This is actually so powerful that you can implement your own path based DSL. I use this in a myriad of ways, for instance like a kind of mini ORM.

[andrew-coleman]

Interesting. I've also been looking at symbols as a way of fixing #296. I'll look into what can be done here. Thanks!

[Allam76]

Just solved this neatly for my use case with proxy and symbol:

    var res = jsonata("data.event.location.parent").evaluate(setProxy(data));

    function setProxy(data){
        var parentSymbol = Symbol("parent");
        var handler = {
            get: function(obj,prop,proxy){
                if(prop == 0){
                    console;
                }
                if(obj[prop]){
                    if(prop.match(/[0-9]+/)){
                        obj[prop][parentSymbol] =  obj[parentSymbol];
                    } else {
                        obj[prop][parentSymbol] = new Proxy(obj,handler);
                    }
                }
                if(!obj[prop] && prop == "parent"){
                    return obj[parentSymbol];
                } else if(typeof obj[prop] == "object"){
                    return new Proxy(obj[prop],handler);
                } else {
                    return obj[prop];
                }
            }
        }
        return new Proxy(data,handler);
    }

Note that for array parent is set, not the array itself. This implementation requires no changes to jsonata at all. In fact, proxies can be used to create surprisingly advanced DSL's.

Maybe useful in a FAQ section or recipe?

[andrew-coleman]

I've now got this mostly implemented. I propose to use the symbol '%' within a path expression to represent the 'parent' of the current context. I define 'parent' to be the enclosing object which has the property representing the context value.

Example:

Account.Order.Product.{
  'Product': `Product Name`,
  'Order': %.OrderID,
  'Account': %.%.`Account Name`

This returns an array of objects for each product in each order in each account. Information from the enclosing Order and Account objects can be accessed using the parent operator.
The repeated combination of %.%. is used to access the grandparent and higher ancestors.

There has been some discussion on slack as to which symbol to use for this. I had originally proposed to use the exclamation mark '!' since that is currently not used. However, there were concerns that this would be mistaken for a Boolean NOT operator as used in many languages (although not JSONata - it has a $not() function in common with XPath). Follow-on proposals included ^^ or ^. My concern with these is that they could get confused with the 'order-by' operator which is also used in path expressions. Although the parser won't get confused (parent is a prefix symbol, order-by is infix), the mixture of the two in a single path expression certainly confused me. The advantages of %:

• Currently only used as the mathematical modulo operator (infix).
  • Prior-art here - the mathematical multiplication operator * is also used as the wildcard symbol.
• Visually looks like it could represent a parent relationship - i.e. an object above another object.

[loganvolkers]

I'm not sure if this is too late to continue the discussion on the syntax. I wanted to think a couple steps ahead to some of the other possibilities for looking "up" the tree to do other ancestor queries.

In general I think that building on the $ operator would be the best way to achieve this, since it aligns with the root ancestor syntax $$.

I'm not sure how the tokenizer and parser might be impacted by some of these changes, sorry if it's obvious why these are impossible

Existing
$ - current
$$ - root ancestor (greatest grandparent)

Parent-only:
$$$ - parent
$$$.$$$ - grandparent

Numbered ancestor:
Similar to the prior art of the sort operator ^ requiring an attached block
$(0) - current -- shortcut: $
$(1) - parent --- shortcut: $$$
$(2) - grandparent
$(3) - great grandparent
$(*) - greatest grandparent -- shortcut: $$

Wildcard ancestor:
$* - wildcard ancestor (would match parent, and any parents
$*[name="Doe"] - wildcard ancestor with filters

[andrew-coleman]

I'd love to see some compelling use-cases that would justify the need for all of these. The case for adding a parent accessor was clearly made (several times) with examples. Currently, to access information in the parent structure means binding it to a variable and referring to it in a sub-expression as in the following example:

Account.Order.(
  $o := $;
  Product.{
    'Product': `Product Name`,
    'Order': $o.OrderID
  }
)

http://try.jsonata.org/rJVVfeUZU

With a parent operator, this is achieved using:

Account.Order.Product.{
  'Product': `Product Name`,
  'Order': %.OrderID
}

which is conceptually much simpler.

Syntactically, the dollar $ represents variable names, and I'd like to keep that association.
There are two 'special' variables - the overall input json (root) is pre-bound to $$ and the unnamed variable $ refers to the current context anywhere in an expression. This context value could be a function (this is a functional language), in which case $(0) would invoke that function with the argument 0. For example:

[$string, $boolean, $sqrt].$(0) evaluates to ["0", false, 0]
[$string, $boolean, $sqrt].$(2) evaluates to ["2", true, 1.414213562373]

Going forward, we could add the operator %% to return a sequence of all ancestors (analogous to ** selecting all descendants), but I'd like to see compelling examples of usage before doing this.

[loganvolkers]

Thanks @andrew-coleman for the illustrative examples.

[$string, $boolean, $sqrt].$(2) evaluates to ["0", false, 0]

This blew my mind and clearly kills the $(1) syntax.

Going forward, we could add the operator %% to return a sequence of all ancestors (analogous to ** selecting all descendants), but I'd like to see compelling examples of usage before doing this.

I don't have these use cases.

I guess $$ is really just a special character, and not an ancestor query, and $ is better left for variables / functions.

In conclusion I support the % operator and retract my support for syntaxes based on $