Output of declare -p / declare / declare -f / trap -p / trap (ble.sh)

[akinomyoga]

Oil does not print the definition of variables for declare -p. The script ble.sh uses declare -p to save states of scalar variables and arrays (saved=$(declare -p foo bar)) which will be later used to restore the states by eval (eval -- "$saved"). I believe this is the expected usage of Bash declare -p (which is the reason that it prints the reusable form for eval).

$ bash -c 'a=1; declare -p a; declare -p b'
declare -- a="1"
bash: line 0: declare: b: not found
$ bin/osh -c 'a=1; declare -p a; declare -p b'
a
$

Similarly Bash prints the definition of all the variables and functions with declare without arguments, but Oil does nothing for this.

$ bin/osh -c 'a=1; declare'
$

In Bash, the function definition can be obtained by declare -f. But Oil does not provide the function body but only the function name. The script ble.sh uses declare -f to save/restore the functions or dynamically patch the behavior of existing functions. (If you are interested, maybe you can have a look at meta functions ble/function#push, ble/function#pop, and ble/function#advice in src/util.sh of the ble.sh source for save, restore, and patching, respectively).

$ bash -c 'function aaa { echo aaa; }; declare -f aaa'
aaa ()
{
    echo aaa
}
$ bin/osh -c 'function aaa { echo aaa; }; declare -F aaa'
aaa
$

Is there any other way in Oil to obtain the body of existing shell functions? I tried traptype but it neither output the body.

$ bash -c 'function aaa { echo aaa; }; type aaa'
aaa is a function
aaa ()
{
    echo aaa
}
$ bin/osh -c 'function aaa { echo aaa; }; type aaa'
aaa is a function
$

Bash builtin trap also produces reusable form which can be used for save and restore of trap states. However, Oil produces not reusable form of outputs.

$ bin/osh -c 'trap "echo hello" INT; trap -p'
2 _TrapHandler

In Bash, trap without arguments prints the current set of the registered traps in the same way as trap -p. But Oil produces an error.

$ bin/osh -c trap
  trap
  ^~~~
[ -c flag ]:1: 'trap' requires a code string
$

[andychu]

Yes I've seen this in other scripts too. Unfortunately it's not implemented, but I'll leave this issue open and see who else runs into it.

[andychu]

Note: Oil is trying to discourage use of eval (and silent evals like export -f), so I'm not sure what we want to do here. Saving state could be done in a different way.

eval from the file system is generally dangerous, even if you think you control the file ...

[akinomyoga]

Thank you very much for reviewing all the issues!

Note: Oil is trying to discourage use of eval (and silent evals like export -f),

OK, for this point, I actually agree with you. One doesn't need eval unless one writes a complicated shell script, and I don't really recommend other people to write complicated programs like ble.sh in shell scripts. One should use other appropriate languages or reformed shell languages like Oil Language (though I haven't learned Oil Language, sorry).

The situation is special for ble.sh which needs to handle user inputs within the shell instance without slow operations like fork & exec. There might be another option of the implementation such as communicating with Bash using sockets or pipes, but ble.sh was started from just a experimental code written directly into my bashrc. I didn't expect the code becomes this large back then, but now I'm somehow satisfied with the current situation.

Saving state could be done in a different way.

Yes, but it's usually more handy and efficient to use declare -p (although some old Bash versions have bugs on declare -p).

eval from the file system is generally dangerous, even if you think you control the file ...

I'm sorry that I didn't explain the usage in details, but save/restore does not (necessarily) mean it's saved into some file and read from it. It's just saved into a shell variable and later read from that variable (as indicated by the original post saved=$(declare -p foo bar) & eval -- "$saved") in ble.sh. Every restore/save mentioned in the original comment is made by using shell variables in ble.sh.

There is also another usage of eval & declare -p than save/restore. When one wants to obtain results of subshells which should be stored in multiple variables, a typical solution is

function fun {
  do_something

  eval -- "$(
    some_command | {
      do_something
      declare -p resultA resultB resultC; } )"

  use_results
}

Digression: Use case of saving states in a shell variable in ble.sh

Maybe it is non-trivinal why ble.sh needs to do such things (saving states in a shell variable). The shell script language lacks the feature of object-oriented structures, so if one wants to handle some objects (which are composed of multiple member variables), some mechanism to carry and specify a set of variables. The script ble.sh uses several different ways to specify the set of variables depending on the usage and frequency of the switching of the object to be handled.

• One of such techniques is actually something like ((${prefix}_memberName)) as you have already seen in osh -n. The object to which the operation is applied is controlled by the shell variable prefix. This is easy for the members of integer types, but one needs to rely on annoying chain of eval or printf -v once one wants to store non-integer values in the members.
• Another way is to store multiple fields separated by special separator characters (such as spaces, commas, or colons) in a single variable. This requires packing and unpacking of the stored values every time we manipulate these values, which is additional computational costs. Also one needs to put some restrictions on the member values (i.e., separator character cannot be used), or one can use a more sophisticated serialization scheme (ble.sh does both in different places).
• When the target object is rarely switched, one can use another way. One can fix the set of the global variables that represents the object state and operate on these variables directly. When one wants to switch the object which one wants to handle, entire state of the set of variables are saved into a shell variable by obj1_saved=$(declare -p list_of_member_variable_names) and then load the state of another object by eval -- "$obj2_saved".

[andychu]

Regarding ble.sh and #653, I also wonder if #704 to eval code in a subinterpreter would eliminate the need for declare -f?

That is, if there is an interpreter for ble.sh and one for the user, then we wouldn't have to save and restore state with declare -f (or declare -p)?

---

But I understand that could be a big change to the codebase and maybe not worth it now. I point it out since declare -f is a starred issue.

(delare -f is possible in Oil but has some relation to memory management, which we haven't tackled quite yet. Or maybe we never need to free functions ...?)

[akinomyoga]

I also wonder if #704 to eval code in a subinterpreter would eliminate the need for declare -f?

Maybe it could partially eliminate the need for declare -f in ble.sh, but not all the usage is related to save/restore for sub-interpreters. Actually, most part of declare -f / declare -p usage is nothing to do with user-command execution. Save/restore for user-command execution is directly made by normal variable assignments. declare -p is used mainly for the case that the number and names of the saved variables change dynamically.

I point it out since declare -f is a starred issue.

The reason why I marked with a star is because there is no other way to achieve this. Actually, this is not so important for running the essential part of ble.sh.

(delare -f is possible in Oil but has some relation to memory management, which we haven't tackled quite yet. Or maybe we never need to free functions ...?)

Maybe you are talking about save/restore of functions without providing a string representation of the function body, but I just want to obtain a string representation of the function body. ble.sh uses declare -f for three different situations:

1. save/restore function definitions (in ble/function#{push,pop})
2. rename functions (in ble/function#advice)
3. show definition of the function to user (in ble/widget/command-help)

[andychu]

OK great, thanks for clarifying the use cases. That helps.