Incorrect behavior of $@ and $* when IFS=''

[xPMo]

set -- a 'b c'
IFS=''
printf '[%s]\n' $@ $*

In bash or dash, this prints:

[a]
[b c]
[a]
[b c]

In osh, this prints:

[ab c]
[ab c]

The same applies to bash arrays under ${a[@]} and ${a[*]}.
In general, an array without quotes should not be subject to joining. First substitute all words, then split them on IFS. (Exceptions: [[ and case)

[andychu]

Wow interesting, I'm not sure why this is yet, I verified that bash/mksh/zsh/busybox ash agree.

However dash 0.5.8 agrees with osh.

[xPMo]

Dash 0.5.10 agrees with Bash, so does whatever version of dash this is

My sha1sum is 847e4cc4f63e83160d166574e5e312f5f85c9772

[xPMo]

2014-10-08 Herbert Xu [email protected]

• Split unquoted $@/$* correctly when IFS is set but empty.

Changelog

Commit

[andychu]

OK thanks a lot! That helps. I verified this bug through the spec tests.

I think it should be pretty easy to fix but I'm not quite sure. I think test case 8 here is related

https://www.oilshell.org/release/0.8.pre2/test/spec.wwz/assoc.html

[xPMo]

I think test case 8 here is related

It is similar, but I don't think it's related. Unquoted $* or $@ shouldn't join, while quoted "$*" shouldn't split:

$ set -- a 'b   c'
$ printf '[%s]\n' "$*"
[a b   c]    # osh prints "[a b c]"

[xPMo]

Generalizing the behavior with an arbitrary string concatenated before or after the ${@}/${*}:

(Suppose "$@" has four elements.)

"pre${@}post"

• Substitute "pre${1}" "${2}" "$3" "${4}post"

"pre${*}post"

• Let $I be the first character of IFS.
• Substitute "pre$1$I$2$I$3$I$4post"

pre${@}post or pre${*}post

• Word splitting behaves differently depending on whether $IFS splits on a usual space character or not. I did not know this until today!
• expand pre${@}post according to the first case, then subject each word (except for what belongs to "pre" or "post" to word splitting.

Examples

EDIT: I just checked osh, and it does normal word splitting correctly, unless it crashes. One sec, I have a new issue :P

[andychu]

FWIW these are the test cases for splitting, although there is a TODO at the end to test more of $@ and $*, which is right where you found the bug! :)

https://www.oilshell.org/release/0.8.pre2/test/spec.wwz/word-split.html

Honestly it surprised me that all shells agree so thoroughly on word splitting, beyond what POSIX seems to specify. (except dash 0.5.8 I guess)

I was able to find divergences in basically all shell features, but somehow with word splitting everyone agrees, even though it's quite a complicated algorithm ...

(And I know how to fix the IFS='\' bug, which is due to an internal implementation detail, but that might take some effort ...)

[Crestwave]

Honestly it surprised me that all shells agree so thoroughly on word splitting, beyond what POSIX seems to specify. (except dash 0.5.8 I guess)

This seems to be POSIX, though? Or are you just referring to the other tests? Interestingly, though, I found this page with the same results as OSH but I don't really know what it's for.

set "abc" "def ghi" "jkl"
...
IFS=" # null
...
printf '%s\n' ${var-$*}
abcdef ghijkl

The actual specification, though, says (emphasis mine):

Expands to the positional parameters, starting from one, initially producing one field for each positional parameter that is set.

[andychu]

#331 also mentions an empty IFS difference