Implementations of Shell Autocompletion

Back to Shell Autocompletion

bash

• GNU readline does ad hoc tokenization of the shell command. Does not use its parser.
• builtins: complete, compopt, compgen
• global variables: COMP_WORDS, COMPREPLY (out)

• completions maintained out of tree in bash-completion

yash

• lineedit/compparse.{c,h} (~1300 lines) is a "simple parser for command line completion" that duplicates knowledge of the shell language. It doesn't use its own parser much, if at all.
• help complete shows help for the extensive completion builtin. Doesn't seem to understand the flags of its own builtins.
• completions maintained in-tree in share/completion

As usual, git is one of the biggest completions!

~/src/languages/yash-2.46/share/completion$ wc -l *|sort -n
...
   497 git-config
   541 tar
   554 find
   597 git-svn
   657 INIT
   759 git
   850 svn
 19921 total

zsh

TODO

• completions maintained in-tree

fish

• has a complete builtin
• completions maintained in-tree

Oil

Oil reuses its parser for autocompletion.

I haven't documented this fully, but here's one blog post:

• http://www.oilshell.org/blog/2020/01/history-and-completion.html#how-it-works

Notes:

• The parser drives the lexer, and most decisions are made from the tokens saved in the "trail". You can examine these tokens even if the parser fails.
• Code is in core/completion.py. RootCompleter::Matches() is the core logic, which is somewhat hairy, but much simpler and more compact than the equivalent other shells. See trail.tokens.
• Other things to search for in the code:
  • Id.Expr_Dummy. This is a dummy token that helps parse incomplete code (I honestly don't remember the details, but I recall that a lot of things fell out nicely with this little mechanism.)
• We don't have to solve the "incremental parsing" problem, because we just reparse/re-lex the entire line every time you hit tab. The parser runs at ~800 lines/ms, so it should take a few microseconds to re-parse. So the parsing time will be dwarfed by running shell completion plugins (which are arbitrary shell), and especially shell plugins that do I/O.

Bugs

• https://github.com/oilshell/oil/issues/227 -- I think this is relatively easily handled by our scheme, but I haven't thought about it in awhile.