Rust CLI Project Template

A base project template for comfortably building small but reliable utilities in the Rust programming language.

Rationale

Historically, I have used Python for quick little scripts (from which my big, long-lived projects tend to unexpectedly develop), but maintaining large things written in Python risks turning into a gumption trap due to the need to either manually test every code path I affect when making changes or pour a lot of effort into writing an automated test suite to do it for me.

Rust's stronger compile-time guarantees remove most of that motivation-sapping busywork, but, since I've spent so many years getting comfortable with Python, working in a new language risks being a chicken-and-egg problem which kills off the will to change tasks. (Once I start something, it's easy to keep going.)

The purpose of this repository is to lower the friction for Rust-based development of such "scripts of unknown potential" as much as possible. (Hence the inclusion of aliases like just run which serve only to remove the need to think about whether to type just or cargo, and the wrappers for cargo-edit commands which regenerate the API documentation I'll have open in a browser tab after adding/removing/updating dependencies.)

In short, this is a "do it properly" answer to the workflow that evolved around the boiler snippet for Python that I keep in my UltiSnips.

License

This repository is licensed under your choice of the MIT or Apache 2.0 licenses with the exception of the license texts themselves. Please replace the LICENSE file in the template folder with your preferred license if you do not habitually start your new projects under the GNU GPLv3.

Features

• Uses StructOpt (with colourized --help output and "Did you mean...?" suggestions enabled) for argument parsing because gumdrop doesn't support OsString-based parsing for correct handling of non-UTF8 paths.
• Uses anyhow for unified error handling.
• Presents an app::main(opts: CliOpts) -> Result<()> function to keep your application logic cleanly separated from argument parsing and handling of fatal errors.
• Exposes clap's support for generating shell completions by providing a --dump-completions <shell> option.
• Enables almost all rustc and clippy lints without making clippy mandatory.
• A comprehensive set of just commands, easily customized via variables (eg. for cross-compilation), including install and uninstall, which also take care of shell completions and a manpage.
• just build-dist for a 100% static x86_64 binary that starts at roughly 272KiB (248KiB with panic="abort") in new projects.
• just install-deps to install all but two optional dependencies on Debian-family distros.
• just install-cargo-deps to install all distro-agnostic dependencies.
• A basic .travis.yml for use with Travis-CI and Nightli.es.
• The just fmt command always calls the nightly version of rustfmt to ensure access to the excessive number of customization options which are gated away as unstable.

Usage

1. Clone a copy of this repository (.git must exist, so no archive downloads)
2. Run apply.py path/to/new/project
3. Edit src/app.rs to implement your application logic

NOTE: As a safety measure, apply.py generates new projects from the git HEAD of the template repository (not the working tree), so any local changes you make will not be picked up until you commit them.

Supplementary Files

Metadata
LICENSE	A copy of the GNU GPLv3 as my "until I've had time to think about it" license of choice. Make a local commit which replaces this with your preferred default for new projects.
CONTRIBUTING	A copy of the Developer Certificate of Origin, which is the Linux kernel developers' more ideologically appropriate alternative to CLAs as a means of legally armouring themselves against bad-faith contributions
Configuration
.gitignore	Ignore /target and other generated files
clippy.toml	Whitelist for CamelCase names which trigger Clippy's "identifier needs backticks" lint
rustfmt.toml	A custom rustfmt configuration which shows TODO/FIXME comments and attempts to make it conform to the style I'm willing to enforce at the expense of not using rustfmt if necessary.
Development Automation
apply.py	Run this to generate new projects as a workaround for cargo-generate's incompatibility with justfile syntax
justfile	Build/development-automation commands via just (a pure-Rust make-alike)
Support Code You May Borrow
gen_justfile_reference.py	Code which is used to regenerate the reference charts for justfile variables and commands in this README so it's easy to keep them up to date.
test_justfile.py	A test suite for my justfile which you may want to adapt for your own projects.

Justfile Reference

Variables (just --evaluate)

Variable	Default Value	Description
CARGO_BUILD_TARGET	x86_64-unknown-linux-musl	The target for cargo commands to use and install-rustup-deps to install
build_flags		An easy place to modify the build flags used
channel	stable	An easy way to override the cargo channel for just this project
features		Extra cargo features to enable
build-dist
sstrip_bin	sstrip	Set this if you need to override it for a cross-compiling sstrip
strip_bin	strip	Set this to the cross-compiler's strip when cross-compiling
strip_flags	--strip-unneeded	Flags passed to strip_bin
upx_flags	--ultra-brute	Flags passed to UPX
kcachegrind
callgrind_args		Extra arguments to pass to callgrind.
callgrind_out_file	callgrind.out.justfile	Temporary file used by just kcachegrind
kcachegrind	kcachegrind	Set this to override how kcachegrind is called
install and uninstall
bash_completion_dir	~/.bash_completion.d	Where to install bash completions. You'll need to manually add some lines to source these files in .bashrc
fish_completion_dir	~/.config/fish/completions	Where to install fish completions. You'll probably never need to change this.
manpage_dir	~/.cargo/share/man/man1	Where to install manpages. As long as ~/.cargo/bin is in your PATH, man should automatically pick up this location.
zsh_completion_dir	~/.zsh/functions	Where to install zsh completions. You'll need to add this to your fpath manually

Commands (just --list)

Command	Arguments	Description
DEFAULT		Shorthand for just fulltest
Development
add	args (optional)	Alias for cargo-edit's cargo add which regenerates local API docs afterwards
bloat	args (optional)	Alias for cargo bloat
check	args (optional)	Alias for cargo check
clean	args (optional)	Superset of cargo clean -v which deletes other stuff this justfile builds
clippy	args (optional)	Alias for cargo clippy which touches src/* to work around clippy bug
doc	args (optional)	Run rustdoc with --document-private-items and then run cargo-deadlinks
fmt	args (optional)	Alias for cargo +nightly fmt -- {{args}}
fmt-check	args (optional)	Alias for cargo +nightly fmt -- --check {{args}} which un-bloats TODO/FIXME warnings
fulltest		Run all installed static analysis, plus cargo test
geiger	args (optional)	Alias for cargo geiger
kcachegrind	args (optional)	Run a debug build under callgrind, then open the profile in KCachegrind
kcov		Generate a statement coverage report in target/cov/
rm	args (optional)	Alias for cargo-edit's cargo rm which regenerates local API docs afterwards
search	args (optional)	Convenience alias for opening a crate search on lib.rs in the browser
test	args (optional)	Alias for cargo test
update	args (optional)	Alias for cargo-edit's cargo update which regenerates local API docs afterwards
Local Builds
build		Alias for cargo build
install		Install the un-packed binary, shell completions, and a manpage
run	args (optional)	Alias for cargo run -- {{args}}
uninstall		Remove any files installed by the install task (but leave any parent directories created)
Release Builds
build-dist		Make a release build and then strip and compress the resulting binary
dist		Call dist-supplemental and build-dist and copy the packed binary to dist/
dist-supplemental		Build the shell completions and a manpage, and put them in dist/
Dependencies
install-apt-deps		Use apt-get to install dependencies cargo can't (except kcov and sstrip)
install-cargo-deps		install-rustup-deps and then cargo install tools
install-deps		Run install-apt-deps and install-cargo-deps. List what remains.
install-rustup-deps		Install (don't update) nightly and channel toolchains, plus CARGO_BUILD_TARGET, clippy, and rustfmt

Tips

• Edit the DEFAULT command. That's what it's there for.

• You can use just from any subdirectory in your project. It's like git that way.

• just path/to/project/ (note the trailing slash) is equivalent to (cd path/to/project; just)

• just path/to/project/command is equivalent to (cd path/to/project; just command)

• The simplest way to activate the bash completion installed by just install is to add this to your .bashrc:

  for script in ~/.bash_completion.d/*; do
    if [ -e "$script" ]; then
      . "$script"
    fi
  done

  foo
  

• The simplest way to activate the zsh completion installed by just install is to add this to your .zshrc:

  fpath=(~/.zsh/functions(:A) $fpath)

• When using clap/StructOpt validators for inputs such as filesystem paths, only use them to bail out early on bad input, not as your only check. They're conceptually similar to raw pointers and can be invalidated between when you check them and when you try to use them because Rust can't control what the OS and other programs do in the interim. See this blog post for more on this idea of references versus values in command-line arguments.

Build Behaviour

In order to be as suitable as possible for building compact, easy-to-distribute, high-reliability replacements for shell scripts, the following build options are defined:

If built via just build:

1. The default CARGO_BUILD_TARGET defined in the justfile will specify a 32-bit x86 build, statically linked against musl-libc for portability comparable to a Go binary. (Unless musl-gcc is installed, this will cause build failures if you depend on any crates which link to C or C++ code.)

If built via cargo build --release:

1. Full LTO (Link-Time Optimization) will be enabled. (lto = true)
2. The binary will be built with opt-level = "z" to further reduce file size.
3. If panic="abort" is uncommented in Cargo.toml, LTO will prune away the unwinding machinery to save even more space, but panics will not cause Drop implementations to be run and will be uncatchable.

If built via just build-dist:

1. Unless otherwise noted, all optimizations listed above. [1]
2. The binary will be stripped with --strip-unneeded and then with sstrip (a more aggressive companion used in embedded development) to produce the smallest possible pre-compression size.
3. The binary will be compressed via upx --ultra-brute. In my experience, this makes a file about 1/3rd the size of the input.

NOTE: --strip-unneeded removes all symbols that readelf --syms sees from the just build output, so it's not different from --strip-all in this case, but it's a good idea to get in the habit of using the safe option that's smart enough to just Do What I Mean™.

If built by just dist:

1. A packed binary will be built via build-dist and copied into dist/
2. Shell completion files and a manpage will also be built and saved into dist/

NOTE: Depending on who you're distributing precompiled binaries to, you may want get an overview of how virus scanners react to your binary using VirusTotal.

Especially with anything involving compression, small numbers of false positives are a fact of life in the world of virus detection. For example, when I tested the official installer for the NSIS authoring tools, which is used by various major companies including McAfeee, two or three no-name entries in the list of 60+ virus scanners they test reported it to have a virus.

If this proves problematic, you can either uninstall UPX or modify the justfile so the dist command always prefers the .stripped copy of the binary over the .packed one.

Experimental Cross-Compilation Support

I am currently in the process of extending this template to support generating Windows binaries, though I have no immediate plans to replace the justfile tasks so, for now, Windows-hosted development will have to settle for calling cargo commands directly.

NOTE: I haven't yet used a fresh Ubuntu install under VirtualBox to verify that I've correctly listed all the steps needed to achieve a working build environment.

To set up an environment where setting CARGO_BUILD_TARGET to x86_64-pc-windows-gnu will complete successfully and produce a .exe file which appears to work under my preliminary testing:

1. Install a MinGW package like Ubuntu's mingw-w64
2. Run rustup target add x86_64-pc-windows-gnu
3. Add the following two lines to ~/.cargo/config:

[target.x86_64-pc-windows-gnu]
linker = "/usr/bin/x86_64-w64-mingw32-gcc"

To make cargo test also work cross-platform:

1. Make sure your ~/.wine is 64-bit (indicated by an #arch=win64 comment in system.reg)
2. Make sure your kernel's binfmt_misc support is configured to allow running .exe files in the terminal via Wine as if they were native binaries.

NOTE: Wine is only suitable for "rapid iteration, approximate compatibility" testing. For proper testing of Windows binaries, the only reliable solution is to download one of the specially licensed "only for testing" Windows VMs that Microsoft offers for download from http://modern.ie/ and those cannot be used to make legally redistributable builds.

If you want to set up a Continuous Deployment-style workflow with testing against real Windows targets, the only viable option is to bypass just and call cargo directly under real Windows. I suggest a CI service like AppVeyor for this. (See also rust-cross.)

Dependencies

In order to use the full functionality offered by this boilerplate, the following dependencies must be installed:

• just add:
  • cargo-edit (cargo install cargo-edit)
• just bloat:
  • cargo-bloat (cargo install cargo-bloat)
• just build-dist:
  • The toolchain specified by the channel variable.
  • The target specified by the CARGO_BUILD_TARGET variable.
  • strip (Included with binutils)
  • sstrip (optional)
  • upx (optional, sudo apt-get install upx)
• just fmt and just fmt-check:
  • A nightly Rust toolchain (rustup toolchain install nightly)
  • rustfmt for the nightly toolchain (rustup component add rustfmt --toolchain nightly)
• just dist-supplemental:
  • help2man (sudo apt-get install help2man)
• just kcachegrind:
  • Valgrind (sudo apt-get install valgrind)
  • KCachegrind (sudo apt-get install kcachegrind)
• just kcov:
  • A Rust-compatible build of kcov
• just rm:
  • cargo-edit (cargo install cargo-edit)
• just test:
  • clippy (rustup component add clippy)
  • cargo-audit (cargo install cargo-audit)
  • cargo-deadlinks (cargo install cargo-deadlinks)
  • cargo-outdated (cargo install cargo-outdated)
• just update:
  • cargo-edit (cargo install cargo-edit)

Dependency Installation

• Debian/Ubuntu/Mint:

  export PATH="$HOME/.cargo/bin:$PATH"
  cargo install just
  just install-deps
  
  # ...and now manually install the following optional tools:
  #  - sstrip (from ELFkickers)
  #  - kcov (version 31 or higher with --verify support)

• Other distros:

  export PATH="$HOME/.cargo/bin:$PATH"
  cargo install just
  just install-cargo-deps
  
  # ...and now manually install the following optional tools:
  #  - help2man
  #  - kcachegrind
  #  - kcov (version 31 or higher with --verify support)
  #  - strip (from binutils)
  #  - sstrip (from ELFkickers)
  #  - upx
  #  - valgrind

TODO

• Add a .travis.yml at the top level to plumb the various test suites (template repo and generated project) into CI and then add a badge.

• Add a #[cfg(windows)] version of the path_output_dir validator and make the libc dependency conditional on not(windows) so that cross-compiling for Windows using the x86_64-pc-windows-gnu target can be a viable way to quickly fire off alpha/beta-testing builds to Windows-using peers.

• Investigate how flexible QuiCLI and its dependency on env_logger are and whether it'd be useful to rebase on it or whether I'd just be reinventing most of it anyway to force the exact look and feel I achieved with stderrlog. (eg. The Verbosity struct doesn't implement "-v and -q are mirrors of each other" and I'm rather fond of stderrlog's approach to timestamp toggling.)

  • What effect does QuiCLI have on the final binary size? (not a huge concern)

• Investigate why cargo-cov isn't hiding the components of the rust standard library and whether it can be induced to generate coverage despite some tests failing. If so, add a command for it.

• Read the callgrind docs and figure out how to exclude the Rust standard library from what Kcachegrind displays.

  • I may need to filter the output. [1]
  • Figure out how to add a just task for a faster but less precise profiler like gprof [1] [2], OProfile [1], or perf [1] to make it easy to leverage the various trade-offs. (And make sure to provide convenient access to flame graphs and at least one perf inspector GUI or TUI.)
  • Include references to these resources on how profilers can mislead in different ways. [1] [2] [3]
  • Look into options for making it as easy as possible to optimize and regression-test runtime performance. [1] [2] [3] [4]

• Test and enhance .travis.yml

  • Consider officially supporting Windows as a target (probably using cargo-make instead of Just) and, if I do, come up with an appveyor.yml... possibly the one from this project.

• Add a run-memstats Just task which swaps in jemalloc and sets MALLOC_CONF=stats_print:true

• Investigate commit hooks [1] [2] [3]

• Once I've cleared out these TODOs, consider using this space for a reminder list of best practices for avoiding "higher-level footguns" noted in my pile of assorted advice. (Things like "If you can find a way to not need path manipulation beyond 'pass this opaque token around', then you can eliminate entire classes of bugs")

• At least list a snip of example code for something like rustyline as the suggested way to do simple user prompting.

• Gather my custom clap validators into a crate, add some more, and have this depend on it:

  • Self-Contained data:
    • Boolean is 1/y/yes/t/true or 0/n/no/f/false (case-insensitive, include a utility function for actual parsing)
    • Integers:
      • Can be parsed as a decimal integer > 0 (eg. number of volumes)
      • Can be parsed as a decimal integer >= 0 (eg. number of bytes)
      • Number of bytes, with optional SI mebi- unit suffix (eg. 16m, including optional b, case-insensitive)
    • Floats:
      • Can be parsed as a float in the range 0.0 <= x <= 1.0
  • Invalidatable/Referential data:
    • Input files:
      • Directory exists and is browsable (+rX)
      • Path is a readable file or browsable directory (ie. read or recurse)
    • Output files:
      • Integers:
        • Augmented "number of bytes, with optional SI mebi- unit suffix" validator with upper limit for producing files representable by ISO9660/FAT32 filesystems on removable media. (2GiB, since some implementations use 32-bit signed offsets)
      • Paths:
        • File path is probably FAT32 writable
          • If file exists, access() says it's probably writable
          • If file does not exist, name is FAT32-valid and within a probably writable directory.
        • File path is probably FAT32 writable, with mkdir -p
          • Nonexistent path components are FAT32-valid
          • Closest existing ancestor is a probably writable directory
    • Network I/O:
      • Integers:
        • Successfully parses into a valid listening TCP/UDP port number (0-65535, I think)
        • Successfully parses into a valid, non-root, listening TCP/UDP port number (0 or 1024-65535, I think)
        • Successfully parses into a valid connecting TCP/UDP port number (1-65535, I think)
      • Strings:
        • Successfully parses into a SocketAddr (IP+port, may perform DNS lookup?)
        • Successfully parses into an IpAddr (may perform DNS lookup?)
      • URLs:
        • Is well-formed relative URL (external dependency behind a cargo feature)
        • Is well-formed absolute URL (external dependency behind a cargo feature)