Link Dump #2 - 2020-04-14

The second, greatly delayed, link dump - a place for me to offload the interesting articles or pieces or software I come across while trawling the internet.

Benchmarking & Stress-testing

Tools to evaluate the performance of commands and the systems they run on.

Hyperfine: A wrapper command to perform statistical analysis on the performance of other CLI tools.

Stress-ng: As a rule, I try to avoid CLI applications which have hyphens in their command names. Not really sure why. Regardless, this one is too good to pass up - it offers a highly configurable way to apply resource stress to a machine (lots of CPU usage, lots of memory accesses, etc).

Backups & Storage

Restic: "Backups done right" - a tagline I whole-heartedly agree with. Restic allows storage to be backed up as encrypted deduplicated snapshots to a hosted server or directly to object storage.

Borg: An alternative to Restic with compression as well as encryption and deduplication, but without the ability to save to object storage.

MinIO (+ RADIO): MinIO is a simple object storage server for self-hosting, with configurable erasure coding across multiple disks and servers. RADIO is a higher-level abstraction which provides caching as well as distributing data (via mirroring, as well as erasure coding) across multiple object storage targets.

Standalone CLI Utilities

A catch-all segment for cool CLI applications I use somewhat frequently.

UPX: A slightly "meta" choice - UPX is used to reduce the size of binaries. Together with the "strip" command, this can make a huge difference to the size of application downloads, and so is something I hope to see used more often.

Monolith: A tool for archiving full websites as a single file, for data hoarding or offline reading purposes.

FirefoxSend CLI: Firefox Send has become my go-to way of moving files between machines where shared-access object storage or NFS mounts aren't available. Being able to limit accesses to a single download, on a single day, behind password encryption makes me much more comfortable about what is effectively "dump this thing on the internet to grab it later". The official website is handy when on a Windows desktop, but for any other use-cases my default is to use this CLI tool.

Creating CLI Utilities

Turning a set of scripts or functions into a tool can range from "easy peasy" (Rust, Go) to "pain in the ass" (Bash, Python) - these links should help with the latter.

The Many Layers of Packaging: Returning to Python - this is the best guide I've come across on how to convert projects into distributable applications.
AppImage: One of the techniques listed in the previous link on Python package has more general usefulness, and can be used to distribute arbitrary executables along with their own dependencies as a single binary (actually an executable SquashFS image).

Security, Isolation & Containerization

I have documented previously that I've tried to use LXD and Multipass to manage development environments. Personally I'm happy with those two, but here are some alternatives which might be more appropriate in other situations.

Firejail: A CLI tool to isolate untrusted applications into their own sandbox, using the full range of isolation capabilities already available via the Linux kernel.

Toolbox: A CLI tool for generating development environments as OCI containers, with a focus on providing the software environment while allowing easy access to the underlying user file-system.

You'll only receive email when ChrisDowning publishes a new post

More fromĀ ChrisDowning