shell
This is a mix of shell, linux, and macOS commands. Comments are welcome with any corrections or suggestions.
CLI Usage
PowerShell & Bash Comparison
Brevity
See all aliases with Get-Alias
and to expedite your cli usage you could use a gist like this: Aliaser.ps1
Note that PowerShell eschews brevity for clarity, but you can alias anything you like to be nearly as succint as bash commands.
IMO readability/brevity trumps succintness. However for interactive terminal usage aliasing can be a great tool. Use VSCode to auto-expand aliases into fully qualified functions if you decide to turn your adhoc work into a script file.
Using pushd
in a PowerShell session actually aliases to Push-Location
.
The difference is in addition to the path string pushd manages, you get a .NET object back with [System.Management.Automation.PathInfo]
information including: Drive, Path, Provider, ProviderPath.
PowerShell Works With Native Tooling
I’ve included the similar PowerShell command to help those jumping between multiple shells.
Please note that unlike Python, PowerShell works as a terminal with native tools + scripting language.
You can use pwsh
in almost every case in Linux & macOS and use the same tools you prefer, while being able to execute PowerShell commands as well.
For example something like aws cli returning json could be automatically unmarshaled into an object instead of using jq
|
|
Another example is paths.
Prerequiresites for the PowerShell examples:
|
|
Command | shell | pwsh |
---|---|---|
View history | history |
Get-History |
Execute Line from History | !Number |
Invoke-Expression (Get-History | Out-ConsoleGridView -OutputMode Single).CommandLine |
Execute Last Command But With Sudo | sudo !! |
|
Test file exists | test -f ./filename |
Test-Path $filename -PathType Leaf or using .NET [io.file]::exists($filename) |
Installation
Common App Installs
Application | Notes | Install Command |
---|---|---|
HomeBrew | Works on Linux and macOS now 👏. | /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" |
Ansible Initialization
|
|
For docker/linux
|
|
A Few More Ansible Commands
Command | Code |
---|---|
Run ansible playbook against a specific tag | ansible-playbook main.yml --inventory inventory --ask-become-pass -t 'ui' |
Install requirements | ansible-galaxy collection install community.general && ansible-galaxy install --role-file requirements.yml --force --ignore-errors |
Installing go-task
This tool is great for cross-platform shell scripting as it runs all the commands in the Taskfile.yml
using a built in go shell library that supports bash syntax (and others).
Quickly get up and running using the directions here: Install Task
Command | Code |
---|---|
Default Installation to local directory with debug logging enabled | sh -c "$(curl -ssL https://taskfile.dev/install.sh)" -- -d |
Installation for user level access | sh -c "$(curl -ssL https://taskfile.dev/install.sh)" -- -d -b /usr/local/bin |
Installing Brew Packages
This eliminates any attempt to install if the package already exists. For quick adhoc installs, this is useful.
|
|
Reduce Noise With Progress Bar
Use unzip with a progress bar to display progress, rather than the thousands of lines of output. This is an example of installing the AWS CLI v2 in a Dockerfile, while not forcing the output of each line when unzipping.
This shows how to use the pv
command line tool to help display progress in both a count fashion, and also by just using as a timer.
|
|
Check for And Install Tooling
This can help give you an example of how to double check that some installed tools are available as part of a setup script.
|
|
Conditional
Only Proceed If First Condition Returns Nothing
|
|
On error do this:
|
|
On success do the next command:
|
|
Web Requests
Fetch A GitHub Release
This contains a few things, including curl, jq parsing, and movement commands.
This provides a shell script example of using those to get the latest release from GitHub, parse the json, then move this to target path. This release doesn’t wrap in a tar file; it’s just a binary.
This might fail due to anonymous API hits on GitHub api are rate limited aggressively.
|
|
Fetch a GitHub Release That Requires Extraction
This is more of a Linux focused shell script example for grabbing a release and extracting the tar file.
|
|
Concepts
Shebang
A common pattern is just #!/bin/bash
.
To make your script more portable, by respecting the users env preferences try:
#!/usr/bin/env bash
#!/usr/bin/env zsh
#!/usr/bin/env sh
Some good info on this from Shebang
If you do not specify an interpreter line, the default is usually the /bin/sh
For a system boot script, use /bin/sh
The /usr/bin/env
run a program such as a bash in a modified environment. It makes your bash script portable. The advantage of #!/usr/bin/env bash is that it will use whatever bash executable appears first in the running user’s $PATH
variable.
SSH
Setup your permissions for ~/.ssh
|
|
For why 0600 see footnote.1
I’ve had issues with macOS adding an @
with ACL issues on the ssh key’s when downloaded.
To resolve this, just copy the contents of the ssh key to a new file and remove the original.
|
|
AWS CLI & Metadata
Retrieve Instance Region
Looks like the metadata service uses tokens now, so this requires an additional step.
|
|
List Matching Instances
You can use output with --output text
but for this example I used json and jq
.
|
|
List Standard Users
|
|
Search Contents of a File
Using ripgrep
you can search very quickly through file contents.
In this example, I found a text string in a PowerShell file that VSCode wasn’t able to find after 1-2 mins due to the size of the directory.
|
|
I ran a quick test to see how ripgrep performed compared to normal grep search. Grep wasn’t optimized, and by default is single threaded. Ripgrep is multithreaded, automatically honors gitignore and more.
|
|
Tool | Time |
---|---|
ripgrep |
5m6s |
grep |
1h+timeout |
Using yq to edit yaml files for Datadog service
GitHub - mikefarah/yq: yq is a portable command-line YAML processor
I’ve use yq to edit yaml files programatically, such as datadog configuration files.
Here’s a few samples on how to use this tool, using datadog agent config files as an example.
Quick Install of Datadog Service
|
|
Start and stop the datadog services
|
|
Edit Default Datadog Config File
Next, configure the main configuration with custom tags and host name, including additional ec2 tags, metadata, and a custom tag to show the specific load test this is capturing.
|
|
Enable Datadog Network Monitoring
|
|
Enable Datadog Process Level Tracking
Enable process level tracking, with specific matching on ssh, sshd
.
|
|
You can do a lot with yq
.
Parse Kubernetes Secrets Using JQ
Using jq, you can parse out secrets from base64 encoded values for some quick scripting.
NOTE: This uses sttr but you can modify to whatever your platform provides (zsh
base64 -decode
or pwsh[System.Convert]::FromBase64String($Base64String)
)) If you have Go installed (everyone should! 😀) then rungo install github.com/abhimanyu003/sttr@latest
.
This example parses an encoded json string to help registry an Azure Container Registry from a Kubernetes stored secret.
|
|
GitHub CLI
View The Logs Of A Prior Run
View the logs of the last run (or toggle to error logs with the switch).
gh run view --log $(gh run list -L1 --json 'databaseId' --jq '.[].DatabaseId')
gh run view $(gh run list --limit 1 --json databaseId --jq '.[0].databaseId' ) --log
This can be chained together with other commands to quickly iterate on testing. When appropriate, you might avoid this by running act but I’ve had limited success with it due to various restrictions.
|
|
Use To Configure Settings on Many Repos At Once
This example uses [gum]2 to filter.
Use tab
when selecting in the multi-entry option.
|
|
Clone All The Desired
Uses gum[^gum-repo] & [ghq]3. See setup directions .
Configure ghq
To configure ghq
defaults run:
|
|
Clone All Repos Selected
|
|
Webmentions
(No webmentions yet.)