shell

2021-07-14 1182 words 6 minutes

This is a mix of shell, linux, and macOS commands. Comments are welcome with any corrections or suggestions.

CLI Usage

PowerShell & Bash Comparison

Brevity

See all aliases with Get-Alias and to expedite your cli usage you could use a gist like this: Aliaser.ps1

Note that PowerShell eschews brevity for clarity, but you can alias anything you like to be nearly as succint as bash commands.

IMO readability/brevity trumps succintness. However for interactive terminal usage aliasing can be a great tool. Use VSCode to auto-expand aliases into fully qualified functions if you decide to turn your adhoc work into a script file.

Using pushd in a PowerShell session actually aliases to Push-Location. The difference is in addition to the path string pushd manages, you get a .NET object back with [System.Management.Automation.PathInfo] information including: Drive, Path, Provider, ProviderPath.

PowerShell Works With Native Tooling

I’ve included the similar PowerShell command to help those jumping between multiple shells.

Please note that unlike Python, PowerShell works as a terminal with native tools + scripting language.

You can use pwsh in almost every case in Linux & macOS and use the same tools you prefer, while being able to execute PowerShell commands as well.

For example something like aws cli returning json could be automatically unmarshaled into an object instead of using jq

1
& (aws ec2 describe-instances | ConvertFrom-Json).Instances.InstanceId

Another example is paths.

Prerequiresites for the PowerShell examples:

1
Install-Module Microsoft.PowerShell.ConsoleGuiTools -Scope CurrentUser -Force

Command	shell	pwsh
View history	`history`	`Get-History`
Execute Line from History	`!Number`	`Invoke-Expression (Get-History \| Out-ConsoleGridView -OutputMode Single).CommandLine`
Execute Last Command But With Sudo	`sudo !!`
Test file exists	`test -f ./filename`	`Test-Path $filename -PathType Leaf` or using .NET `[io.file]::exists($filename)`

Installation

Common App Installs

Application	Notes	Install Command
HomeBrew	Works on Linux and macOS now 👏.	`/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"`

Ansible Initialization

1
2
3
#!/usr/bin/env bash
sh -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
brew install python3 ansible

For docker/linux

1
2
3
4
sudo python3 -m pip install --upgrade pip
CFLAGS=-Qunused-arguments CPPFLAGS=-Qunused-arguments python3 -m pip install --user ansible
echo "======== Ansible Version Info ======== "
ansible-galaxy --version

A Few More Ansible Commands

Command	Code
Run ansible playbook against a specific tag	`ansible-playbook main.yml --inventory inventory --ask-become-pass -t 'ui'`
Install requirements	`ansible-galaxy collection install community.general && ansible-galaxy install --role-file requirements.yml --force --ignore-errors`

Installing go-task

This tool is great for cross-platform shell scripting as it runs all the commands in the Taskfile.yml using a built in go shell library that supports bash syntax (and others).

Quickly get up and running using the directions here: Install Task

Command	Code
Default Installation to local directory with debug logging enabled	`sh -c "$(curl -ssL https://taskfile.dev/install.sh)" -- -d`
Installation for user level access	`sh -c "$(curl -ssL https://taskfile.dev/install.sh)" -- -d -b /usr/local/bin`

Installing Brew Packages

This eliminates any attempt to install if the package already exists. For quick adhoc installs, this is useful.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
#!/usr/bin/env bash

# Minimize Homebrew updates for each run, speeding things up
export HOMEBREW_NO_AUTO_UPDATE=1

# if linux install script, might want to include this: export PATH="/home/linuxbrew/.linuxbrew/bin:$PATH"

# Example of installing with a tap
brew tap lucagrulla/tap
package=cw
brew list $package &>/dev/null || brew install $package

# git-delta needs an updated version, so make sure it's available
package=less
brew list $package &>/dev/null || brew install $package

package=git-delta
brew list $package &>/dev/null || brew install $package

Conditional

Only Proceed If First Condition Returns Nothing

1
2
package=cw
brew list $package &>/dev/null || brew install $package

On error do this:

1
test -f nonexistentfile || echo "😢 boo. file does not exist"

On success do the next command:

1
test -f ~/.bashrc && echo "✅ congrats, you have a bashrc file"

Web Requests

Fetch A GitHub Release

This contains a few things, including curl, jq parsing, and movement commands.

This provides a shell script example of using those to get the latest release from GitHub, parse the json, then move this to target path. This release doesn’t wrap in a tar file; it’s just a binary.

This might fail due to anonymous API hits on GitHub api are rate limited aggressively.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
#!/usr/bin/env bash

echo "Grabbing latest release of fetch (a github cli for release downloads)"
USER=gruntwork-io
REPO=fetch
TAG=latest
ASSET=fetch_linux_amd64
FILE=fetch
curl --silent "https://api.github.com/repos/$USER/$REPO/releases/latest" \
| jq -r ".assets[] | select(.name | test(\"${ASSET}\")) | .browser_download_url" \
| wget -qi - --output-document=$FILE --progress=bar:force

echo "setting as executable and moving to /usr/local/bin"
chmod +x $FILE
sudo mv fetch /usr/local/bin
echo "Downloaded $(fetch --version) successfully"

Fetch a GitHub Release That Requires Extraction

This is more of a Linux focused shell script example for grabbing a release and extracting the tar file.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
#!/usr/bin/env bash
sudo apt -qqy update
sudo apt -qqy -o Dpkg::Progress-Fancy=true install wget

curl -s https://api.github.com/repos/GitTools/GitVersion/releases/latest \
| grep "browser_download_url.*gitversion\-debian.*\-x64.*\.tar\.gz" \
| cut -d ":" -f 2,3 \
| tr -d \" \
| wget -qi -

tarball="$(find . -name "gitversion-debian*.tar.gz")"
tar -xzf $tarball

sudo chmod +x gitversion
sudo mv gitversion /usr/local/bin

sudo rm $tarball
echo ">>>> gitversion version: $(~/gitversion /version)"
echo "Trying to install dotnet tools version"
dotnet tool update --global GitVersion.Tool

Concepts

Shebang

A common pattern is just #!/bin/bash.

To make your script more portable, by respecting the users env preferences try:

#!/usr/bin/env bash
#!/usr/bin/env zsh
#!/usr/bin/env sh

bash.cyberciti.biz reference

Some good info on this from Shebang

If you do not specify an interpreter line, the default is usually the /bin/sh

For a system boot script, use /bin/sh

The /usr/bin/env run a program such as a bash in a modified environment. It makes your bash script portable. The advantage of #!/usr/bin/env bash is that it will use whatever bash executable appears first in the running user’s $PATH variable.

SSH

Setup your permissions for ~/.ssh

1
2
3
4
5
echo "Setting full user permissions for ~/.ssh"
chmod -R u+rwX ~/.ssh
echo "Remove group access for ~/.ssh"
chmod go-rwx ~/.ssh
echo "now set any pem files to chmd 400 \$key to ensure read-only"

Troubleshooting macOS permissions

I’ve had issues with macOS adding an @ with ACL issues on the ssh key’s when downloaded.

To resolve this, just copy the contents of the ssh key to a new file and remove the original.

1
cat original_key.pem > key.pem

AWS CLI & Metadata

Retrieve Instance Region

Looks like the metadata service uses tokens now, so this requires an additional step.

1
2
3
TOKEN=`curl --silent --show-error --fail -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
RESPONSE=`curl --silent  --show-error --fail -H "X-aws-ec2-metadata-token: $TOKEN"  http://169.254.169.254/latest/meta-data/placement/region`
echo "Current Region is: [$RESPONSE]"

List Matching Instances

You can use output with --output text but for this example I used json and jq.

1
2
aws ec2 describe-instances --filters "Name=tag:Name,Values={{ .EC2_NAME_FILTER }}" --output json \
--query 'Reservations[*].Instances[*].{Instance:InstanceId}' | jq --compact-output '.[][].Instance'

List Standard Users

1
getent passwd {1000..60000}

How To List Users In Linux

Webmentions

(No webmentions yet.)