powershell
Requirements
- Going forward most examples will focus on Powershell (>= v7), the modern successor to Windows PowerShell & PowerShell Core. Install this easily through chocolatey. Just install chocolatey, and then run
choco upgrade powershell powershell-core -y --source chocolatey
and you should have both 5.1 and core ready to go if your Windows version supports it. If you are on Windows 7 as a developer, there is no 🌮 for you, just get upgraded already. Linux has it’s own set of directions. - Anything manipulating system might need admin, so run as admin in prompt.
Install-Module PSFramework
// I use this module for better logging and overall improvements in my quality of life. It’s high quality, used by big projects like DbaTools and developed by a Powershell MVP with lots of testing. Forget regularWrite-Verbose
commands and just use theWrite-PSFMessage -Level Verbose -Message 'TacoBear'
instead.
PSFramework
I use PSFramework on all my instances, as it’s fantastic expansion to some core areas with PowerShell. This great module (along with other ancillary supporting ones like PSModuleDevelopment
are well tested and usable for solving some issues in a much more elegant and robust manner than what is natively offered.
A few key elements it can help with are:
- Improving Configuration and variable handling without complex scope issues
- Improving overall logging and troubleshooting
- Improving object manipulation
- Runspace usability enhancements
- Scripted properties
Development (Optional)
- Install VSCode (Users)
choco upgrade vscode-powershell -y
or install in extension panel in VSCode. If you are using ISE primarily…. move on already.
String Formatting
Type | Example | Output | Notes |
---|---|---|---|
Formatting Switch | ‘select {0} from sys.tables’ -f ‘name’ | select name from sys.tables | Same concept as .NET [string]::Format(). Token based replacement |
.NET String Format | [string]::Format(‘select {0} from sys.tables’,‘name’) | select name from sys.tables | Why would you do this? Because you want to showoff your .NET chops? |
Math & Number Conversions
From | To | Example | Output | Notes |
---|---|---|---|---|
scientific notation | Decimal | 2.19095E+08 / 1MB | 208.945274353027 MB | Native PowerShell, supports 1MB, 1KB, 1GB |
Date & Time Conversion
Converting dates to Unix Epoc time can be challenging without using the correct .NET classes. There is some built in functionality for converting dates such as (Get-Date).ToUniversalTime() -UFormat '%s'
but this can have problems with time zone offsets. A more consistent approach would be to leverage the following. This was very helpful to me in working with Grafana and InfluxDb which commonly leverage Unix Epoc time format with seconds or milliseconds precision.
|
|
Credential Management
Setup for BetterCredentials
First, don’t store anything as plain text in your files. That’s a no no. Secondly, try using BetterCredentials. This allows you to use Windows Credential Manager to store your credentials and then easily pull them back in for usage later in other scripts you run. I’ve found it a great way to manage my local credentials to simplify my script running.
|
|
Personally, I use BetterCredential\Get-Credential
which is module\function
syntax if I’m not certain I’ve imported first. The reason is auto-discovery of module functions in PowerShell might use the default Get-Credentials
that BetterCredentials overloads if you don’t import first. BetterCredentials overrides the default cmdlets to improve for using CredentialManager, so make sure you import it, not assume it will be correctly imported by just referring to the function you are calling.
Creating a Credential
Then to create credentials try using this handy little filter/function
|
|
Example on using this quick function (if you don’t want to use my quick helper, then just use code in the function as an example).
|
|
Clearing all credentials in credential repo
|
|
Using Credentials in a Script
Using with credential object
|
|
Extracting out the raw name and password for things that won’t take the credential object, but want the password as a string (uggh)
|
|
Using Basic Authorization With REST
When leveraging some api methods you need to encode the header with basic authentication to allow authenticated requests. This is how you can do that in a script without having to embed the credentials directly, leveraging BetterCredentials
as well.
|
|
Load Functions from a Folder
Prefer to use modules, but for quick adhoc work, you can organize your work in a folder and use a subfolder called functions. I find this better than trying to create one large script file with multiple functions in it.
|
|
Select Object Manipulation
Expanding Nested Objects
One thing that I’ve had challenges with is expanding nested objects with AWSPowershell, as a lot of the types aren’t formatted for easy usage without expansion.
For example, when expanding the basic results of Get-EC2Instance
you can try to parse out the state, but it won’t behave as expected.
For example, if you run:
|
|
InstanceId | State |
---|---|
i-taco1 | Amazon.EC2.Model.InstanceState |
i-taco2 | Amazon.EC2.Model.InstanceState |
i-taco3 | Amazon.EC2.Model.InstanceState |
Trying to expand gets you the state, but now you don’t have the original object alongside it.
Code | Name |
---|---|
16 | running |
16 | running |
16 | running |
PSFramework makes this easy to work with by simply referencing the object properties for parsing in the Select-PSFObject
statement.
|
|
The result is exactly you’d you need to work with
InstanceId | StateCode | StateName |
---|---|---|
i-taco1 | 16 | running |
i-taco2 | 16 | running |
i-taco3 | 16 | running |
Get / Set Accessors in PowerShell
In C# you use get/set accessors to have more control over your properties. In PowerShell, thanks to PSFramework, you can simplify object pipelines by using Select-PSFObject
to do the equivalent and have a scripted property that handles a script block to provide a scripted property on your object.
For example, in leveraging AWS Profiles, I wanted to get a region name mapped to a specific profile as a default region. You can do this in a couple steps using ForEach-Object
and leverage [pscustomobject]
, or you can simplify it greatly by running Select-PSFObject
like this:
|
|
Another good example might be the desire to parse out the final key section from S3, to determine what the file name would actually be for easier filtering or searching. In this case, a simple script property could parse out the name, and then return the last item in the array using Powershell’s shortcut of $array[-1]
to get the last item.
|
|
Parallel Tips & Tricks
If you are using -Parallel
with the newer runspaces feature in PowerShell 7 or greater, then long running operations such as queries or operations that take a while might be difficult to track progress on.
In my case, I wanted to be able to see the progress for build process running in parallel and found using the synchronized hashtable I was able to do this.
|
|
I put the delay in there to show that the asynchronous nature doesn’t mean 1-100, it could do some faster than others and this shows on the output with content like:
|
|
A more advanced way to use this might be to help guage how long something might take to complete when running parallel SQL Server queries.
|
|
The Various Iteration Methods Possible
PowerShell supports a wide range of iteration options. Not all are idiomatic to the language, but can be useful to know about.
I recommend when possible to default to $Items | ForEach-Object { }
as your default approach.
This ensures a pipeline driven solution that can be enhanced later or piped to other cmdlets that are compatible with the pipeline.
foreach
loop.These are ranked in the order I recommend using by default.
|
|
- The default go to for major loop work.
- Default first positional argument is
-Process {}
but mostly that is not provided and just the curly braces. - It is by default the slowest on a scale of raw performance.
- Each item is loaded into memory, and it frees memory as it goes through pipeline.
- Pipelines can be chained passing input as the pipeline progresses.
- Break, continue, return behave differently as you are using a function, not a language operator.
|
|
- Magic operator.
- Seriously, I’ve seen it called that.
- It’s only in version >= 4 Magic Operators.
- Loads all results into memory before running, so can be great performance boost for certain scenarios that a
ForEach-Object
would be slower at.
|
|
- This is the standard
foreach
loop. - It is the easiest to use and understand for someone new to PowerShell, but highly recommend that it is used in exceptions and try to stick with
ForEach-Object
as your default for idiomatic PowerShell if you are learning. - Standard break, continue, return behavior is a bit easier to understand.
|
|
- If you find yourself exploring delegate functions in PowerShell, you should probably just use C# or find a different language as you are probably trying to screw a nail with a hammer. 😁
|
|
Cool Tricks
Output the results of your code into a Console GUI Gridview. This recent module provides a fantastic solution to allowing filtering and selection of results passed into it.
Install it with: Install-Module Microsoft.PowerShell.ConsoleGuiTools -Scope CurrentUser -Confirm:$false
|
|
For quick access, save this to a Visual Studio Code snippet like below:
|
|
Puzzles - Fizz Buzz
I did this to participate in Code Golf, and felt pretty good that I landed in 112 🤣 with this. Really pains me to write in the code-golf style.
Syncing Files Using S5Cmd
Other Cheatsheets
Trying a new gist driven approach here, so this will auto update.
AWS Tools
Install AWS.Tools
Going forward, use AWS.Tools modules for newer development. It’s much faster to import and definitely a better development experience in alignment with .NET SDK namespace approach.
Use their installer module to simplify versioning and avoid conflicts with automatic cleanup of prior SDK versions.
|
|
Using Systems Manager Parameters (SSM) To Create A PSCredential
|
|
Using AWS Secrets Manager To Create a PSCredential
Note that this can vary in how you read it based on the format.
The normal format for entries like databases seems to be: {"username":"password"}
or similar.
|
|
Generate a Temporary Key
Useful for needing to generate some time sensitive access credentials when connected via SSM Session and needing to access another account’s resources.
|
|
Install SSM Agent Manually
This is based on the AWS install commands, but with a few enhancements to better work on older Windows servers.
|
|
AWS PowerShell Specific Cheatsheets
Pester
Many changes occurred after version 5. This provides a few examples on how to leverage Pester for data driven tests with this new format.
BeforeAll And BeforeDiscovery
One big change was the two scopes. Read the Pester docs for more details.
The basic gist is that BeforeAll is in the “run” scope, while the test generation is BeforeDiscovery.
While older versions of Pester would allow a lot more foreach
type loops, this should be in the discovery phase now, and then -Foreach
(aka -TestCases
) hashtable can be used to iterate more easily now through result sets.
Pester Container To Help Setup Data Driven Tests
Example of setting up inputs for the test script from your InvokeBuild job.
|
|
Pester Configuration Object
Now, you’d add this PesterContainer
object to the PesterConfiguration
.
PesterConfiguration]::Default
and then explore sub-properties with actions like: [PesterConfiguration]::Default.Run | Get-Member
.
|
|
This pester configuration is a big shift from the parameterized arguments provided in version < 5.
Invoke Pester
Run this with: Invoke-Pester -Configuration $Configuration
To improve the output, I took a page from PSFramework
and used the summary counts here, which could be linked to a chatops message.
Otherwise the diagnostic output should be fine.
|
|
Use Test Artifact
Use the artifact generated in the Azure Pipelines yaml to publish pipeline test results.
|
|
Azure Pipelines Tips
Create a link to a pipeline for your chatops.
|
|