progress

• Built some go functions for build tasks work with terraform and setup of projects using taskflow.

Learned one one to pass in arguments using slices. I’m pretty sure you can use some stringbuilder type functionality to get similar behavior, but this worked fine for my use case.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11

cmdParams := []string{}
cmdParams = append(cmdParams, "-chdir="+tfdir)
cmdParams = append(cmdParams, "init")
cmdParams = append(cmdParams, …

progress

• Evaluated Mage as a replacement for bash/pwsh based tasks for automation with Azure Pipelines.
• Was able to get terraform to run with dynamic configuration using the following approach:

Install with

1
2
3
4
5

go get -u github.com/magefile/mage/mg
go mod init mage-build
go get github.com/magefile/mage/mg
go get github.com/magefile/mage/sh
go mod tidy

Then to get mage-select run:

1
2
3

GO111MODULE=off go get github.com/iwittkau/mage-select
cd $GOPATH …

Fixing Terraform provider paths in state might be required after upgrading to 0.13-0.14 if your prior state has the following paths.

First, get the terraform providers from state using: terraform providers

The output should look similar to this:

  read more

Templates

This isn’t meant to be an exhaustive template overview. I’m just going to give an example of what I found useful as I’ve been meaning to leverage templates for a while and finally got around to having a great use for it.

My Use Case

I am a heavy user of InvokeBuild . It’s a very robust task runner tool that I’ve used to coordinate many project oriented actions, similar to Make, but also DevOps oriented work like AWS SSM tasks and more.

In this scenario, I …

Resources

Git-Town

Painful But Powerful

Let’s get this out of the way.

Git isn’t intuitive.

It has quite a bit of a learning curve.

However, with this flexibility comes great flexibility. This tool has powered so much of modern open-source development.

Optimize for the Pain

To improve the development experience some tools can help provide structure.

This won’t be an attempt to compare every git GUI, or push any specific tooling. It’s more sharing my experience and what …

progress

• Solved [Hamming Distance] on exercism.io
• Simple problem, but reminded me of how to use string split.

1
2
3
4
5
6
7
8
9

diffCount := 0
aString := strings.Split(a, "")
bString := strings.Split(b, "")

for i, x := range aString {
  if x != bString[i] {
    diffCount++
  }
}

• Reviewed other solutions, and found my first attempt to split the string wasn’t necessary. Looks like I can just iterate on the string directly. I skipped this as it failed the first …

Saying that we do knowledge work but have no time for learning is like saying we do farm work but have no time for planting.

— Allen Holub (@allenholub) February 13, 2021

It’s really hard to prioritize when life gets busy, but it’s important that continued improvement is a priority. Great at Work: How Top Performers Do Less, Work Better, and Achieve More was a really interesting book. The fact that small incremental improvement done daily can make such a difference is pretty …

Who This Might Be For

• PowerShellers wanting to know how to create json and yaml dynamically via pscustomobject.
• Anyone wanting to create configs like Datadog or other tools dynamically without the benefit of a configuration management tool.
• Anyone else wanting to fall asleep more quickly. (I can think of better material such as the Go spec docs, but hey, I can’t argue with your good taste 😄)

YAML

It’s readable.

It’s probably cost all of us hours when debugging yaml …

progress

• Iterated through AWS SDK v1 S3 buckets to process IAM policy permissions.
• Unmarshaled policy doc into struct using Json-To-Struct.

links

• JSON-to-Go

I’m a huge static site fan (lookup jamstack).

What I’ve historically had a problem with was hosting. For public pages, it’s great.

For private internal docs, it’s been problematic. It’s more servers and access control to manage if you want something for a specific group inside a company to access.

This new update is a big deal for those that want to provide an internal hugo, jekyll, mkdocs, or other static generate based documentation site for their team.

Access …

Open IntelliJ via terminal: open "/Users/$(whoami)/Applications/JetBrains Toolbox/IntelliJ IDEA Ultimate.app"

This will ensure your .profile, .bashrc, and other profile settings that might be loading some default environment variables are available to your IDE. For macOS, you’d have to set in the environment.plist otherwise to ensure they are available to a normal application.

ref: OSX shell environment variables – IDEs Support (IntelliJ Platform) | JetBrains

First, I’m a big believer in doing infrastructure as code.

Using the AWS SDK with any library is great, but for things like S3 I’d highly recommend you use a Terraform module such as Cloudposse terraform-aws-s3-bucket module . Everything Cloudposse produces has great quality, flexibility with naming conventions, and more.

Now that this disclaimer is out of the way, I’ve run into scenarios where you can have a bucket with a large amount of data such as databases which would be …

Progress

• Evaluated gorm usage best practices with Slack Gopher community.
• Obtained a great example to get me started on go routine and channels usage with multi-database queries.

Do you have a Chrome suggestion for a domain, url, or specific site location that you don’t want anymore?

You can remove an invalid suggestion that you don’t want popping up each time by using shift+delete.

If it’s the first entry it will remove it, or use arrow keys to highlight a different entry and press the same command.

Also relevant: To Delete Chrome Address Bar Suggestions with Mouse

Day 25 of 100

progress

• Worked with Gorm, my first experience with ORM based querying.
• Used gofakeit with gorm to connect to MSSQL Server in Docker and randomize name entries.
• Learned a bit about Context package and how it helps with propagation of cancellation.
• As a new user to this, the examples were very confusing as it mixed up go-mssqldb along with examples using just gorm. As I get better with it, I’ll try to log better introductory examples.

links

• Gorm
• Gofakeit

Day 24 of 100

progress

• Used glamour to render some nice console output.
• Formatted text testing some text tooling.
• Progressbar ui tested.
• Connected successfully to MSSQL Docker instance and returned results of query, rendering to to console .

links

• Glamour
• go-mssqldb

Day 23 of 100

progress

• Used Viper to load configuration for a CLI tool.
• Ran into problems with trying to print out map[string]interface{} type. Lots of magic performed by .NET to do this type of action. This is a lot more hands-on 😁.
• Had stability issues with VSCode today, so I finally switched over to Intellij with the Go plugin and it worked well. The keyboard mappings are different so that was painful, but still overall a good experience that got me unblocked.

links

• Viper
• Tablewriter

Simplicity Over Syntactic Sugar

As I’ve been learning Go, I’ve grown to learn that many decisions to simplify the language have removed many features that provide more succinct expressions in languages such as Python, PowerShell, C#, and others. The non-orthogonal features in the languages result in many expressive ways something can be done, but at a cost, according to Go’s paradigm.

My background is also heavily focused in relational databases and set based work, so I’m …

Day 22 of 100

progress

Using Dash, I read through much of the language specification. Dry reading for sure, but helped a bit in understanding a little more on stuff like arrays, slices, loops, etc.

Nothing profound to add, except to say I don’t think I want to write a language specification.

Day 21 of 100

progress

• Signed up for exercism.io, which is a pretty great website to work through progressively harder exercises.
• Did Hello world to start with as requires progressive steps through the exercises.
• Did a string concatenation exercise as well (Two Fer).

I like the mentor feedback system concept and submission of work. After I finish this, would be good to add myself as a mentor and contribute back to this community. This is a fantastic concept to help get acclimated to a new …

Day 20 of 100

progress

Hackerrank challenge “Bubble Sort”

Day 19 of 100

progress

Hackerrank challenge “Counting Valleys”

Day 18 of 100

progress

Hackerrank challenge “Sales by Match”

Excited. Have a new solution in the works to deploy Ola Hallengren via SSM Automation runbook across all SQL Server instances with full scheduling and synchronization to S3. Hoping to get the ok to publish this soon, as I haven’t seen anything like this built.

Includes:

• Building SSM Automation YAML doc from a PS1 file using AST & metadata
• Download dependencies from s3 automatically
• Credentials pulled automatically via AWS Parameter Store (could be adapted to Secrets Manager as well) …

Source

The source code for this repo is located here:

What This Is

This is a quick overview of some AWS SDK Go work, but not a detailed tutorial. I’d love feedback from more experienced Go devs as well.

Feel free to submit a PR with tweaks or suggestions, or just comment at the bottom (which is a GitHub issue powered comment system anyway).

Image Age

Good metrics can help drive change. If you identify metrics that help you quantify areas of progress in your DevOps process, you’ll …

I asked my daughter (3) how much she loved me. She held up her hands and said: “Five”.

I’ll take that as a win considering that’s all the fingers on that hand. 😂

Free Means You Are the Product

Over time, I’ve begun to look at products that are free with more judgment. The saying is: “If it’s free, you are the product”. This often means your data and privacy are compromised as the product.

This has resulted in me looking more favorably at apps I would have dismissed in the past, such as Leave Me Alone .

Leave Me Alone

The notion of buying credits for something I could script, click, or do myself made me use sporadically last year. …

Day 17 of 100

progress

• reviewed adding excel conversion to cli
• shelved this after reviewing implementation requirements
• this is one of those cases where PowerShell makes much more sense for adhoc work as converts pscustomobject (similar to struct) via pipeline automatically to excel sheet.

Day 16 of 100

progress

• refactored AWS SDK call to export a named file using flags.
• Iterated through regions so cli call aggregated all results from all regions into single JSON.
• Working with v1 makes me want v2 so much more. The level of pointers required is ridiculous. At one point I had something like &*ec2 due to the SDK requirements. Having to write a filter with: Filters: { Name: aws.String("foo")} is so clunky. I believe in v2 this is greatly simplified, and the code is much …

Where Filtering

I explored a bit on filtering objects with Go for the AWS SDK v1.

Coming from PowerShell, I’m used to a lot of one-liners that do a lot of magic behind the scenes, so Go is both refreshing in it’s clarity, and a bit verbose at times since the language features are more sparse.

In PowerShell, you can filter results with a variety of methods, including examples like below (but not limited to)

• $collection.Where{$_.Value -eq 'foo'}
• $collection | Where-Object {$_.Value …

Day 15 of 100

progress

• figured out scope issues with pointer and struct
• Used omitempty in struct
• exported final report in json format after searching for matching image id from ec2 instance image id
• Find it interesting how much more wordy the go search method was, but appreciate it in a way as the “syntactic” sugar that’s missing also is the reason there is more complication at times in languages like PowerShell/C#.

Day 14 of 100

progress

• built golang function with aws-sdk that returned ec2 instances, images
• Joined the data together together to search for matching image from the ec2 metadata
• generated json report from results and final struct

Day 14 of 100

progress

• Migrated my new aws lambda logger from zap to zerolog. Zap gave me some problems initially so zerolog is my favorite structured logger right now, much simpler.
• Constructed go-task runner file for launching go test and go build/run.
• Structured logging required a little bit of refactor but worked.

Here’s an example of providing back a logged string (don’t log secrets normally, but I’m in testing phase) with structure.

1
2
3
4

	log.Debug().
		Str( …

Who Am I Talking To

• You use bash or python.
• PowerShell seems wordy, extra verbose, and annoying.
• It’s a windows thing, you say… why would I even look at it.
• Pry bash out of my fingers if yuo dare (probably not for you 😁)

What PowerShell Is

• The best language for automating Windows… period.
• A great language for development tooling and productivity scripts.
• One of the best languages for automation with interactivity. Python is fantastic. The REPL isn’t meant for the …

Workflow Tooling

Development workflow, especially outside of a full-fledged IDE, is often a disjointed affair. DevOps oriented workflows that often combine cli tools such as terraform, PowerShell, bash, and more all provide more complexity to getting up to speed and productive.

Currently, there is a variety of frameworks to solve this problem. The “gold standard” most are familiar with in the open-source community would be Make .

Considering Cross-Platform Tooling

This is not an …

I couldn’t get past this for a while when I accidentally stumbled across a fix. I believe the fix was merged, however this problem still existed in 0.13.4 so I stuck with it.

Intro

OSS 2.0 is a release candidate at this time, so this may change once it’s released.

It wasn’t quite clear to me how to get up and running quickly with a docker based setup for OSS 2.0 version, so this may save you some time if you are interested. It also should be very similar to the Windows workflow excepting the basic brew commands and service install commands you’ll just want to flip over to choco install telegraf .

Docker Compose

Grabbed this from a comment and …

Day 13 of 100

progress

• Worked with type asserts in my efforts to generate json collection from the parsed front matter.

links

• Handling Type Assertion Failure

My morning. Explaining set and intersect theory basics to my 10 year old with Minecraft gamer tags. Trying to justify the need to know this, the best I could come up with was his future need to build a shark attack report accurately.

Kids are the best. Tech is fun. What job would have me spin up with docker-compose up -d my MSSQL container, write a quick SQL example with INTERSECT, UNION and all to demonstrate this magic.

Followed it up with a half-hearted lie that my day is comprised of cmatrix …

I’ve been enjoying Codespaces local development workflow with Docker containers.

I’m using macOS and on Docker experimental release. Here are some ideas to get started on improving the development experience.

• Clone the repository in the virtual volume (supported by the extension) to eliminate the binding between host and container. This would entail working exclusively inside the container.
• Increased Docker allowed ram to 8GB from the default of 2GB.

Any other ideas? Add a comment …

I took a quick step back when too many parentheses started showing up. If you question the complexity of your quick snippet, you are probably right that there is a much simpler way to do things.

I wanted to get a trimmed message of the results of git status -s. As I worked on this snippet, I realized it was becoming way overcomplicated. 😆

1

$(((git status -s) -join ',') -split '')[0..20] -join ''

I knew my experimentation was going down the wrong road, so I took a …

Day 12 of 100

progress

• Worked on Algolia index project to do atomic updates on search index on my blog.
• Worked with json, structs, ranges, and more.
• Saw success with the first value in my output now correctly parsing out the title from the front matter.
• Implemented zerolog.
• Used front library to parse yaml front matter into map.
• Accessed map to get title into json.

Hoping that eventually I can build out a Go app for sharing that’s the equivalent of “atomic alogia” allowing …

Day 11 of 100

progress

• Pluralsight Go material and some reading.
• Start Learning Go Web Development course by Jon Calhoun.
• Created repo and logged initial progress

Link

• New Repo: web-development-with-go to log my public progress

Beat Deadcells with 3 cells active. Uninstalled. There is no way I’d find any pleasure in life trying to do more. This game is an endless pit of “git gud”.

Now to go do something productive 😄

A quick fix to improve your debugging of remote commands in AWS is to install cw .

With a quick install, you can run a command like: cw tail -f --profile=qa --region=eu-west-1 ssm/custom-automation-docs/my-custom-doc. This will give you a real-time stream of what’s running.

You can also use the AWS Visual Studio Code extension, but I prefer having a terminal open streaming this as I don’t have to go in and refresh any further tools to see what’s happening. I tend to always …

Checkout delta for a much-improved git diff experience. I typically use VSCode or a GUI based editor because I find the diff view pretty messy by default.

This new diff view is a perfect example of a simple CLI tool that improves a development workflow by just fixing something I didn’t know could easily be fixed. 😀

  read more

Day 10 of 100

progress

• Experimented with CLI tool using go-prompt
• Customized initial options
• OS independent call to get user home directory.
• Iterated through a directory listing
• Used path join to initialize path for directory search.
• One challenge in working with structs being returned was figuring out how to print the values of the struct. Initially, I only had pointers to the values coming back. This made sense, though, as I watched a tutorial this weekend on slices, and better understand …

Day 9 of 100

progress

• Watched some Pluralsight and other material on testing with Go

Day 8 of 100

progress

• Worked through Algorithms in Go: Determine if a number is in a list
• Passed the tests without needing to find the solution, so that’s a win.
• Had to remember to grab the second value from the range operator like for _, i := range items instead of using foreach($i in $items) like PowerShell would do. It’s more similar to using hashtables with GetEnumerator().
• Used codespaces with Docker and also experimented with WSL2 and Visual Studio Code mounting the directory …

Migrated a forked copy of a module over to a new module with similar schema. There were some additional properties that were removed. In rerunning the plan I was expecting to see some issues with resources being broken down and rebuilt. Instead, Terraform elegantly handled the module change.

I imagine this has to do with the resource name mapping being the same, but regardless it’s another great example of how agile Terraform can be.

Disclaimer

Newbie Gopher. Much of what I observe is likely to be half right. I’ll probably look back at the end of the year and shake my head, but gotta start the journey somewhere, right? 😁

My Background

I’ve learned my development skills primarily in the dotnet world. Coming from SQL Server performance, schema, and development, I transitioned into learning PowerShell and some C#. I found for the most part the “DevOps” nature of what I was doing wasn’t a good fit …

Day 7 of 100

progress

• Successfully deployed lambda to AWS
• Figured out how to return value cleanly from AWS Lambda vs the log output.

Day 6 of 100

progress

• Learned a little on bubble sort algorithm
• Edited aws-go lambda tests
• Used debugger a bit more to look at error return values on failure

I use Dash for improved doc access. Terraform updated recently to 0.13.x and I began having odd issues with AWS provider results coming through. If you need to rollback, just go to the preferences and pick an older docset, in my case 0.13.0 worked correctly. Make sure to remove the problematic version (the uninstall refers to just the most recent, not any additional versions you selected under the dropdown)

If the index doesn’t rebuild, you can close the app, manually remove the index, …

What a polished setup experience from gitpod. Super impressed as I’ve never seen an interactive terraform setup asking for user input.

This basically generated an entire setup for GitPod, and persisted the Terraform plan for the entire stack it created in a directory for management and adjustments as desired.

I’m seriously impressed.

Check this out at: Install on AWS

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# Setup Gitpod with self-setup docker installer
# …

Day 5 of 100

progress

• I created my first unit test for go
• It’s a bit interesting coming from a background with PowerShell and Pester as my primary unit test framework. For instance, in Pester you’d declare the anything, but autodiscovery works with *.tests.ps1, being the normal convention.
• There is no pointer value providing the test package, it’s just other PowerShell calling PowerShell.
• I’m biased I know, but the first test condition being like below seems clunky. I …

Day 4 of 100

progress

• In the last week some more Pluralsight Go (Nigel’s courses are always stellar)
• I’ve done some odd work as had time in between last update, but nothing structured. Might continue with learn-go-with-tests but also came across exercism and might tackle that soon as well.
• Setup new project in Visual Studio Codespaces
• Used Serverless Framework to generate aws-go template project
• Imported aws-go-sdk resources for AWS SSM Secrets Manager
• Grabbed SSM Secrets manager …

Day 3 of 100

progress

• Learned about GOROOT, GOPATH and how to configure
• Ran into problems with Visual Studio code reporting:

1

Failed to find the "go" binary in either GOROOT() or PATH(/usr/bin:/bin:/usr/sbin:/sbin. Check PATH, or Install Go and reload the window.

• After attempting solution with various profile files, I tried setting the setting: "go.goroot": "/usr/local/opt/go/libexec/", in settings.json and this resolved the issue.
• After it recognized this, …

• Using cmd+= results in zooming to 120%,145%,170%
• Using cmd+- results in zooming out to 85%, 70%, 60%

What I want to do

1
2
3
4
5
6
7
8

# create file local.users.yml
user:
  - name: foobar1
    email: [email protected]
  - name: foobar2
    email: [email protected]
  - name: foobar3
    email: [email protected]

1
2
3
4
5

locals {
  users_file         = "local.users.yml"
  users_file_content = fileexists(local.users_file) ? file(local.users_file) : "NoSettingsFileFound: true"
  users_config       = yamldecode(local.users_file_content)
}

What I want to work:

1
2
3 …