All microblog

2021

Go R1 Day 35

progress

  • Worked with Taskflow a bit more.
  • Need to identify better error handling pattern on when to resolve vs handle internal to a function, as it feels like I’m doing needless error checking.
  • Wrote func to run terraform init, plan, and apply.
  • This takes dynamical inputs for vars and backend file.
  • Also dynamically switches terraform versions by running tfswitch.

Definitely more verbose code than powershell, but it’s a good way to get used to Go while achieving some useful automation tasks I need to do.

Example of some code for checking terraform path.

1
2
3
4
5
6
7
8
9
func terraformPath(tf *taskflow.TF) (terraformPath string, err error) {
	terraformPath = path.Join(toolsDir, "terraform")
	if _, err := os.Stat(terraformPath); os.IsNotExist(err) {
		tf.Errorf("โ— cannot find terraform at: [%s] -> [%v]", terraformPath, err)
		return "", err
	}
	tf.Logf("โœ… found terraform at: [%s]", terraformPath)
	return terraformPath, nil
}
1
2
3
4
5
terraformPath, err := terraformPath(tf)
if err != nil {
  tf.Errorf("โ— unable to proceed due to not finding terraform installed [%v]", err)
  return
}

However, once I call this, I’m see more effort in handling, which feels like I’m double double work at times.

Go R1 Day 34

progress

  • figured out how to import util/logger.go as a package
  • after much confusion due to logger, log, *zerolog.Logger and more variables all deviously similar in name… how to pass around the initialized package logger that I configure.
  • learned that global scope and package scoped loggers being initialized at run is concerned an anti-pattern
  • properly wrapping to avoid the log initialization on import with type Logger struct { logger: *zerolog.Logger; } as an example avoids the same behavior as: var Log *zerolog.Logger
  • will evaluate better scoping in the future, but for now figured it would be a ๐Ÿš€ #shipit moment to improve as I can later. 1

  1. The package level logger anti pattern โ€“ The acme of foolishness ↩︎

Go R1 Day 33

progress

  • successfully created logging package using zerolog
  • learned about scoping with packages
  • linked to a private internal repository and how to leverage the module replace operator to temporarily alter path import from url to local override.
  • middleware is a newer concept, so I need to learn more on this later so I can understand how to use to inject special log handling for http requests and other actions.

Thoughts for today are that the pressure of jumping into an existing codebase is resulting in me moving faster than I probably should. I’m going to take some time to keep doing the web fundamentals, lambda, and exercisms to ensure I’m setting a better foundation long-term, and not just winging it. ๐Ÿ˜„

Go R1 Day 32

progress

  • created some structured logging improvements with zerolog
  • began exploration of middleware concepts for logging
  • generated test stubs using gotests

Go R1 Day 31

progress

  • Learned a bit about idiomatic patterns wtih error handling.
  • Learned about inline block intiailization of variables using if err := method(); err != nil {...} approach.
  • Considered a bit more idiomatic patterns when I noticed excessibe nested if blocks.
1
2
3
4
5
6
tfdir := tf.Params().String("tfdir")
if tfdir != "" {
  tf.Logf("tfdir set to: [%s]", tfdir)
} else {
  tf.Errorf("๐Ÿงช failed to get tfdir parameter: [%v]", tfdir)
}

This would probably be more in alignment with Go standards by writing as:

1
2
3
4
5
6
tfdir := tf.Params().String("tfdir")
if tfdir == "" {
  tf.Errorf("๐Ÿงช failed to get tfdir parameter: [%v]", tfdir)
  return
}
tf.Logf("tfdir set to: [%s]", tfdir)

This reduces the noise and keeps things pretty flat.

When Should I Use One Liner if…else Statements in Go?)

Go R1 Day 30

progress

  • Built some go functions for build tasks work with terraform and setup of projects using taskflow.

Learned one one to pass in arguments using slices. I’m pretty sure you can use some stringbuilder type functionality to get similar behavior, but this worked fine for my use case.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
cmdParams := []string{}
cmdParams = append(cmdParams, "-chdir="+tfdir)
cmdParams = append(cmdParams, "init")
cmdParams = append(cmdParams, "-input=false")
cmdParams = append(cmdParams, "-backend=true")
cmdParams = append(cmdParams, "-backend-config="+tfconfig)
terraformCmd := tf.Cmd(terraformPath, cmdParams...)
if err := terraformCmd.Run(); err != nil {
  tf.Errorf("โญ• terraform init failed: [%v]", err)
  return
}

Go R1 Day 29

progress

  • Evaluated Mage as a replacement for bash/pwsh based tasks for automation with Azure Pipelines.
  • Was able to get terraform to run with dynamic configuration using the following approach:

Install with

1
2
3
4
5
go get -u github.com/magefile/mage/mg
go mod init mage-build
go get github.com/magefile/mage/mg
go get github.com/magefile/mage/sh
go mod tidy

Then to get mage-select run:

1
2
3
GO111MODULE=off go get github.com/iwittkau/mage-select
cd $GOPATH/src/github.com/iwittkau/mage-select
mage install

Configure some constants, which I’d probably do differently later. For now, this is a good rough start.

1
2
3
4
5
6
7
const (
	repo          = "myrepopath"
	name          = "myreponame"
	buildImage    = "mcr.microsoft.com/vscode/devcontainers/base:0-focal"
	terraformDir  = "terraform/stack"
	config_import = "qa.config"
)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
func TerraformInit() error {
	params := []string{"-chdir=" + terraformDir}
	params = append(params, "init")
	params = append(params, "-input=false")
	params = append(params, "-var", "config_import="+config_import+".yml")

	// Backend location configuration only changes during the init phase, so you do not need to provide this to each command thereafter
	// https://github.com/hashicorp/terraform/pull/20428#issuecomment-470674564
	params = append(params, "-backend-config=./"+config_import+".tfvars")
	fmt.Println("starting terraform init")
	err := sh.RunV("terraform", params...)
	if err != nil {
		return err
	}
	return nil
}

Once terraform was initialized, it could be planned.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
func TerraformPlan() error {
	mg.Deps(TerraformInit)
	params := []string{"-chdir=" + terraformDir}
	params = append(params, "plan")
	params = append(params, "-input=false")
	params = append(params, "-var", "config_import="+config_import+".yml")
	fmt.Println("starting terraform plan")
	err := sh.RunV("terraform", params...)
	if err != nil {
		return err
	}
	return nil
}

  • Of interest as well was mage-select, providing a new gui option for easier running by others joining a project.

https://d33wubrfki0l68.cloudfront.net/5376c39d3b86933ae6e4903f91d67da01b139a59/e09d9/images/r1-d029-mages.png

Fix Terraform Provider Path in State

Fixing Terraform provider paths in state might be required after upgrading to 0.13-0.14 if your prior state has the following paths.

First, get the terraform providers from state using: terraform providers

The output should look similar to this:

https://d33wubrfki0l68.cloudfront.net/e61b0d6e20dc87b619af5451369e686c56ca45f3/dca85/images/2021-03-10-microblog-provider-list-01.png

To fix these, try running the commands to fix state. Please adjust to the required providers your state uses, and make sure your tooling has a backup of the state file in case something goes wrong. Terraform Cloud should have this backed up automatically if it’s your backend.

1
2
3
4
terraform state replace-provider -- registry.terraform.io/-/aws registry.terraform.io/hashicorp/aws
terraform state replace-provider -- registry.terraform.io/-/random registry.terraform.io/hashicorp/random
terraform state replace-provider -- registry.terraform.io/-/null registry.terraform.io/hashicorp/null
terraform state replace-provider -- registry.terraform.io/-/azuredevops registry.terraform.io/microsoft/azuredevops

The resulting changes can be seen when running terraform providers and seeing the dash is now gone.

https://d33wubrfki0l68.cloudfront.net/c9803d991ff425002e999dbed1e9e5b07211e988/4998c/images/2021-03-10-microblog-provider-list-02.png

Upgrading to Terraform v0.13 - Terraform by HashiCorp

Loop

If you have multiple workspaces in the same folder, you’ll have to run fix on their seperate state files.

This is an example of a quick adhoc loop with PowerShell to make this a bit quicker, using tfswitch cli tool.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
tf workspace list | ForEach-Object {
    $workspace = $_.Replace('*','').Trim()
    Write-Build Green "Selecting workspace: $workspace"
    tf workspace select $workspace
    tfswitch 0.13.5
    tf 013.upgrade
    tfswitch
    tf init
    # Only use autoapprove once you are confident of these changes
    terraform state replace-provider -auto-approve -- registry.terraform.io/-/aws registry.terraform.io/hashicorp/aws
    terraform state replace-provider -auto-approve -- registry.terraform.io/-/random registry.terraform.io/hashicorp/random
    terraform state replace-provider -auto-approve -- registry.terraform.io/-/null registry.terraform.io/hashicorp/null
    terraform state replace-provider -auto-approve -- registry.terraform.io/-/azuredevops registry.terraform.io/microsoft/azuredevops
    tf validate
}

Go R1 Day 28

progress

  • Solved [Hamming Distance] on exercism.io
  • Simple problem, but reminded me of how to use string split.
1
2
3
4
5
6
7
8
9
diffCount := 0
aString := strings.Split(a, "")
bString := strings.Split(b, "")

for i, x := range aString {
  if x != bString[i] {
    diffCount++
  }
}
  • Reviewed other solutions, and found my first attempt to split the string wasn’t necessary. Looks like I can just iterate on the string directly. I skipped this as it failed the first time. The error is: invalid operation: x != b[i] (mismatched types rune and byte).

This threw me for a loop initially, as I’m familar with .NET char datatype.

Golang doesn’t have a char data type. It uses byte and rune to represent character values. The byte data type represents ASCII characters and the rune data type represents a more broader set of Unicode characters that are encoded in UTF-8 format. Go Data Types

Explictly casting the data types solved the error. This would be flexibly for UTF8 special characters.

1
2
3
4
5
for i, x := range a {
  if rune(x) != rune(b[i]) {
    diffCount++
  }
}

With this simple test case, it’s it’s subjective if I’d need rune instead of just the plain ascii byte, so I finalized my solution with byte(x) instead.

1
2
3
4
5
for i, x := range a {
  if byte(x) != byte(b[i]) {
    diffCount++
  }
}

Incremental and Consistent

It’s really hard to prioritize when life gets busy, but it’s important that continued improvement is a priority.

Great at Work: How Top Performers Do Less, Work Better, and Achieve More was a really interesting book. The fact that small incremental improvement done daily can make such a difference is pretty interesting.

It’s similar to Agile tenets in how to approach software design. Smaller iterations with rapid feedback is better than large isolated batches work delivered without regular feedback.

If you find yourself saying, “But I don’t have time” or “When I have some time” it might be indicative of a failure to grasp this. When I catch myself saying this I try to reword it and say “Whenever I make time for this” instead.

You’ll always have pressure on you. The further along in your career and life you go, the more pressure is likely to be on you.

You have to “make” time for improvement and learning if it’s a priority.

Nativefier

Ran across this app, and thought was kinda cool. I’ve had some issues with Chrome apps showing up correctly in certain macOS windows managers to switch context quickly.

Using this tool, you can generate a standalone electron app bundle to run a webpage in as it’s own dedicated window.

It’s cross-platform.

For a site like Azure DevOps, you can run:

1
2
3
4
5
6

$MYORG = 'foo'
$MYPROJECT = 'bar'
$BOARDNAME = 'bored'
nativefier https://dev.azure.com/$MYORG/$MYPROJECT/_boards/board/t/$BOARDNAME/Backlog%20items/?fullScreen=true ~/$BOARDNAME

If redirects for permissions occur due to external links opening, you might have to open the application bundle and edit the url mapping. GitHub Issue #706

1
/Users/$(whoami)/$BOARDNAME/APP-darwin-x64/$BOARDNAME.app/Contents/Resources/app/nativefier.json

Ensure your external urls match the redirect paths that you need such as below. I included the standard oauth redirect locations that Google, Azure DevOps, and Microsoft uses. Add your own such as github to this to have those links open inside the app and not in a new window that fails to recieve the postback.

1
"internalUrls": "(._?contacts\.google\.com._?|._?dev.azure.com_?|._?microsoft.com_?|._?login.microsoftonline.com_?|._?azure.com_?|._?vssps.visualstudio.com._?)",

Go R1 Day 27

progress

  • Iterated through AWS SDK v1 S3 buckets to process IAM policy permissions.
  • Unmarshaled policy doc into struct using Json-To-Struct.

Github Pages Now Supports Private Pages

I’m a huge static site fan (lookup jamstack).

What I’ve historically had a problem with was hosting. For public pages, it’s great.

For private internal docs, it’s been problematic. It’s more servers and access control to manage if you want something for a specific group inside a company to access.

This new update is a big deal for those that want to provide an internal hugo, jekyll, mkdocs, or other static generate based documentation site for their team.

Access control for GitHub Pages - GitHub Changelog

Ensuring Profile Environment Variables Available to Intellij

Open IntelliJ via terminal: open "/Users/$(whoami)/Applications/JetBrains Toolbox/IntelliJ IDEA Ultimate.app"

This will ensure your .profile, .bashrc, and other profile settings that might be loading some default environment variables are available to your IDE. For macOS, you’d have to set in the environment.plist otherwise to ensure they are available to a normal application.

ref: OSX shell environment variables โ€“ IDEs Support (IntelliJ Platform) | JetBrains

Leverage Renovate for Easy Dependency Updates

Renovate is a great tool to know about. For Go, you can keep modules updated automatically, but still leverage a pull request review process to allow automated checks to run before allowing the update.

This is particularly useful with Terraform dependencies, which I consider notoriously difficult to keep updated. Instead of needing to use ranges for modules, you can start specifying exact versions and this GitHub app will automatically check for updates periodically and submit version bumps.

Why? You can have a Terraform plan previewed and checked for any errors on a new version update with no work. This means your blast radius on updates would be reduced as you are staying up to date and previewing each update as it’s available.

No more 5 months of updates and figuring out what went wrong ๐Ÿ˜

Here’s an example json config that shows how to allow automerging, while respecting minor/major version updates not enabling automerge.

Note that you’d want to install the auto-approver app they document in the marketplace if you have pull request reviews required.

In addition, if you use CODEOWNERS file, this will still block automerge. Consider removing that if you aren’t really leveraging it.

Go R1 Day 26

Progress

  • Evaluated gorm usage best practices with Slack Gopher community.
  • Obtained a great example to get me started on go routine and channels usage with multi-database queries.

Remove Chrome Autocomplete Suggestion

Do you have a Chrome suggestion for a domain, url, or specific site location that you don’t want anymore?

You can remove an invalid suggestion that you don’t want popping up each time by using shift+delete.

If it’s the first entry it will remove it, or use arrow keys to highlight a different entry and press the same command.

Also relevant: To Delete Chrome Address Bar Suggestions with Mouse

Go R1 Day 25

Day 25 of 100

progress

  • Worked with Gorm, my first experience with ORM based querying.
  • Used gofakeit with gorm to connect to MSSQL Server in Docker and randomize name entries.
  • Learned a bit about Context package and how it helps with propagation of cancellation.
  • As a new user to this, the examples were very confusing as it mixed up go-mssqldb along with examples using just gorm. As I get better with it, I’ll try to log better introductory examples.

Go R1 Day 24

Day 24 of 100

progress

  • Used glamour to render some nice console output.
  • Formatted text testing some text tooling.
  • Progressbar ui tested.
  • Connected successfully to MSSQL Docker instance and returned results of query, rendering to to console .