All posts

2021

Contents

Setup Sourcegraph Locally

I went through the Sourcegraph directions, but had a few challenges due to the majority of code being behind SSH access with Azure DevOps.

Finally figured out how to do this, with multiple repos in one command and no need to embed a token using https.

Navigate to: manage-repos and use this.1 Better yet, use Loading configuration via the file system (declarative config) - Sourcegraph docs and persist locally in case you want to upgrade or rebuild the container.

{
  "url": …
  read more

Docker Healthchecks for Spinning Up Local Stacks

I’ve used a few approaches in the past with “wait-for-it” style containers.

Realized there’s some great features with healthchecks in Docker compose so I decided to try it out and it worked perfectly for Docker compose setup.

This can be a great way to add some container health checks in Docker Compose files, or directly in the Dockerfile itself.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42 …
  read more

Go R1 Day 85

progress

πŸŽ‰Finished Ultimate Syntax course.

Worked on Enumerators concept using iota.

I still find this very confusing in general.

Here’s the gist I created.

Go R1 Day 84

progress

Ultimate Syntax (Ardan Labs - Bill Kennedy) and went back through various topics such as:

  • Pointers: One thing mentioned that resonated with me was the confusion regarding pointers in parameter declarations. I also find the usage strange that the deference operator is used to denote a pointer value being dereferences in the parameter. I’d expect a pointer value to pass clearly with func (mypointer &int) and not func (mypointer int) with a pointer call.
  • Literal Structs: Great …
  read more

Thermal Throttling Mac Intel Woes

I lost roughly half a day in productivity. CPU hit 100% for an extended period, making it difficult to even use the terminal.

Initially, I thought the culprit was Docker, as it was running some activity with local codespaces and linting tools. Killing Docker did nothing.

Htop pointed out kernel as the primary hog, and you can’t kill that.

After digging around online, I found further mentions about charging on the right side, not the left to avoid thermal issues causing CPU throttling.

The …

  read more

Go R1 Day 83

progress

Revisited Ultimate Syntax (Ardan Labs - Bill Kennedy) and went back through various topics such as:

  • Variables: When to use var vs walrus operator for readability and zero value initialization.
  • Type Conversions: How identical named types aren’t the same in Go’s eyes when named.
  • Pointers: General overview. Mostly stuff I knew, but good basic overview again.

Keyboard Remapping in Goland

This is a continuation of my evaluation using Goland.

I tend to be a VSCode user primarily, but am working on adopting Goland incrementally to leverage some of it’s features that work a bit more consistently (like refactoring).

Losing keyboard shortcuts is painful.

VSCode lets you assign keyboard shortcuts in command pallette by clicking on the gear icon in the command you hover over. For a while I couldn’t find this in Goland. I see now I can assign any action defined from the quick …

  read more

Go R1 Day 80

progress

Built a Mage task to handle creation of Azure DevOps pull requests. Since the tooling out there is primarily around Github, this was useful to help standardize PR creation with autocomplete, conventional commit naming, and other properties that typically require manual changes. I found a great little TUI components library that simplified using Bubbletea components: Promptkit.

In addition, noticed some new linting help from golangci-lint for varnamelen.

This was useful as it analyzes …

  read more

Go R1 Day 79

progress

Felt like revisiting Exercism since the great updates and wanting a break from the more deep concept dives on mechnical sympathy and optimization in the Ultimate Go course (which is stellar).

Completed the following to continue on syllabus. It’s taking me back to basics, but stuff I can knock out pretty quick.

  read more

Go R1 Day 78

progress

Ultimate Go: 2.3.4 - Pointers-Part 4 (Stack Growth)

Scenario:

  • 2k stack for each goroutine.
  • 50,000 goroutines.
  • Eventually, you’ll want to make the function call and you’ll want to grow the stack if the current stack limit is hit.
  • We’ll want to use contigous stacks.
  • The new stack will be a new contigous allocated block of memory.
  • The stack growth requires all the prior values to be moved over to the new doubled stack.

A goroutine can only share values from the heap. …

  read more

Go R1 Day 77

progress

More Fun With Golangci-lint

Been doing a ton the last month with golangci-lint tooling. I think this has been one of the most educational tools for learning Go I’ve come across. It has forced me to evaluate why issues are flagged, if they are opinionated preferences or best practices.

For example, wsl ensures that statements are not cuddled. This follows Dave Cheney’s writing about having an empty line seperate phrases of thought.

It’s a bit annoying to implement as a …

  read more

Go R1 Day 76

progress

Lots of more technical detail on Memory & Data Semantics Section.

I value this as my background in databases means that things like Stack/Heap weren’t part of my education as much. This has been useful as there are new concepts, paradigms, and ways of thinking and it’s providing a lot more foundational concepts for me to better understanding the system and compiler behavior in Go.

Ultimate Go: Pointers-Part-3: Escape Analysis

Notes

  • Stacks are self-cleaning
  • Compiler …
  read more

Go R1 Day 74

progress

  • Worked with DynamoDB schema (NoSQL Document database).
  • Invoked local lambda function using Docker and also remove invocation with serverless invoke. This took a bit of work to figure out as the parameters weren’t quite clear. Upon using --path for the json template I got from AWS Lambda console, I was able to to get to invoke, and stream the logs with --log.
  • More mage magic with promptui and other features, so I can now test a full tear down, build, publish, and invoke a test …
  read more

Go R1 Day 73

progress

  • I’m more on day 90+ but haven’t made the time to log it.

  • Been working a ton with Mage and have some really cool work I’ll be writing up soon, including using for health checks on projects, setup of core tooling, encoding credentials, and more. πŸ’―

  • Also been working heavily on Go mono-repo module structure and some serverless based architecture tests that will make a fun write-up. I’m thinking of a demo of “Disappointment as a Service” with a …

  read more

Go R1 Day 72

progress

  • Learned more on “mechanical sympathy” related to Go and memory management.
  • How the Go compiler works related to frames with memory allocation.
  • Built Mage tasks for licensing and validating problematic licenses aren’t included in a project.
  read more

Go R1 Day 71

progress

  • Learn Go With Tests -> Using select with channels to wait for multiple goroutines.
  • Of particular interest is this:

Notice how we have to use make when creating a channel; rather than say var ch chan struct{}. When you use var the variable will be initialised with the “zero” value of the type. So for string it is “”, int it is 0, etc. For channels the zero value is nil and if you try and send to it with <- it will block forever because you cannot send to …

  read more

Go R1 Day 70

progress

  • Concurrency section wrap-up with Learn Go With Tests.
  • Reviewed material learned from: Go R1 Day 61
  • Read material, but didn’t do a lot of tests on this, per mostly concept oriented. Used concurrent progressbar example from uiprogress project to test concurrent UI updates.
  • My last concurrency test case was to launch many concurrent processes for a load test. This didn’t leverage goroutinues as typically used, since it was calling to an executable on the host machine. However, …
  read more

Go R1 Day 68

progress

  • Did exercism.io for gigasecond puzzle.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
package gigasecond

// import path for the time package from the standard library
import (
	"time"
)

// gigasecond represents a very very very small portion of a second.
const gigasecond = 1000000000

// AddGigasecond adds a very very very small portion of a second called a gigasecond to a provided time input.
func AddGigasecond(t time.Time) time.Time {
	gcDuration := gigasecond * …
  read more

Go R1 Day 67

progress

Built functionality in my blog repo to create a new 100DaysOfCode post using Mage. This provides an interactive prompt that automatically tracks the days left and increments the counter as this progresses.

  • ingest toml configuration
  • unmarshal to struct
  • update struct
  • marshal and write back to the toml configuration file
  • replace matched tokens in file

Go R1 Day 66

progress

This wasn’t specific to Go, but was the first step towards using Go in a distributed test.

Dapr

I had an interesting project today with my first development level effort using Kubernetes. Here’s my log of attempting to use Getting started with Dapr | Dapr Docs and getting two Go APIs to talk to each other with it.

First, what is Dapr?

Dapr is a portable, event-driven runtime that makes it easy for any developer to build resilient, stateless and stateful applications that …

  read more

Go R1 Day 65

progress

  • Built mage tasks for go formatting and linting.

Using this approach, you can now drop a magefile.go file into a project and set the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
// +build mage

package main

import (

	"github.com/magefile/mage/mg"
	"github.com/pterm/pterm"

	// mage:import
	"github.com/sheldonhull/magetools/gotools"
)

Calling this can be done directly now as part of a startup task.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
// Init runs …
  read more

First Pass With Pulumi

Why

Instead of learning a new domain specific language that wraps up cloud provider API’s, this let’s the developer use their preferred programming language, while solving several problems that using the API’s directly don’t solve.

  • Ensure the deployment captures a state file of the changes made.
  • Workflow around the previews and deployments.
  • Easily automated policy checks and tests.

This can be a really useful tool to bring infrastructure code maintainability directly …

  read more

SweetOps Slack Archive

Just wanted to give props to the Cloudposse team lead by Erik Osterman @eosterman.

Slack provides a great community chat experience, but there are quite a few problems about using it for Q&A. 1 Since the free plans for communities hide content over 10,000 messages, a healthy community will go over this quickly.

With all the great conversations, I want to find prior discussions to benefit from topics already covered.

Cloudposse archives the community discussions so they can be searched. …

  read more

Go R1 Day 62

progress

  • Worked with mocks.

Still using the is package to test.

1
2
3
if !reflect.DeepEqual(want, spySleepPrinter.Calls) {
  t.Errorf("wanted calls %v got %v", want, spySleepPrinter.Calls)
}

was replaced with:

1
is.Equal(spySleepPrinter.Calls, want) // spySleepPrinter shows correct order of calls

Go is messing with my head with how everything gets simplified to the lowest common interface when possible. Instead of buffer, I’d want to use io.Writer for example. This …

  read more

Go R1 Day 61

progress

It was a dark and stormy night. The world was bleak. A command was sent to The Compiler.

The assumptions were flawed. The command was rejected.

You have given me an invalid command with -argument-that-shall-not-work 5 being invalid.

But I’ve provided --argument-that-shall-not-work, the indomitable wizard said.

Your command is unworthy.

Digging into esoteric tomes of knowledge, the wizard discovered others have had similar issues when calling external processes using the …

  read more

Go R1 Day 60

progress

  • Finished up the basics of dependency injection and how this helps with testable code.
  • Worked through concurrency test using channels and go routines. This will take a bit more to get comfortable with as there is a variety of concepts here. My initial attempts finally started working using an anonymous function, but couldn’t finalize due to some issue with launching the parallel executable being called. Not sure why the --argname seemed to be trimming the first dash from the …
  read more

Go R1 Day 59

progress

  • Built some Mage tasks for setup of local projects.
  • Used retool post from Nate Finch with Mage project and found it promising to vendor Go based tooling into a local project and not rely on the global package library.
  • Created magetools repo to house some basic general mage tasks I could share with other projects.
1
2
year, month, day := time.Now().Date()
dateString := fmt.Sprintf("%d-%02d-%d", year, month, day)

Use padding in the Sprintf call to ensure a date comes back …

  read more

Go R1 Day 57

progress

  • Did some adhoc work in this repo (the hugo repo that contains this blog) testing out Mage, which is a Go based Make alternative.
  • Generated dynamic target directory for hugo posts using stringify for kebab case.
  • Unexpected behavior when generating dynamic file path including date.
1
2
3
year, month, day := time.Now().Date()
str := stringy.New(title)
slugTitle := strings.Join([]string{string(year), string(month), string(day), str.KebabCase("?", "").ToLower()}, …
  read more

Go R1 Day 56

progress

  • Worked on ci-cd pipelines for Go apps.
  • Learned a bit more on building Azure Pipeline templates to use with Go, learning a bit from SignalFx and how they did it.
  • Read more on semver versioning, and still can’t find anything as good as GitVersion
  read more

Go R1 Day 58

progress

  • Avoiding a panic in Go for missing dictionary match is very straight forward. The error pattern for failed conversions and out of range index matches is the same, with: ok, err := action.
  • TODO: Figure out if ok to reference an error in a test by: is.Equal(error.Error(),"unable to find value in map"). Linter warns me with: Method call 'err.Error()' might lead to a nil pointer dereference.
  • Started work with dependency injection.
  read more

My Goland Log

Purpose

This is a log of my second journey in using Goland from Jetbrains for Go development.

I’ve got VSCode in a great state. It’s flexible, powerful, and I’ve highly customized it to my workflow.

However, doing Go development, I’d like to better explore Jetbrains Goland and see if the experience proves positive.

I’ll log updates and issues here as I work through it in the hope that it might provide you better information if you are considering Goland as well. …

  read more

Go R1 Day 55

progress

In Go, when you call a function or a method the arguments are copied. 1

  • Built some test cases for working with pointers and test methods.
  • Did this in Goland to become more familar with it and figured out how to use the nice tests explorer with a floating window to auto-test.
  • Built a filewatcher configuration watcher as alternative to tests panel, allowing automated run in terminal or output panel of go test ./....
  • Couldn’t figure out how to run every test in project, as …
  read more

Go R1 Day 54

progress

  • worked with tests and Goland.
  • Modified table driven tests to remove hard coded test case inputs.

Steampipe Is Sql Magic

Up And Running In Minutes

I tried Steampipe out for the first time today.

 Follow Steampipe On Twitter

I’m seriously impressed.

I built a project go-aws-ami-metrics last year to test out some Go that would iterate through instances and AMIs to build out aging information on the instances.

I used it to help me work through how to use the AWS SDK to iterate through regions, instances, images, and more.

In 15 mins I just solved the equivalent issue in a way that would benefit anyone on …

  read more

Go R1 Day 53

progress

  • Troubleshooting on: typechecking loop that helped me learn a bit more on how the compiler parsing occurs. Quick fix was to simply change var logger *logger.Logger to var log *logger.Logger.
  • Read up on dependency injection concepts and clean architecture design.
  read more

Go R1 Day 52

progress

  • published extension pack for Go1
  • Learned about magic number linter in golanglint-ci. For instance this would be flagged as a bad practice (while not applicable for a simple test like this, having a const makes sense in almost all other cases).
1
2
3
func Perimeter(width float64, height float64) float64 {
	return 2 * (width + height)
}
  • Learned a few extra linter violations and how to exclude including:
    • lll: for maximum line length
    • packagetest: for emphasizing blackbox testing. …
  read more

My First Vscode Extension Pack for Go

Took a swing at creating my own extension pack for Go.

  sheldonhull/extension-pack-go - GitHub

This was a good chance to familarize myself with the eco-system and simplify sharing a preset group of extensions.

Setup the repo with a Taskfile.yml to simplify running in the future. If frequent updates needed to happen, it would be easy to plug this into GitHub actions with a dispatch event and run on demand or per merge to main.

Here’s the marketplace link if you want to see what it …

  read more

Go R1 Day 51

progress

  • Did iteration exercise, however, I skipped ahead and did strings.Repeat instead of iteration because I’m lazy. πŸ˜€
  • Moved all tests into blackbox test packages.
  • Worked through variadiac functions.
  • Tweaked my VSCode autotest to run on save.
  • Further tweaks to golanglint-ci to reduce noise on linting checks.

  read more

Use Driftctl to Detect Infra Drift

Use Driftctl to detect drift in your your infrastructure. This snippet generates a html report to show coverage and drift figures of the target.

For multiple states, you’ll need to adapt this to provide more --from paths to ensure all state files are used to identify coverage.

1
2
3
4
5
6
7
8
9
$S3BucketUri = "terraform-states-$AWS_ACCOUNT_NUMBER/$AWS_REGION/$TERRAFORMMODULE/terraform.tfstate"
$Date = $(Get-Date -Format 'yyyy-MM-dd-HHmmss')
$ArtifactDirectory = ( …
  read more

Go R1 Day 50

progress

At this point, I’m still struggling with the proper way to abstract a logging wrapper that calls a logging library. There’s enough boilerplate for setup of my preferred defaults in zerolog that I want to include …

  read more

Go R1 Day 49

progress

  • Learned about White-Box vs Black-Box testing. Apparently, you can access all indentifiers of a package if you use the same package name such as: package packagename. If you are testing as a consumer might be, then you can use package packagename_test for only accessing the exported identifiers.
  • Used examples in test file to provide self-documentation of how to use the method.
  • Worked further with golanglint-ci and found it challenging when working with multiple modules in …
  read more

How To Reduce Noise From Slack

Noise

Slack can be a really effective tool with asynchronous work, but the defaults aren’t great out of the box.

I want to get you a few tips on how to use Slack effectively.

Piping through release notifications, work-item updates, alerts and more can help you reduce context switching with other tools, but without proper control you’ll likely find it overwhelming.

Sections

Use sections to organize your content and customize the level of priority you want to assign to the grouped …

  read more

Go R1 Day 48

progress

  • worked through more test concepts with learn-go-with-tests, including test helpers.
  • enjoyed the results GitHub copilot was producing. I can see a lot of promise for speeding up things and reducing the need for snippets as it builds code suggestions on the fly with far more context than a general snippet could do.

  read more

Luminar AI

Update

Customer service bumped my older version of Luminar 3 to 4. The issues with distortion on the preview screen aren’t there with version 4.

The new AI enhancements aren’t there, so it’s missing some of the new features. For now I’m going to try and use this as my daily driver for images and see how it goes. …

  read more

Simplify Aws Developer Security With Leapp

Security Is Necessary Work

Security can add overhead, get in the way, complicate development, generally can cause challenges to development efforts. The balance of security and usability is one that all development organizations have to deal with.

I could be ultra-secure by requiring carrier pigeon multi-factor authentication for my team, but I’m not sure that would be well received.

  read more

Go R1 Day 47

progress

  • worked with lefthook to run golang-lint tools and discovered how vast the configuration for this was, simplifying many of the other linting tools that was running seperately.
  • created some goyek tasks for bringing up and down docker compose stacks.
  • learned that file.Close() shouldn’t be used with defer to avoid unsafe file handling operations.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
// taskComposeDestroy tears down the docker stack including volumes
// this is …
  read more

Go R1 Day 46

progress

  • At $work I focused on dockertest.
  • Modified the provided test statements to ensure autoremoval of container occurred on failure.
  • Had packet issues I couldn’t figure out this time, so shelved for now. packets.go:37: unexpected eof
  • At home, I played with bubbleteam a bit, and decided while an epic TUI interface, the framework is far too involved for what I want to mess around with at this time. For instance, it doesn’t provide multi-select, instead much of that is manually …
  read more

Understanding The Basics of SQL Server Security

Confusing

As I’ve worked with folks using other database engines, I’ve realized that Microsoft SQL Server has some terminology and handling that is a bit confusing. Here’s my attempt to clarify the basics for myself and others needing a quick overview. This is not comprehensive coverage of security architecture, which is a very complex topic, more just terminology.

Terminology

Note that it’s best to consider SQL Server as it’s own operating system, not just a …

  read more

Go R1 Day 45

progress

  • Worked through merging flags and the “run” approach from Mat Ryer.
  • Used ff for parsing.

The resulting logic seems correct with main being very simple with:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
package main
import(
    "io"
    "flag"
    "os"
    "github.com/peterbourgon/ff/v3"
    "github.com/peterbourgon/ff/v3/ffcli"
    "github.com/peterbourgon/ff/v3/fftoml"
)
const (
 // exitFail is the …
  read more

Go R1 Day 44

progress

  • Generated tests for Gorilla Mux and realized need more context and reading on how to properly test Gorilla Mux to avoid excessively mocking.
  • Reviewed two different paradigms for struct methods, using parameters and using global configuration objects.
  read more

Go R1 Day 43

progress

  • Modified forked SharedBrain repo to use yaml parser instead of toml.
  • Modified tests handle invalid casting of interface, which was causing a panic.
1
2
3
4
otherDate, ok := otherDateInt.(time.Time)
if !ok {
  log.Printf("[time.Parse] probable invalid date format %s", plainFilename)
}
  • Improved tests to align to markdown standard formatting.
  • FOSS license scanned on 4 repos to test compliance of licensing for badge.
  • Use goyek templates to build out initial go based build …
  read more

Go R1 Day 41

progress

  • Enabled Go code coverage for tests in VSCode
  • go install github.com/jpoles1/[email protected] to install tooling for generating code coverage badge for readme.
  • Set ![gopherbadger-tag-do-not-edit]() in the readme, and then this gets replaced with a code coverage percentage badge.
  • Generate the required code coverage reports using: go test -coverprofile cover.out followed by go tool cover -html=cover.out -o coverage.html for a visual report.
  read more

Go R1 Day 40

progress

  • Learned a bit more modules vs packages, including internal package special behavior.
  • Configured Azure DevOps remote module source with package level imports.
  • Successfully migrated a utility logger for setup of Zerolog into a seperate remote module for importing.
  • Setup 50% test coverage for logger using gotest test generator.
  • Used Is for test setup. (this would be a cool use of gotest templates test generation)
  • Modified sharedbrain forked repo to generate yaml front matter with dashes. …
  read more

Go R1 Day 39

progress

  • built a better understanding of how to resolve Go modules from private Azure DevOps instances

Using Azure DevOps for Private Go Modules

TL;DR

This took a few hours of work to iron out, so figured maybe I’d save someone time.

⚑ Just keep it simple and use SSH

⚑ Use dev.azure.com even if using older project.visualstudio.com to keep things simple.

Modules Support

Unlike GitHub, Azure DevOps has some quirks to deal with, specifically in the odd path handling.

My original goal was to set the default handling to be https support, with the SSH override in git config allowing me to use SSH.

This didn’t work.

  • HTTPS requires …
  read more

GitPod User Feedback

  1. Thank you GitPod for integrating with VSCode. πŸŽ‰
  2. Upcoming ssh support, so freaking cool! Are you using Chisel in the backend per the GitHub issues I found discussing workaround? As a Go dev, I’m interested ⚑ I believe Jetbrains + VSCode users will now both be able to leverage the same backend.
  3. GitHub took over the codespace arena with Visual Studio Codespaces being abandoned. Enterprise customers using Azure Repos can’t benefit from any of this new flexibility πŸ’”. Is it in the roadmap …
  read more

Log4 Brains Improves ADR Experience

I’m a fan of architectural decision records. Docs as code sets a practice for a team to track decisions in process and design.

Log4Brains provides markdown template based adr creation, with an option to render a static site for reading.

If you want to start managing your ADRs in a structured way, but benefit from simple markdown as the source files, then this is a great option to consider.

It’s really easy to get started with docker, no npm install required.

Just run: docker run --rm …

  read more

Career Advice From a Drunk Dev on Reddit

Drunk Post: Things I’ve learned as a Sr Engineer has trending with some fantastic discussion on Twitter & Reddit.

What makes it so interesting is the raw and unfiltered message, mixed in with some great humor.

My Background

The author of that post comes from a Data Engineer side.

I’ve been a Database Developer, Production …

  read more

Highlight Problem Characters in Code

Use Vscode Gremlins to help flag characters that shouldn’t be in your code.

Specifying a Range of Invalid Characters

You can give a range to flag multiple characters with a single rule.

For example, if using macOS and the option key is set to a modifier, it’s easy to accidentally include a Latin-1 Supplemental Character that can be difficult to notice in your code.

To catch the entire range, the Latin-1-Supplement link provided shows a unicode range of: 0080β€”00FF

Configure a rule …

  read more

The Dark Side

Saw this on a Twitter thread and had to save it. If you’ve ever debated light vs dark themes, you might find this as funny as I did.

Nearly choked on my dinner.

Go R1 Day 38

progress

  • Worked on cross platform binary builds and CICD with GoReleaser.
  • Lots of container work here.
  • Started blog post on how to use GoReleaser.

Value Local Art

Our culture is driven on some much virtual comsumption of media and performances. It’s important to remember just how critical it is to support local art and performance.

In a YouTube culture, it’s easy to default to virtualized arts. I think it’s easy to lose the appreciation for the disciplined effort creating music, dance, art requires when it’s all just something on a screen.

It also gives an unrealistic expectation for live performances if our only consumption is the …

  read more

Dance Time

Uploaded a few to my gallery from time I had with my daughter watching her live performance this weekend. Take a peek.

I need to catch-up on a few posts with the family with all the great stuff they’ve been doing lately. (My son just got his black belt in Taekwondo πŸ‘)

Dance Time

The Open Guide to Amazon Web Services

The guide you didn’t know you needed for AWS. Came across this when Corey Quinn 1 mentioned this on a podcast.

This is a resource I wish I’d known about earlier. It covers the core essentials on services, and cuts through the confusion of the AWS Docs pretty well.

For example, I noticed the ELB references for ALB docs and this caused me some confusion.

Right front and center on Load Balancer Basics 2:

Before the introduction of ALBs, “Classic Load Balancers” were known …

  read more

Cloudflare Analytics for Your Jamstack Site

A new alternative to Google Analytics if you want to keep things simple.

Cloudflare Web Analytics seems promising.

I’d probably have started with this if I was setting up my site from scratch.

The problem I have with adopting a tool like this is no basic import of metrics from Google Analytics means I’d lose years of history.

Does the history matter signficantly? Not really for me. I just don’t like losing out the entire history to see how things have improved over time.

  read more

Make Vscode Annoy Me When I Make a Typo

Not sure why, but I’ve had 2 typos that keep plauging me.

  • ngnix should be nginx
  • chocolatey should be chocolatey

With Go, I get compile errors with typos, so no problem. With PowerShell or Bash, this can be really annoying and waste time in debugging.

You can configure many autocorrect tools on a system level, but I wanted a quick solution for making it super obvious in my code as I typed without any new spelling extensions.

Install Highlight: fabiospampinato.vscode-highlight 1

Configure …

  read more

Cool Way to Compare Features in Note Apps

Nice little site I came across for comparing various popular knowledge & note taking tools with ability to select features most important to you.

NoteApps

Note typically a fan of compare based tools, as they often feel like marketing junk, but this was a a pretty cool little implementation.

Azure Devops PR Automation Requires Generic Contribute

Big fan of renovate for terraform dependency management.

Ran into some complications with restarting an onboarding with Whitesource Renovate in Azure DevOps Repos and the Azure Pipeline automation. I’ve run into this before, so logging this for my sanity.

  • If you failed to onboard with Azure DevOps, you can’t rename the PR like in GitHub to simplify restarting the onboarding process.
  • Instead, delete the renovate.json file and commit to your default branch.
  • Then re-add and get the …
  read more

Hackattic

Came across this pretty nifty site HackAttic that provides a small set of practical oriented exercises. Will look at using further in my Go journey, as seems really useful to stretch one’s development skills in less familar, but still practical ways.

A few of the challenges:

  • Generate an SSL certificate according to given data.
  • Extract some data from Redis. In a way.
  • Handle some incoming JWTs for us.

Take a look at the newsletter topics covered. I love that sense of humor.

  read more

Diagrams as Code

Why

It’s not for everyone.

I prefer to write diagrams out logically and then let the visual aspect be generated.

This helps me by making the visual presentation an artifact of what is basically a written outline.

Presentation Matters

I’ve learned that communicating flows of information can often be better presented visually than trying to write them out, especially once it involves more than a few “nodes” of responsibility. Visualizing a complicated process can be done …

  read more

Go R1 Day 37

progress

  • initialized new start of cli tool to try to parse markdown content using Goldmark1 and then convert markdown links into reference style links
  • Improved default template for starting new basic cli tools with Go to include zerolog and run based design instead of the majority of logic being in main.

  1. goldmark ↩︎

  read more

Go R1 Day 36

progress

  • more taskflow work
  • learned how to use io.Multiwriter to have both a stdout stream as well as output captured to a stringbuilder.
  • got confused on reading docs, only to realize I was reading docs for a C# solution. πŸ˜‚
  • used Pterm1 to beautiful console output, which was pretty cool to see in action.

  1. pterm/pterm: ✨ #PTerm is a modern go module to beautify console output ↩︎

  read more

Go R1 Day 35

progress

  • Worked with Taskflow a bit more.
  • Need to identify better error handling pattern on when to resolve vs handle internal to a function, as it feels like I’m doing needless error checking.
  • Wrote func to run terraform init, plan, and apply.
  • This takes dynamical inputs for vars and backend file.
  • Also dynamically switches terraform versions by running tfswitch.

Definitely more verbose code than powershell, but it’s a good way to get used to Go while achieving some useful …

  read more

Go R1 Day 34

progress

  • figured out how to import util/logger.go as a package
  • after much confusion due to logger, log, *zerolog.Logger and more variables all deviously similar in name… how to pass around the initialized package logger that I configure.
  • learned that global scope and package scoped loggers being initialized at run is concerned an anti-pattern
  • properly wrapping to avoid the log initialization on import with type Logger struct { logger: *zerolog.Logger; } as an example avoids the same …
  read more

Go R1 Day 33

progress

  • successfully created logging package using zerolog
  • learned about scoping with packages
  • linked to a private internal repository and how to leverage the module replace operator to temporarily alter path import from url to local override.
  • middleware is a newer concept, so I need to learn more on this later so I can understand how to use to inject special log handling for http requests and other actions.

Thoughts for today are that the pressure of jumping into an existing codebase is …

  read more

Go R1 Day 32

progress

  • created some structured logging improvements with zerolog
  • began exploration of middleware concepts for logging
  • generated test stubs using gotests

Go R1 Day 31

progress

  • Learned a bit about idiomatic patterns wtih error handling.
  • Learned about inline block intiailization of variables using if err := method(); err != nil {...} approach.
  • Considered a bit more idiomatic patterns when I noticed excessibe nested if blocks.
1
2
3
4
5
6
tfdir := tf.Params().String("tfdir")
if tfdir != "" {
  tf.Logf("tfdir set to: [%s]", tfdir)
} else {
  tf.Errorf("πŸ§ͺ failed to get tfdir parameter: [%v]", tfdir)
}

This would probably be …

  read more

Go R1 Day 30

progress

  • Built some go functions for build tasks work with terraform and setup of projects using taskflow.

Learned one one to pass in arguments using slices. I’m pretty sure you can use some stringbuilder type functionality to get similar behavior, but this worked fine for my use case.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
cmdParams := []string{}
cmdParams = append(cmdParams, "-chdir="+tfdir)
cmdParams = append(cmdParams, "init")
cmdParams = append(cmdParams, …
  read more

Go R1 Day 29

progress

  • Evaluated Mage as a replacement for bash/pwsh based tasks for automation with Azure Pipelines.
  • Was able to get terraform to run with dynamic configuration using the following approach:

Install with

1
2
3
4
5
go get -u github.com/magefile/mage/mg
go mod init mage-build
go get github.com/magefile/mage/mg
go get github.com/magefile/mage/sh
go mod tidy

Then to get mage-select run:

1
2
3
GO111MODULE=off go get github.com/iwittkau/mage-select
cd $GOPATH …
  read more

Fix Terraform Provider Path in State

Fixing Terraform provider paths in state might be required after upgrading to 0.13-0.14 if your prior state has the following paths.

First, get the terraform providers from state using: terraform providers

The output should look similar to this:

  read more

Azure Pipelines Template Iteration

Templates

This isn’t meant to be an exhaustive template overview. I’m just going to give an example of what I found useful as I’ve been meaning to leverage templates for a while and finally got around to having a great use for it.

My Use Case

I am a heavy user of InvokeBuild. It’s a very robust task runner tool that I’ve used to coordinate many project oriented actions, similar to Make, but also DevOps oriented work like AWS SSM tasks and more.

In this scenario, I …

  read more

Git Workflow With Git Town

Resources

Git-Town

Painful But Powerful

Let’s get this out of the way.

Git isn’t intuitive.

It has quite a bit of a learning curve.

However, with this flexibility comes great flexibility. This tool has powered so much of modern open-source development.

Optimize for the Pain

To improve the development experience some tools can help provide structure.

This won’t be an attempt to compare every git GUI, or push any specific tooling. It’s more sharing my experience and what …

  read more

Go R1 Day 28

progress

  • Solved [Hamming Distance] on exercism.io
  • Simple problem, but reminded me of how to use string split.
1
2
3
4
5
6
7
8
9
diffCount := 0
aString := strings.Split(a, "")
bString := strings.Split(b, "")

for i, x := range aString {
  if x != bString[i] {
    diffCount++
  }
}
  • Reviewed other solutions, and found my first attempt to split the string wasn’t necessary. Looks like I can just iterate on the string directly. I skipped this as it failed the first time. …
  read more

Incremental and Consistent

It’s really hard to prioritize when life gets busy, but it’s important that continued improvement is a priority. Great at Work: How Top Performers Do Less, Work Better, and Achieve More was a really interesting book. The fact that small incremental improvement done daily can make such a difference is pretty …

  read more

Working With Powershell Objects to Create Yaml

Who This Might Be For

  • PowerShellers wanting to know how to create json and yaml dynamically via pscustomobject.
  • Anyone wanting to create configs like Datadog or other tools dynamically without the benefit of a configuration management tool.
  • Anyone else wanting to fall asleep more quickly. (I can think of better material such as the Go spec docs, but hey, I can’t argue with your good taste πŸ˜„)

YAML

It’s readable.

It’s probably cost all of us hours when debugging yaml …

  read more

Nativefier

Update 2021-09-20
Updated with improved handling using public docker image.
Update 2021-05-10
Added additional context for setting internal-urls via command line.

  read more

Go R1 Day 27

progress

  • Iterated through AWS SDK v1 S3 buckets to process IAM policy permissions.
  • Unmarshaled policy doc into struct using Json-To-Struct.

Github Pages Now Supports Private Pages

I’m a huge static site fan (lookup jamstack).

What I’ve historically had a problem with was hosting. For public pages, it’s great.

For private internal docs, it’s been problematic. It’s more servers and access control to manage if you want something for a specific group inside a company to access.

This new update is a big deal for those that want to provide an internal hugo, jekyll, mkdocs, or other static generate based documentation site for their team.

Access …

  read more

Ensuring Profile Environment Variables Available to Intellij

Open IntelliJ via terminal: open "/Users/$(whoami)/Applications/JetBrains Toolbox/IntelliJ IDEA Ultimate.app"

This will ensure your .profile, .bashrc, and other profile settings that might be loading some default environment variables are available to your IDE. For macOS, you’d have to set in the environment.plist otherwise to ensure they are available to a normal application.

ref: OSX shell environment variables – IDEs Support (IntelliJ Platform) | JetBrains

  read more

Create an S3 Lifecycle Policy with PowerShell

First, I’m a big believer in doing infrastructure as code.

Using the AWS SDK with any library is great, but for things like S3 I’d highly recommend you use a Terraform module such as Cloudposse terraform-aws-s3-bucket module. Everything Cloudposse produces has great quality, flexibility with naming conventions, and more.

Now that this disclaimer is out of the way, I’ve run into scenarios where you can have a bucket with a large amount of data such as databases which would be …

  read more

Leverage Renovate for Easy Dependency Updates

Update 2021-06-30

Added example from renovate documentation with some notes on the Azure DevOps Pipeline to leverage their free renovate service. GitHub users benefit from the Renovate app, but Azure Pipelines should use an Azure Pipeline definition.

Follow the instructions from the Renovate Me task linked in resources, and ensure the …

  read more

Go R1 Day 26

Progress

  • Evaluated gorm usage best practices with Slack Gopher community.
  • Obtained a great example to get me started on go routine and channels usage with multi-database queries.