Ergonomic Multi-Line Strings for bash, zsh, and Other Shells


## The State of Multi-Line Strings

Multi-line strings in shells are a pain.

Suppose you want to create a file, using a shell script, which contains the following content

```ts
export default class Greeter {
  greet(name: string) { return 'Hello, ' + name + '!'; }
}
```

How can this be achieved?

### Some Methods

#### Method 1: a multi-line variable

This simple solution works when the variable definition is not indented at all

```sh
var="export default class Greeter {
  greet(name: string) { return 'Hello, ' + name + '!'; }
}"
```

```
$ echo $var
export default class Greeter {
  greet(name: string) { return 'Hello, ' + name + '!'; }
}
```

But what if we're defining `$var` within a function, and we want it indented along with the rest of the function body?

```sh
function my_function() {
  var="export default class Greeter {
    greet(name: string) { return 'Hello, ' + name + '!'; }
  }"
  echo $var
}
```

```sh
$ my_function
export default class Greeter {
    greet(name: string) { return 'Hello, ' + name + '!'; }
  }
```

Oh, well, that's obviously not what we want.

So, simple variable assignment: it works in a very limited subset of cases, when the variable definition is not indented at all. Let's try another method.

#### Method 2: a single-line variable with '`\n`'s for line breaks

In this method, we replace all of the line breaks in the multiline string with `\n` line break characters:

```sh
function my_function() {
  var="export default class Greeter {\n  greet(name: string) { return 'Hello, ' + name + '!'; }\n}"
  echo $var
}
```

```sh
$ my_function
export default class Greeter {
  greet(name: string) { return 'Hello, ' + name + '!'; }
}
```

The result looks good, but the method is messy. What if we want to reformat this like

```ts
export default class Greeter {
  greet(name: string) {
    return 'Hello, ' + name + '!';
  }
}
```

That would involve adding more `\n` characters, and spaces to match the indentation. It's not extremely straightforward:

```sh
function my_function() {
  var="export default class Greeter {\n  greet(name: string) {\n    return 'Hello, ' + name + '!';\n  }\n}"
  echo $var
}
```

So, explicit line break characters: this works if the text you want formatted won't change often, and if the readability of the implementation doesn't matter. If you want the text-generating code itself to be readable or maintainable, this is not a great solution.

So what else can we do?

#### Method 3: Heredocs

Multiline strings are what [Heredocs](https://en.wikipedia.org/wiki/Here_document) were made for:

> In computing, a here document (here-document, here-text, heredoc, hereis, here-string or here-script) is a file literal or input stream literal: it is a section of a source code file that is treated as if it were a separate file. The term is also used for a form of multiline string literals that use similar syntax, preserving line breaks and other whitespace (including indentation) in the text.

So let's see how well they work for our problem

```sh
function my_function() {
  var=$(cat <<EOF
  export default class Greeter {
    greet(name: string) { return 'Hello, ' + name + '!'; }
  }
  EOF)
  echo $var
}
```

```sh
$ my_function
/Users/andrew/test.sh:8: parse error near `var=$(cat <<EOF'
```

Oh, uh, yeah, obviously the delimiter sequence (`EOF` in this case), cannot be indented, and must appear on a line by itself, so we have to write

```sh
function my_function() {
  var=$(cat <<EOF
  export default class Greeter {
    greet(name: string) { return 'Hello, ' + name + '!'; }
  }
EOF
  )
  echo $var
}
```

...which is fine, but sort of breaks indentation of the rest of the function body. It also doesn't work:

```sh
$ my_function
  export default class Greeter {
    greet(name: string) { return 'Hello, ' + name + '!'; }
  }
```

Just like Method #1, this method adds to the output the whitespace we used to indent the function body, which we don't want.

So how can we preserve _only_ the indentation _we want_ (and maybe get rid of that ugly heredoc delimiter)?

### My Method

Here's how I do it

```sh
function my_function() {
  var="$(sed -e 's/^[ ]*\| //g' -e '1d;$d' <<'--------------------'
    | 
    | export default class Greeter {
    |   greet(name: string) { return `Hello, ${name}!`; }
    | }
    | 
--------------------
    )"
  echo $var
}
```

I use pipe characters `|` to define a "margin", which I then strip out using `sed`. `sed -e 's/^[ ]*\| //g'` will remove any number of space characters (`[ ]*`) at the beginning of the line (`^`), followed by a pipe (`|`), followed by one space character (`[ ]`).

> This "margin" method was inspired by [Scala's `String#stripMargin` functionality](https://www.oreilly.com/library/view/scala-cookbook/9781449340292/ch01s03.html), which behaves in a very similar way.

The second `sed` expression, `-e '1d;$d'`, removes the first and last line. I add blank lines to provide a bit of visual whitespace around the content I want to write to the variable. If you don't want one or both of these blank lines, remove them with this slight variation on my method

```sh
function my_function() {
  var="$(sed -e 's/^[ ]*\| //g' <<'--------------------'
    | export default class Greeter {
    |   greet(name: string) { return `Hello, ${name}!`; }
    | }
--------------------
    )"
  echo $var
}
```

I don't mind the line of hyphens, either, as the `EOF` replacement, because it sort of acts like the top and bottom margin of the content. But, if you put the heredoc delimiter in quotes, as I have above, you can also include whitespace in it. So you could do something like

```sh
function my_function() {
  var="$(sed -e 's/^[ ]*\| //g' <<'    +'
    | export default class Greeter {
    |   greet(name: string) { return `Hello, ${name}!`; }
    | }
    +
    )"
  echo $var
}
```

Though I personally think this leaves a bit too much whitespace under the content. Also, many syntax highlighting algorithms have trouble with this.

With any of these variations, you can indent the content to whatever level you like

```sh
function my_function_1() {
  var="$(sed -e 's/^[ ]*\| //g' <<'--------------------'
    | export default class Greeter {
    |   greet(name: string) { return `Hello, ${name}!`; }
    | }
--------------------
    )"
  echo $var
}

function my_function_2() {
  var="$(sed -e 's/^[ ]*\| //g' <<'--------------------'
| export default class Greeter {
|   greet(name: string) { return `Hello, ${name}!`; }
| }
--------------------
    )"
  echo $var
}

function my_function_3() {
  var="$(sed -e 's/^[ ]*\| //g' <<'--------------------'
              | export default class Greeter {
              |   greet(name: string) { return `Hello, ${name}!`; }
              | }
--------------------
    )"
  echo $var
}
```

```sh
$ my_function_1; my_function_2; my_function_3
export default class Greeter {
  greet(name: string) { return `Hello, ${name}!`; }
}
export default class Greeter {
  greet(name: string) { return `Hello, ${name}!`; }
}
export default class Greeter {
  greet(name: string) { return `Hello, ${name}!`; }
}
```

The flexibility -- combined with the visual aesthetics -- of this method is why it's recently become my go-to for multi-line strings in the shell.

Elegant Multi-Line Shell Strings

Debugging a New Feature in a Vercel TypeScript GitHub Repo


**Part III: The Deep End**

At this point, I wasn't sure where to go next.

Why should [these blog posts now have a seemingly random publication date](https://www.awwsmm.com/blog/whats-wrong-this-time-part-2), which was not the current date, and was not the date they were first added to the repo? What was going on?

So I did what most programmers do in a situation like this: I added some `console.log()`s.

## Digging Deeper

I wanted to know what commits were available, and what information they had. In other words, I wanted better [_observability_](https://dzone.com/articles/rethinking-programming-automated-observability) into what was going on in this bit of the codebase.

My first idea was to just [print out the dates of every line returned from `git log`](https://github.com/awwsmm/awwsmm.com/commit/a43bf7c356aa4b649ddfbf26528a454ae94bd807). The output of that (for a given blog post) looked something like

```
lines returned from git log
2022-01-09T20:48:23+00:00
2022-01-06T09:18:41+00:00
```

...okay, not extremely useful. Why are there only two commits here? This is just the same information we had on the website. Maybe [printing out the hash of each commit](https://github.com/awwsmm/awwsmm.com/commit/0477f384c0cb604c8cd4b64307c1b37b150575dc) would be a bit more helpful? Then I could check if there was anything unusual about these commits

```
lines returned from git log
69e038a919e448251fa2211a9fcf3fda914812fe @ 2022-01-09T20:48:23+00:00
d5cf8fbc05891ac9d8d7067b5cb1fb195dc2cf99 @ 2022-01-06T09:18:41+00:00
```

Now we can search GitHub for that commit [`dc2cf99`](https://github.com/awwsmm/awwsmm.com/commit/d5cf8fbc05891ac9d8d7067b5cb1fb195dc2cf99).

But this commit doesn't add or update _any_ blog posts... so why is it being returned from `git log <path/to/blogpost>`?

What if I `git log`ged a file that has _definitely_ been around since the first commit, like `index.tsx`. I tried printing out every log line for this file and saw the following on Vercel

```
lines returned from git log index.tsx
69e038a919e448251fa2211a9fcf3fda914812fe @ 2022-01-09T20:48:23+00:00
88c420835d35a008de808b7cef04980a15b029bc @ 2022-01-09T12:55:49+00:00
0a882cf5062e4c0ac4505ed609ca77f14b35a76a @ 2022-01-08T20:15:44+00:00
d4a9a360c38398cdd41825aa0fe193e8176cb4fd @ 2022-01-07T22:41:52+00:00
3acb76c1f6c6d1b4cdb76939496e251220aa29ea @ 2022-01-06T20:09:17+00:00
```

It only goes back five commits! The commit history looked the same for other long-lived files as well. Only ever going back to that last commit on January 6.

Running the same code on my local machine gives many more commits, going back all the way to the first commit on January 2.

What gives?

## The Shallow End

At this point, I wasn't sure how much more debugging I could do. So I started doing a bit of research.

And I found [this issue ("How to unshallow repo?")](https://github.com/vercel/vercel/discussions/5737) on the Vercel GitHub repo

> _"Hello, I need to define a variable at build time that depends on `git describe` which depends on git history, but it seems the repo in vercel build enviroment is a shallow clone with only few last commits."_

That sounds like my problem! And it sounds like it's caused by Vercel making [a _shallow clone_ of my git repo](https://www.perforce.com/blog/vcs/git-beyond-basics-using-shallow-clones) before building. I'd never encountered shallow cloning _in the wild_ before, but I knew of it as a concept, which is how I found that GitHub issue.

So how can we work around this? We simply won't have the information available at build time to determine the correct "published" and "last updated" dates for a given blog post.

But there's always a way to work around these kinds of limitations. In this case, that involves a cache.

## Cache Rules Everything Around Me

There are a few ways we could solve this problem. We could, for instance, use the GitHub API to pull commit information from the repo hosted on GitHub.com. I chose not to do this as I preferred to keep the solution self-contained: we have all the information available at build time _when running locally_, so how could we make that information available when building _for production_ (on Vercel), as well? (Where we'll have a shallow clone of the repo.)

Rather than make API calls over the internet for information which is available locally, I thought we could simply save this information in a cache, and then use that cache when building on Vercel.

The workflow I came up with for writing blog posts (and caching the important git info) looked something like this

1. draft a `wip-` post (these are [ignored for version control by my `.gitignore`](https://github.com/awwsmm/awwsmm.com/blob/4281812447b58b08bcd613530e4d8db2b1855be6/.gitignore#L23))
2. when the draft is ready, `git commit` it to the `development` branch and push to Vercel
3. for...
    - new blog posts (where the only commit in `git log` is the current commit), Vercel assumes the post is brand-new and uses the date of the current commit for the "published" and "last updated" times
    - old blog posts (where more than one commit references this blog post), Vercel looks for cached "published" and "last updated" times, and throws an error if it doesn't find any

There are a few small problems with this.

First, when do we update the cache? You'll notice that there is no step in the workflow above for ensuring that the cache is up-to-date. Since we only have access to the required information when building locally, we have to update the cache when building locally. But when does this information get pushed to the remote repo? We have to enforce that, as well.

Second, the above workflow has a problem when we merge the `development` branch into the `master` branch when promoting a new release to production -- the merge commit itself means that the "new" blog post is now in two commits. As outlined above, this will cause Vercel to throw an error if the post isn't in the cache (it won't be).

## So... What Now?

I've got some hacky fixes for the above problems implemented.

For instance, I've got a [pre-push git hook](https://github.com/awwsmm/awwsmm.com/blob/4281812447b58b08bcd613530e4d8db2b1855be6/git-hooks/pre-push) which runs a build before each `git push`. This means that -- in theory -- the cache is always up to date. But of course, I need to make sure to `git add` it in the _next_ commit.

As for the "merging creates a new commit" issue, I've tried two solutions so far.

The first was to distinguish between commits on the `development` branch and commits on the `master` branch. Only blog posts with commits on `master` should be considered as "old". This works great when running locally, but the clone that Vercel creates seems to _rename_ this `development` branch to `master` when building a preview deployment. So that's a no-go.

The second solution (which I'm currently using) is to simply [ignore merge commits](https://github.com/awwsmm/awwsmm.com/blob/4281812447b58b08bcd613530e4d8db2b1855be6/lib/blog/SlugFactory.ts#L28).

So far, the above appears to be working. But it feels like an overly complex and fragile solution, and I hope to improve upon it in the future. Maybe just querying GitHub for the commit history is easier than going through all of this cache trouble.

## Conclusion

So that's it! The goal was simple: get rid of arbitrary "published" times on blog posts and pull that data directly from the project's git history. But the solution ended up being much more complex and nuanced than I had initially planned.

But along the way, I learned some new tools and tricks, I learned a bit more about how my repo is built and deployed on Vercel, and I have some ideas for how I can make things more streamlined in the future. And that's what this is all meant to be, really, a learning experience.

In the future, maybe I'll do away with this overly-complex caching mechanism, but I do want to get the "published" and "last updated" dates from the repo's git history. This initial solution, while messy, does the job for now.

What's Wrong This Time? Part III: The Deep End


**Part II: The Bugs**

There's an old programming joke that goes [something like](https://martinfowler.com/bliki/TwoHardThings.html)

> _There are only two hard problems in computer science: cache invalidation, naming things, and off-by-one errors._

I think we should add a third (fourth?) problem to that list: sorting things.

## Sorting Things

There are [lots of different ways to sort things](https://en.wikipedia.org/wiki/Sorting_algorithm) in computer science. C.S. students learn about time and space complexity of these sorting algorithms, YouTubers make [cool visualisations](https://www.youtube.com/watch?v=BeoCbJPuvSE) of them, and occasionally, [a guy named Tim](https://en.wikipedia.org/wiki/Timsort) will invent a new one.

But there's one aspect of sorting algorithms that -- for me, at least -- seems completely impossible: remembering in which direction things are sorted.

If you say to a group of people: "okay, everyone, stand in a single-file line, ordered by height", the next question you might ask is "okay, but in which direction?" Who should stand at the _front_ of the line? The shortest person or the tallest person?

In programming, we define comparison functions, which describe how to order whatever objects we're interested in.

Some comparison functions seem obvious. For example, in TypeScript, using the default `string` comparison...

```ts
const array: string[] = ["cherry", "apple", "banana"]
array.sort()
//...
```

...we would expect `array` to be sorted alphabetically, with `apple` as the first (`0`th) element of the sorted array

```ts
//...
console.log(array)    // [ 'apple', 'banana', 'cherry' ]
console.log(array[0]) // apple
```

> Note that `<array>.sort()` in JavaScript sorts the array "in place", so that the original, unsorted `array` no longer exists afterward. In some languages, and for some sorting algorithms, arrays are not sorted in place, and a new array will be returned. This new array should be assigned to a new variable.

But often we will be working with objects more complex than `string`s, and we will need to define custom comparison functions. These are functions which take two elements of type `T` and return a `number`, and are used to sort arrays of type `T`:

```ts
type T = string

const newArray: T[] = ["cherry", "apple", "banana"]

function comparison(t1: T, t2: T): number {
  return t1.charCodeAt(0) - t2.charCodeAt(0)
}

newArray.sort(comparison)

console.log(newArray)    // ?
console.log(newArray[0]) // ?
```

Without [reading the docs](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort), will the `console.log()`s above give the same result as the earlier ones? How about something a bit simpler -- sorting an array of `number`s:

```ts
type T = number

const newArray: T[] = [42, 2112, 19]

function comparison(t1: T, t2: T): number {
  return t2 - t1
}

newArray.sort(comparison)

console.log(newArray)    // ?
console.log(newArray[0]) // ?
```

Will the first element above be `19`? Or `2112`? Are you sure?

I understand the utility of sorting algorithms, and I understand the need for a ternary (greater than, less than, or equal) return value, and hence `number` as the return type instead of `boolean`, but comparison functions are just one of those things that I've always had to test every time. Sometimes in development, and sometimes in production.

## So What Happened?

With what we learned above, you should now be able to see what went wrong with [my initial code](https://github.com/awwsmm/awwsmm.com/commit/69e038a919e448251fa2211a9fcf3fda914812fe). The problem was here

```ts
    // get the blog post date from its git commit date
    const gitLog = SlugFactory.git.log({ file: `blog/${slug.params.slug}.md` });

    return gitLog.then(lines => {
      const dates = lines.all.map(each => each.date);

      // if blog post hasn't been committed yet, use current date
      const date = dates[0] ?? new Date().toISOString();

      return new FrontMatter(slug.params.slug, title, description, date, rawContent);
    });
```

`git log` returns commits sorted by date, such that _newer_ commits come first and _later_ commits come afterward. So `dates[0]`, above, is the _newest commit_ returned from `git log`, and each blog post was being given a "publication" date of the most recent commit in which that post was modified.

When were these blog posts most recently modified? Well, all of them were modified in [that same commit](https://github.com/awwsmm/awwsmm.com/commit/69e038a919e448251fa2211a9fcf3fda914812fe), because the point of the commit was to remove the `date` parameter from the front matter. Essentially, I was mixing up the `lastUpdated` date and the `published` date. One of these is the first element in the list (`dates[0]`) and one of them is the last element in the list (`dates[dates.length-1]`).

So like I said, there are four hard problems in computer science.

## On To The Next One

With that fixed, we're off to the races, right?

![Screenshot from awwsmm.com showing blog posts with incorrect dates](https://i.imgur.com/ME7yZQ9.png)

Oh... well, that's not right.

Those two posts were both committed on January 2 ([Hello, World!](https://github.com/awwsmm/awwsmm.com/commit/ec96618d38c71134f4a9ed14d6ae6d7a2b5c9e59) and [Git Hooks](https://github.com/awwsmm/awwsmm.com/commit/b2e504f52e4df0ddf77c662903e39b4aaf12f242)), not on January 6. So why did they both have the wrong date?

That's right, it's another bug... Or is it?

Find out in [the _thrilling_ final installation of this debugging mystery!](https://www.awwsmm.com/blog/whats-wrong-this-time-part-3)

What's Wrong This Time? Part II: Electric Bugaloo


**Part I: The Idea**

## _"I have an idea"_*, I said to myself.

"What if, instead of adding arbitrary dates to blog posts as I write them, I just got the dates from git?"

> [Git](https://git-scm.com/) is a [version control system](https://www.atlassian.com/git/tutorials/what-is-version-control) used by software developers which tracks changes to files, as well as creation and deletion of files. Every change, addition, or deletion is timestamped and has a [hash](https://en.wikipedia.org/wiki/Hash_function) which uniquely identifies those changes, like a fingerprint.

"[Each git commit](https://github.com/awwsmm/awwsmm.com/commits/master) has a date, so I should be able to programatically determine when a file in my [repo](https://www.gitkraken.com/learn/git/tutorials/what-is-a-git-repository) was added, or edited."

That was my idea for simplifying the metadata ("front matter") that I add to each of my blog post files on [my website](https://github.com/awwsmm/awwsmm.com). Before this adventure, that metadata looked [something like this](https://raw.githubusercontent.com/awwsmm/awwsmm.com/ec96618d38c71134f4a9ed14d6ae6d7a2b5c9e59/blog/hello-world.md):

```txt
---
title: 'Hello, World!'
description: 'Rewriting My Personal Website in Next.js and TypeScript'
date: '2022-01-01'
---
```

What I wanted to do was get rid of that arbitrary `date` field I was adding to every blog post, and just let git tell me when I published or updated a file. I could pull this information from the history of file changes that git keeps track of.

Sounds relatively simple, right?

Well, there were a few roadblocks to getting this up and running. Before we get into them, let's take a look at my initial, quick-and-dirty solution.

## First Attempt

### Git background

When working locally (on your personal computer), you can interact with git using the command line. One command, [`git log`](https://git-scm.com/docs/git-log), gives a list of all of the changes saved ("committed") to the project

```txt
$ git log --oneline -n 5
b6eea85 (HEAD -> master, origin/master, origin/HEAD) Merge pull request #15 from awwsmm/development
bc222b3 (origin/development, development) automatically date blog posts using git history
88c4208 order blog posts on index page reverse chronologically
9368f24 new blog post about nominal, structural, and duck typing
6bbe307 Merge pull request #1 from awwsmm/development
```

The options I passed to `git log`, `--oneline` and `-n 5`, tell git that I only want one line of minimal information for each commit (just the hash and the first line of the commit message), and that I only want the five most recent commits. By default, git shows something like

```txt
$ git log
commit b6eea85256fe54f34013f17aca4e892aba0778dd (HEAD -> master, origin/master, origin/HEAD)
Merge: 88c4208 bc222b3
Author: Andrew <aww@awwsmm.com>
Date:   Fri Jan 14 19:01:11 2022 +0000

    Merge pull request #15 from awwsmm/development

    automatically date blog posts using git history
```

...with the commit hash, the commit message, the date of the commit, the author, and so on. But lots of other options can be set for `git log`, for instance, specifying a hash will show only that commit and its ancestors (commits that came before it), and `--name-only` shows the names of all files modified in that commit

```txt
$ git log bc222b3 --name-only -n 1
commit bc222b37913ec2cdd7ff5f152dfaca06d33ec9f3 (origin/development, development)
Author: Andrew Watson <aww@awwsmm.com>
Date:   Fri Jan 14 18:54:39 2022 +0000

    automatically date blog posts using git history

.gitignore
README.md
blog/cache.json
blog/git-commit-hooks.md
blog/hello-world.md
blog/nominal-vs-structural-vs-duck-typing.md
lib/blog/FrontMatterFactory.ts
lib/blog/SlugFactory.ts
package-lock.json
package.json
pages/blog/[slug].tsx
pages/index.tsx
```

You can also pass a file path to `git log` and it will only show you commits in which that file was modified

```txt
$ git log blog/hello-world.md
commit bc222b37913ec2cdd7ff5f152dfaca06d33ec9f3 (origin/development, development)
Author: Andrew Watson <aww@awwsmm.com>
Date:   Fri Jan 14 18:54:39 2022 +0000

    automatically date blog posts using git history

commit ec96618d38c71134f4a9ed14d6ae6d7a2b5c9e59
Author: Andrew Watson <aww@awwsmm.com>
Date:   Sun Jan 2 12:11:54 2022 +0000

    attempting to deploy to Vercel
```

That's all well and good, but I wanted to get this information when building my website, which is written in TypeScript. So I would need a TypeScript or a JavaScript library to be able to do this easily. That's where [`simple-git`](https://www.npmjs.com/package/simple-git) comes in.

### Git in TypeScript

`simple-git` is a JavaScript library which [allows you to run git commands](https://github.com/steveukx/git-js) from within TypeScript / JavaScript code.

For instance, instead of the above `git log blog/hello-world.md`, we can write something like

```ts
import simpleGit from 'simple-git'

const git = simpleGit()
const logResult = await git.log({ file: 'blog/hello-world.md' })
const commits = logResult.all
```

Then, we can loop over all commits and easily access each commit's data

```ts
commits.forEach(commit => {        // example:
  console.log(commit.author_email) // aww@awwsmm.com
  console.log(commit.author_name)  // Andrew Watson
  console.log(commit.body)         //
  console.log(commit.date)         // 2022-01-14T18:54:39+00:00
  console.log(commit.diff)         // null
  console.log(commit.hash)         // bc222b37913ec2cdd7ff5f152dfaca06d33ec9f3
  console.log(commit.message)      // automatically date blog posts using git history
  console.log(commit.refs)         // origin/development, development
})
```

So that should be it: when compiling the project, we should be able to use this library to get all log lines which refer to a specific file. The most recent one should be the `lastUpdated` date of the file, while the least recent one should be the `published` date of the file.

[I quickly coded this up](https://github.com/awwsmm/awwsmm.com/commit/69e038a919e448251fa2211a9fcf3fda914812fe), ran it locally, and everything looked fine. I thought that would be the end of the story.

But then I pushed it to production.

## First Problem

The first problem I hit was that all of my blog posts now had the same date (which at the time was the current date):

![Screenshot of awwsmm.com showing multiple blog posts with the same date](https://i.imgur.com/C0WGWdp.png)

Do you see any problems with [the code I wrote here?](https://github.com/awwsmm/awwsmm.com/commit/69e038a919e448251fa2211a9fcf3fda914812fe)

What's going on here?

Check out the answer in [the second part of this series](https://www.awwsmm.com/blog/whats-wrong-this-time-part-2).

---

> \* This is the traditional phrase for conjuring bugs in your programs. See also: _"I'll just do a little refactoring"_.

What's Wrong This Time? A Debugging Mystery in Three Parts

Nominal, Structural, and Duck Typing in Java, TypeScript, and JavaScript


## Nominal Typing

Most major programming languages with objects use [nominal typing](https://en.wikipedia.org/wiki/Nominal_type_system), where the _name_ (or [fully qualified class name, FQCN](https://en.wikipedia.org/wiki/Fully_qualified_name)) of an object determines whether or not it is equal to another object, or assignable to a variable of a particular type. [For example, in Java](https://www.mycompiler.io/view/BDKXhWg)

```java
class Dog {
  public String name;
  public Dog (String name) {
    this.name = name;
  }
}

class Person {
  public String name;
  public Person (String name) {
    this.name = name;
  }
}
```

If you create a `Dog` and a `Person` with the same `name`, [Java will tell you that they're not the same thing](https://www.mycompiler.io/view/JJSt7fK), even though both classes have the same _structure_ (a single `String` field named `name`) and the same internal state (`name` is `"Fido"`)

```java
Dog dog = new Dog("Fido");
Person person = new Person("Fido");

// System.out.println(dog == person); // error: incomparable types: Dog and Person
System.out.println(dog.equals(person)); // false
System.out.println(person.equals(dog)); // false
```

And [you cannot pass a `Dog` to a method which expects a `Person`](https://www.mycompiler.io/view/88L7lwy)

```java
class Greeter {
    public static void greet (Person person) {
        System.out.println("Hello, " + person.name + "!");
    }
}

// ...

Greeter.greet(person); // Hello, Fido!
// Greeter.greet(dog); // error: incompatible types: Dog cannot be converted to Person
```

## Structural Typing

In contrast, TypeScript allows for _structural typing_ in some scenarios, where the structure of the objects is all that matters (and class names are immaterial). If we had the following two classes in TypeScript, we will see -- similar to Java -- that [they are not equal when compared with `==`](https://www.typescriptlang.org/play?#code/MYGwhgzhAEAiD2BzaBvAUNaA7MBbApgFzQQAuATgJZaIbTDxZnkCuwp850AFDgcc2qIAlKjqZSAC0oQAdH3zQAvNjz4A3HQC+aHWlCQYABXzkIjMZgUCKQugyYU2HLrzU2qNUekwTpchWVVAk1MHT0HMmgAEyQgrHwAdzgkbgAiADFKWLThTUjSaAAHU3MseKToEzNGdKycvLR9RnMQfFkQVNjkJRUSmqw86AB6YegAMzAQCHwgA)

```ts
class Dog {
  name: string
  constructor (name: string) {
    this.name = name;
  }
}

class Person {
  name: string
  constructor (name: string) {
    this.name = name;
  }
}

const dog = new Dog("Fido");
const person = new Person("Fido");

console.log(dog == person); // false
```

But suppose [we write our `Greeter` class in TypeScript as](https://www.typescriptlang.org/play?#code/MYGwhgzhAEAiD2BzaBvAUNaA7MBbApgFzQQAuATgJZaIbTDxZnkCuwp850AFDgcc2qIAlKjqZSAC0oQAdH3zQAvNjz4A3HQC+aHWlCQYABXzkIjMZgUCKQugyYU2HLrzU2qNUekwTpchWVVAk1MHT0HMmgAEyQgrHwAdzgkbgAiADFKWLThTUjSaAAHU3MseKToEzNGdKycvLR9RnMQfFkQVNjkJRUSmqw86AB6YegAMzAQCHwmgyhoAHFyfHxSU0sSUjBSSmBoRBW1nkPV7YAjNuIUYKItz2Qtb3F6Fvg2jtSAAwAJfBBOgAaaAAEhQpzWYEu7QUWgAhF9GmFdE1lmdTLIIaRuP0ykNRtA-gD4MD6vA4Wg0WsMVjuN18WMiUDoGS4UA)

```ts
class Greeter {
  static greet (greetable: { name: string }) {
    console.log(`Hello, ${greetable.name}!`);
  }
}

Greeter.greet(person); // Hello, Fido!
Greeter.greet(dog); // Hello, Fido!
```

TypeScript simply checks that the object passed to `greet()` has a `name: string` field, because that's the type we specified for `greetable`: an object with a `name: string` field. It doesn't care what class `greetable` might be, [or if it has other fields and methods as well](https://www.typescriptlang.org/play?#code/MYGwhgzhAEAiD2BzaBvAUNaA7MBbApgFzQQAuATgJZaIbTDxZnkCuwp850AFDgcc2qIAlKjqZSAC0oQAdH3zQAvNjz4A3HQC+aHWlCQYABXzkIjMZgUCKQugyYU2HLrzU2qNUekwTpchWVVAk1MHT0HMmgAEyQgrHwAdzgkbgAiADFKWLThTUjSaAAHU3MseKToEzNGdKycvLR9RnMQfFkQVNjkJRUSmqw86AB6YegAMzAQCHwmgyhoAHFyfHxSU0sSUjBSSmBoRBW1nkPV7YAjNuIUYKItz2Qtb3F6Fvg2jtSAAwAJfBBOgAaaAAEhQpzWYEu7QUWgAhF9GmFdE1lmdTLIIaRuP0ykNRtA-gD4MD6vA4Wg0WsMVjuN18WMiUDoGSKfpwAsAEKUcjRTYMTrkDx2Kzue4i16OVjsTg8AWcYU0YHWcVeTZ+GSyeVcFTa0K+KSawIqBT68JzFqFc48vkmyrc3npFbRNLAtKc+AATxdjSp63ImKO2OtvIZhP+zI93rhQA)

```ts
class Bird {
  color: string
  name: string
  constructor (color: string, name: string) {
    this.color = color;
    this.name = name;
  }
}

const bird = new Bird("red", "Boyd");
Greeter.greet(bird); // Hello, Boyd!
```

## Duck Typing

In JavaScript, we might [rewrite the above TypeScript classes like](https://jsfiddle.net/y5vts2Lk/)

```js
class Dog {
  constructor (name) {
    this.name = name;
  }
}

class Person {
  constructor (name) {
    this.name = name;
  }
}

class Bird {
  constructor (color, name) {
    this.color = color;
    this.name = name;
  }
}

const dog = new Dog("Fido");
const person = new Person("Fido");
const bird = new Bird("red", "Boyd");
```

But, as JavaScript doesn't specify types using the `:` as TypeScript does, we also have to [rewrite our `Greeter` slightly](https://jsfiddle.net/y5vts2Lk/1/)

```js
class Greeter {
  static greet (greetable) {
    console.log("Hello, " + greetable.name + "!");
  }
}

Greeter.greet(person); // Hello, Fido!
Greeter.greet(dog); // Hello, Fido!
Greeter.greet(bird); // Hello, Boyd!
```

In this case, there are no type or structural constraints at all on `greetable`. JavaScript is using [duck typing](https://en.wikipedia.org/wiki/Duck_typing) here, where field and method accesses are only checked at runtime (and not at compile time, because JavaScript isn't compiled). If a `greetable` has all of the required fields, no errors will be thrown.

However, [if a field is missing...](https://jsfiddle.net/bjgz7cuo/)

```js
class Complimenter {
  static compliment (target) {
    console.log("Hello, " + target.name + "!");
    console.log("What a nice shade of " + target.color + " you are!");
  }
}

Complimenter.compliment(person); // Hello, Fido! What a nice shade of undefined you are!
Complimenter.compliment(dog); // Hello, Fido! What a nice shade of undefined you are!
Complimenter.compliment(bird); // Hello, Boyd! What a nice shade of red you are!
```

...we can get `undefined` results.

> The name "duck typing" comes from the phrase ["If it looks like a duck, swims like a duck, and quacks like a duck, then it probably is a duck"](https://en.wikipedia.org/wiki/Duck_test). When using duck typing, we run the program as long as we can until we hit a runtime error -- until the "duck" no longer looks like a duck.

What's the Difference Between Nominal, Structural, and Duck Typing?

Writing Git Commit Hooks in Bash, Java, and Scala


## Writing a Git Hook in Bash

[Git Hooks](https://www.atlassian.com/git/tutorials/git-hooks) can be a very useful addition to a developer's workflow.

For instance, as I'm working on [rewriting my new website using Next.js](https://www.awwsmm.com/blog/hello-world), I've set up [eslint](https://eslint.org) and have pretty strict linting rules when running Next builds.

Unfortunately, Git doesn't follow these rules by default. So it's easy to forget to run the linter, run a `git push`, and end up with a failing production build on [Vercel](https://vercel.com).

To avoid this, I've set up a simple Git Hook which runs `eslint` before every `git push`, and fails the `push` if the linter returns any `error`s:

```sh
# .git/hooks/pre-push

# pipe output (with colors) to .temp file
RESULT=$(script -q .temp npm run lint)

# if .temp file contains "error", fail
if grep --quiet error .temp; then
  printf "\n  !! Will not push -- found linting errors:\n"
  sed '1,5d' .temp | cat
  rm .temp
  exit 1
else
  rm .temp
  exit 0
fi
```

The above script runs `npm run lint`, captures the output ([with text colors](https://stackoverflow.com/a/3515296/2925434)) to a `.temp` file, and then `grep`s for `error` in that file. If it finds any instances of the word "error", it won't `push`, but will instead print the output of the `npm run lint` command:

```sh
$ git push

  !! Will not push -- found linting errors:

/Users/andrew/Git/awwsmm.com/components/CommitGroupComponent.tsx
   2:1  error  Imports should be sorted alphabetically  sort-imports
  24:4  error  Missing semicolon                        semi

✖ 2 problems (2 errors, 0 warnings)
  1 error and 0 warnings potentially fixable with the `--fix` option.

error: failed to push some refs to 'github.com:awwsmm/awwsmm.com.git'
```

This prevents useless builds on Vercel which will fail because of linting errors.

**Note:** make sure your `.git/hooks/<file>` is executable (`chmod +x .git/hooks/<file>`), or it will not prevent the particular action from occurring. It's probably best to play around with a `pre-commit` hook rather than a `pre-push` one, when testing.

## Writing a Git Hook in Java

We can do a similar thing using Java instead of bash scripting. For this example, we'll look at writing a `pre-commit` hook in Java. Instead of writing Java code to the `pre-commit` file directly, we'll add a bit of indirection.

First, create a file called `PreCommit.java` in the `.git/hooks/` directory and add some pre-commit logic to it. In this case, we're just going to fail every commit:

```java
public class PreCommit {
  public static void main (String[] args) {
    System.out.println("Java says: no committing please.");
    System.exit(1);
  }
}
```

The `System.exit(1)` gives the same result as `exit 1` in the Bash script -- the commit will fail to complete.

Next, we'll make the `.git/hooks/pre-commit` file executable with

```sh
$ chmod +x .git/hooks/pre-commit
```

...and modify its contents to read:

```sh
#!/bin/sh

# get the path to this script file
DIR=$(dirname "$0")

# Java >= 11
exec java $DIR/PreCommit.java "$@"
```

> If you're using Java 11+, you can also write this [all as a single file](https://stackoverflow.com/a/61899256/2925434), though I personally don't recommend this method as it combines Bash and Java syntax in a single file, which can be difficult for IDEs to handle in terms of syntax highlighting, etc.

With this in place, we'll now see a message like

```sh
$ git commit -m "update gitignore, more hook testing"
Java says: no committing please.
```

Note that in Java versions &lt; 11, we need to compile this script first with `javac`, so `pre-commit` should instead look something like

```sh
#!/bin/sh
DIR=$(dirname "$0")

# Java < 11
javac $DIR/PreCommit.java "$@"
exec java -classpath $DIR PreCommit
!#
```

## Writing a Git Hook in Scala

The process for [writing a Git Hook in Scala](https://alvinalexander.com/scala/scala-shell-script-example-exec-syntax/) is very similar to writing one in Java. First, define a `pre-commit.scala` file with the following contents

```scala
object Hook {
  def main (args: Array[String]): Unit = {
    System.out.println("Scala says: no committing please.");
    System.exit(1);
  }
}
```

(Note that, in Scala, the name of the class doesn't have to match the name of the file.)

Then, modify the contents of `pre-commit` to read

```sh
#!/bin/sh
DIR=$(dirname "$0")
exec scala $DIR/pre-commit.scala "$@"
!#
```

Finally, try to commit:

```sh
$ git commit -m "update gitignore, more hook testing"
Scala says: no committing please.
```

That's all there is to it.

As it takes a while for the JVM to boot up when running Git Hooks in Java or Scala, it's probably not extremely practical to do so... unless you have a lot of logic in your hook which is easier to read in Java / Scala than in a Bash script.

But anyway, it's good to know that it can be done.

Eliminate Unnecessary Builds With Git Hooks

Rewriting My Personal Website in Next.js and TypeScript


New Year's Day is as good a time as any to roll out a new blog, I suppose.

As one of my New Year's Resolutions this year, I'm going to revamp my website, and I'm going to do it in public.

I've spent the last few weeks working with Next.js for the first time (and brushing up on my TypeScript) in order to start hosting my technical blog on my own website. (In addition to mirroring the posts [on Dev.to](https://dev.to/awwsmm), where I've been writing for a few years now.)

In addition to the ol' blog, I've also created **/project** pages for a few ongoing things I've got in the works (including this very website). The project pages synthesize groups of commits from GitHub with manual "update" log posts to give an idea as to the current status of each of my ongoing projects. I think this is a bit more dynamic than a static "portfolio" page, and it reflects a tenet of my own personal philosophy that [_un ouvrage n’est jamais achevé... mais abandonné_](https://quoteinvestigator.com/2019/03/01/abandon/).

Follow along here, [on Dev.to](https://dev.to/awwsmm), or [on Twitter](https://twitter.com/awwsmm_dot_com) for continuing updates.

Thanks for reading!

Hello, World!


Boy have I learned a lot over the past few days.

As you can see from [the recent round of commits I've made](https://github.com/awwsmm/awwsmm.com/commits/master), I have been working on a few things...

### Requiring `Commit`s to have a defined `date`

[This commit](https://github.com/awwsmm/awwsmm.com/commit/2b7c742d264a5571efb76c10b88fe8cc71d742e1) moves from a `Commit` with a `date: string | undefined` field to one with a `date: string` field.

Originally, I tried to make my model class (`Commit`) match the GitHub API as closely as possible, and since [`GET /repos/{owner}/{repo}/commits`](https://docs.github.com/en/rest/reference/commits#list-commits) can return commit objects with possibly `null` `author`s as well as `committer`s, it's possible for a commit returned from the GitHub API to not have a date at all.

That... seems unlikely, so I now assume that all commits will indeed have dates, and throw an `Error` in the `Commit` `constructor` if `date` is `undefined`. This can be tested during development, and (combined with the next change) means that I've removed all explicit `undefined` types from the current codebase.

### Require `CommitGroup`s to have at least one `Commit`

[This commit](https://github.com/awwsmm/awwsmm.com/commit/cb8eb7b1b2c902a4d66cd454152e3dc3641fb4ff) changes a lot of things. Firstly, it requires that `CommitGroup`s have at least one `Commit` in them. It does this by throwing an `Error` from the `constructor` if an attempt is made to create a `CommitGroup` with an empty array of `Commit`s.

This is really helpful because if a `CommitGroup` must have at least one `Commit` and (from the last update) if a `Commit` must have a definite `date`, then `CommitGroup`s can have definite `start()` and `end()` timestamps. This means that `start()` and `end()` can have type `Date` rather than `Date | undefined`. That change removes the last explicit `undefined` types from the codebase. This means we shouldn't have to do much more "`?.`" / "`??`" checking.

### Ease serialisation by moving from methods to fields

The reason I haven't updated this project in a few days is because I tried to do too much all at once (I fall into that rabbit hole sometimes). One of the things I was trying to do was to return `class`es (rather than plain objects) from the `getStaticProps` methods in my dynamic route definition files (`/pages/blog/[slug].md` and `/pages/projects/[id].md`).

I ran into some issues here (specifically around defining custom `toJSON()` methods) and everything went a bit haywire so I've backed that out of this group of commits for now.

Anyway, the point is: serialisation is confusing enough as-is without including a bunch of methods and custom `class`es. So I'm trying to store as much data in object fields, rather than in methods which calculate some value. To that end, I moved all of the methods in `ProjectUpdate` to fields in these updates, which means that `LogEntry` and `CommitGroup` also have these as fields now instead of methods (`start` instead of `start()` for example).

This made it easier to make the next change, as well.

### Make `CommitGroup` class immutable

As `start` and `end` are now fields of `CommitGroup` rather than methods, and as `Commit`s are themselves immutable (containing only `readonly` fields), it made sense to try to make `CommitGroup`s immutable as well.

> In general, immutability of objects is something worth working toward. It makes the state of an object easier to reason about and can eliminate race conditions in multithreaded environments (so, not JavaScript, really). However, immutability can make your program less performant, as you have to create an entirely new object whenever you want to update a field.

This was easy enough -- just making the array of `commits` `readonly`, and making `add()` return a new `CommitGroup` rather than mutating the current one makes the class effectively immutable. We just need to remember to save the result of `add()` to a new variable when updating a `CommitGroup`.

### Try not to hit the GitHub rate limit

Finally, I made one small band-aid kind of change, where I don't query GitHub for commits when in development mode. This is easy enough to check in Next.js

```js
if (process.env.NODE_ENV === "development") {
  // do development stuff
} else {
  // do other stuff
}
```

so I simply return an empty array of `Commit`s when in development mode. A better solution here would be to

- query GitHub for commits
- save those commits to a temporary cache file
- don't query a second time if a cache file is present
- delete the cache file on a `git push`

This would give a better idea of what the site will look like when it goes live, but it's a bit more involved, so I'll do that later. For now, I just return an empty array.

### Potpourri

Finally, in [this commit](https://github.com/awwsmm/awwsmm.com/commit/e31fdc8274f2e131fd3f8b0ec508f90930b3144d) I did some refactoring, moving some classes around in a way that seemed to be a bit more organised.

And in the last few commits this round, I hit a bug where I was trying to use `String.replaceAll`, but I was targeting an older ES version which doesn't support that method.

So, all in all, lots done here, but lots more to do.

Learning about TypeScript, Next.js, the GitHub API, and more

The Learning Process

awwsmm.com

ConwayScalaJS


This is an example log entry.

It's just a placeholder until other "update" log entries are available.

I'm using it to test the rendering and ordering of these updates, which are written in _markdown_, just like the blog posts.

Here is some **other markdown stuff** so I can see how it's [rendered](https://www.google.com)
- some stuff
- some more stuff
- additional stuff

## A Level-2 Heading

Some content

# A Level-1 Heading

Some more content

> And this is a blockquote

```
// And this is a code block
const answer = 42;
```

A subtitle with gives some additional info

Andrew Watson

Blog

Projects