Understanding const generics in the Rust programming language with a few examples


[I was working through an example](https://github.com/awwsmm/daily-bevy/tree/time/virtual_time) in [the repo for the Bevy game engine](https://github.com/bevyengine/bevy/blob/v0.13.0/examples/time/virtual_time.rs) recently and came across this code

```rust
/// Update the speed of `Time<Virtual>.` by `DELTA`
fn change_time_speed<const DELTA: i8>(mut time: ResMut<Time<Virtual>>) {
    let time_speed = (time.relative_speed() + DELTA as f32)
        .round()
        .clamp(0.25, 5.);

    // set the speed of the virtual time to speed it up or slow it down
    time.set_relative_speed(time_speed);
}
```

This is a function (`fn`) which takes a `mut`able argument called `time`. The type of `time`, `ResMut<Time<Virtual>>`, comes after the colon, `:`.

The thing that caught my eye here was the generic parameter: `<const DELTA: i8>`. What is that?

Here's another example from Bevy

```rust
pub unsafe fn read<T>(self) -> T {
    let ptr = self.as_ptr().cast::<T>().debug_ensure_aligned();
    // -- snip --
}
```

The `read` function takes a generic type parameter `T` and uses it in two places: in the body of the function, and as a return type. Programmers who are familiar with generics know that an unconstrained `T` is a placeholder that means "any type"; it could be `String` or `bool` or anything else.

> In languages with a global type hierarchy, like Java, a value `t: T` has _some_ operations which can be performed on it, like `.toString()`, because _every_ type `T` in Java extends the base `Object` type. Rust has no such global type hierarchy, and no root `Object` type, so there's not much at all you can do with an instance of an unconstrained type.

Going back to the first example, `const DELTA: i8` clearly already _has_ a type, appearing after the colon, `:`. (It is `i8`, an 8-bit signed integer.) So what is it doing sitting between those angle brackets (`<>`) where generic parameters usually sit?

In this position, `const DELTA: i8` is acting as a [const generic](https://doc.rust-lang.org/reference/items/generics.html#const-generics).

## What Are Const Generics?

Const generic parameters are a new ([ish](https://blog.rust-lang.org/2021/02/26/const-generics-mvp-beta.html)) kind of generic parameter in Rust, similar to type parameters (e.g. `T`) and lifetime parameters (e.g. `'a`). In the same way that a function (or method, `struct`, `enum`, `impl`, `trait`, or `type` alias) can use a generic _type_ parameter, it can also use _const generic_ parameters.

Const generic parameters are what power `[T; N]` type annotation of arrays in Rust. They are why `[T; 3]` (an array of three `T` values) and `[T; 4]` (an array of four `T` values) are _different types_, but different types which can be handled _generically_ as specific implementations of `[T; N]`.

Const generic parameters allow items to be generic over _constant values_, rather than over types.

The difference can be subtle. Here's a simple example

```rust
fn add<const b: i8>(a: i8) -> i8 {
  a + b
}
```

Here, `b` is not a "type parameter"; it is a _value_, and so it can be treated exactly as a value, used in expressions, and so on. But since it is `const`, the value of `b` must be known at compile time. For example, the following will not compile

```rust
fn example(a: i8, b: i8) -> i8 {
    add::<b>(a) // error: attempt to use a non-constant value in a constant
}
```

The logic in this function, of course, could also be expressed like

```rust
fn add(a: i8, b: i8) -> i8 {
  a + b
}
```

...so what's the benefit of const generics? Let's look at some other examples

## Using Const Generics to Enforce Correctness

There are a few examples from linear algebra where const generics are very helpful. For example, the [dot product](https://en.wikipedia.org/wiki/Dot_product) of two vectors `a` and `b`, is defined for any two vectors of any dimensionality (length), provided they have the _same_ dimensionality

```rust
struct Vector<const N: usize>([i32; N]);

impl<const N: usize> Vector<N> {
    fn dot(&self, other: &Vector<N>) -> i32 {
        let mut result = 0;
        for index in 0..N {
            result += self.0[index] * other.0[index]
        }
        result
    }
}
```

We get a compile-time error if we try to find the dot product of two vectors with different numbers of elements

```rust
fn main() {
    let a = Vector([1, 2, 3]);
    let b = Vector([4, 5, 6]);
    
    assert_eq!(a.dot(&b), 32); // ok: a and b have the same length
    
    let c = Vector([7, 8, 9, 10]);
    
    a.dot(&c); // error: expected `&Vector<3>`, but found `&Vector<4>`
}
```

Const generics can be applied to [matrix multiplication](https://amacal.medium.com/learning-rust-const-generics-8c29fc26fad4), as well. Two matrices [can be multiplied](https://en.wikipedia.org/wiki/Matrix_multiplication) only if the first one has `M` rows and `N` columns and the second has `N` rows and `P` columns. The resulting matrix will have `M` rows and `P` columns.

```rust
struct Matrix<const nRows: usize, const nCols: usize>([[i32; nCols]; nRows]);

impl<const M: usize, const N: usize> Matrix<M, N> {
    fn multiply<const P: usize>(&self, other: &Matrix<N, P>) -> Matrix<M, P> {
        todo!()
    }
}
```

Here, we again get a compile-time error if we ignore this constraint

```rust
fn main() {
    let a = Matrix([[1, 2, 3], [4, 5, 6]]); // 2 x 3 matrix
    let b = Matrix([[1, 2, 3, 4], [2, 3, 4, 5], [3, 4, 5, 6]]); // 3 x 4 matrix
    
    a.multiply(&b); // ok: 2 x 4 matrix
    
    let c = Matrix([[1, 2, 3], [2, 3, 4]]); // 2 x 3 matrix
    
    a.multiply(&c); // error: expected `&Matrix<3, <unknown>>`, but found `&Matrix<2, 3>`
}
```

These constraints can be enforced at runtime without const generics, but const generics can help [shift these issues left](https://en.wikipedia.org/wiki/Shift-left_testing), catching them earlier in the development process, tightening [the inner dev loop](https://www.getambassador.io/docs/telepresence/latest/concepts/devloop#what-is-the-inner-dev-loop).

## Using Const Generics to Conditionally Implement `trait`s

(Adapted from [Nora's example here](https://nora.codes/post/its-time-to-get-hyped-about-const-generics-in-rust/).)

Const generics also enable really powerful patterns, like compile-type checks on values in signatures. For example...

```rust
struct Assert<const COND: bool> {}
```

...this `struct` takes a constant generic `bool` parameter, `COND`. If we define a `trait` `IsTrue`...

```rust
trait IsTrue {}

impl IsTrue for Assert<true> {}
```

...we can conditionally implement `trait`s by requiring some `Assert` to `impl IsTrue`, like so

```rust
trait IsOdd<const N: i32> {}

impl<const N: i32> IsOdd<N> for i32 where Assert<{N % 2 == 1}>: IsTrue {}
```

> The above `Assert<{N % 2 == 1}>` requires <span style="white-space: nowrap;">`#![feature(generic_const_exprs)]`</span> and the `nightly` toolchain. See https://github.com/rust-lang/rust/issues/76560 for more info.

Above, `trait IsOdd` is implemented for the `i32` type, but only on values `N` which satisfy `N % 2 == 1`. We can use this trait to get compile-time checks that constant (hard-coded) `i32` values are odd

```rust
fn do_something_odd<const N: i32>() where i32: IsOdd<N> {
    println!("oogabooga!")
}

fn do_something() {
    do_something_odd::<19>();
    do_something_odd::<42>(); // does not compile
    do_something_odd::<7>();
    do_something_odd::<64>(); // does not compile
    do_something_odd::<8>(); // does not compile
}
```

The above will generate a compiler error like

```rust
error[E0308]: mismatched types
  --> src/main.rs:70:5
   |
70 |     do_something_odd::<42>();
   |     ^^^^^^^^^^^^^^^^^^^^^^^^ expected `false`, found `true`
   |
   = note: expected constant `false`
              found constant `true`
```

## Using Const Generics to Avoid Complex Return Types

Finally, const generics can be used to make code more readable, and more performant. The example from the beginning of this post comes from Bevy, and the reason const generics are used there is because Bevy is expecting a function pointer as an argument to a method

```rust
fn main() {
    App::new()
        // -- snip --
        .add_systems(
            Update,
            (
                // -- snip --
                change_time_speed::<1>.run_if(input_just_pressed(KeyCode::ArrowUp)),
                // -- snip --
            ),
        )
        .run();
}
```

`change_time_speed::<1>`, above, is a function pointer. We can rearrange this method to take an argument, rather than using a const generic parameter...

```rust
change_time_speed_2(1).run_if(input_just_pressed(KeyCode::ArrowUp)),
```

...but then we would have to change the return type as well

```rust
/// Update the speed of `Time<Virtual>.` by `DELTA`
fn change_time_speed_2(delta: i8) -> impl FnMut(ResMut<Time<Virtual>>) {
    move |mut time| {
        let time_speed = (time.relative_speed() + delta as f32)
            .round()
            .clamp(0.25, 5.);

        // set the speed of the virtual time to speed it up or slow it down
        time.set_relative_speed(time_speed);
    }
}
```

To many, the original function may be more readable

```rust
/// Update the speed of `Time<Virtual>.` by `DELTA`
fn change_time_speed<const DELTA: i8>(mut time: ResMut<Time<Virtual>>) {
    let time_speed = (time.relative_speed() + DELTA as f32)
        .round()
        .clamp(0.25, 5.);

    // set the speed of the virtual time to speed it up or slow it down
    time.set_relative_speed(time_speed);
}
```

Remember, as well, that Rust uses [monomorphization of generics](https://doc.rust-lang.org/book/ch10-01-syntax.html#performance-of-code-using-generics) to improve runtime performance. So not only is the const generic version of this function more readable, but it's possible (though I haven't benchmarked) that it's more performant as well. Either way, it's good to know that there are multiple ways to attack a problem, and to be able to weigh the pros and cons of each approach.

Hopefully this discussion has helped you to understand what const generics are, and how they can be used in Rust.

What Are Const Generics and How Are They Used in Rust?

Use types and let the compiler do the hard work of data validation for you


[[edit history]](https://github.com/awwsmm/awwsmm.com/commits/master/blog/make-invalid-states-unrepresentable.md)

Suppose you have a `Person` class in your program, and that a `Person` has an `age`. What type should the `age` be?

## `age` as a `String`

```scala
case class Person(age: String)
```

"Of course it shouldn't be a `String`" you might think. But why? The reason is that we can then end up with code like

```scala
val person = Person("Jeff")
```

If we ever wanted to _do_ anything with an `age: String`, we would need to validate it everywhere

```scala
def isOldEnoughToSmoke(person: Person): Boolean = {
  Try(person.age.toInt) match {
    case Failure(_) => throw new Exception(s"cannot parse age '${person.age}' as numeric")
    case Success(value) => value >= 18
  }
}

def isOldEnoughToDrink(person: Person): Boolean = {
  Try(person.age.toInt) match {
    case Failure(_) => throw new Exception(s"cannot parse age '${person.age}' as numeric")
    case Success(value) => value >= 21
  }
}

// etc.
```

This is cumbersome for the programmer _writing_ the code, and makes it difficult for any programmer _reading_ the code, as well.

We could move this validation to a separate method

```scala
def parseAge(age: String): Int = {
  Try(age.toInt) match {
    case Failure(_) => throw new Exception(s"cannot parse age '$age' as numeric")
    case Success(value) => value
  }
}

def isOldEnoughToSmoke(person: Person): Boolean =
  parseAge(person.age) >= 18

def isOldEnoughToDrink(person: Person): Boolean =
  parseAge(person.age) >= 21
```

...but this is still not ideal. The code is a bit cleaner, but we still need to parse a `String` into an `Int` every time we want to do anything numeric (comparison, arithmetic, etc.) with the `age`. This is often called ["stringly-typed" data](https://en.wiktionary.org/wiki/stringly-typed).

This can also move the program into an illegal state by throwing an `Exception`. If we're going to fail anyway, we should [fail fast](https://www.martinfowler.com/ieeeSoftware/failFast.pdf). We can do better.

## `age` as an `Int`

Your first instinct might have been to make `age` an `Int`, rather than a `String`

```scala
case class Person(age: Int)
```

If so, you have good instincts. An `age: Int` is much nicer to work with

```scala
def isOldEnoughToSmoke(person: Person): Boolean =
  person.age >= 18

def isOldEnoughToDrink(person: Person): Boolean =
  person.age >= 21
```

This 
- is easier to write
- is easier to read
- fails fast

You _cannot construct_ an instance of the  `Person` class with a `String` `age` now. That is an invalid state. We have [made it unrepresentable](https://www.cs.rice.edu/~javaplt/411/23-spring/NewReadings/functional_programming_on_Wall_Street.pdf), using the type system. The compiler will not allow this program to compile.

Problem solved, right?

```scala
val person = Person(-1)
```

This is clearly an invalid state as well. A person cannot have a negative age.

```scala
val person = Person(90210)
```

This is also invalid -- it looks like someone accidentally entered their [ZIP code](https://en.wikipedia.org/wiki/Beverly_Hills,_California) instead of their age.

So how can we constrain this type even further? How can we make even more invalid states unrepresentable?

## `age` as an `Int` with constraints

### at runtime

We can enforce _runtime constraints_ in any statically-typed language

```scala
case class Person(age: Int) {
  assert(age >= 0 && age < 150)
}
```

In Scala, `assert` will throw a `java.lang.AssertionError` if the assertion fails.

Now we can be sure that the `age` for any `Person` will always be within the range `[0, 150)`. Both

```scala
val person = Person(-1)
```

and

```scala
val person = Person(90210)
```

will now fail. But they will fail at runtime, halting the execution of our program.

This is similar to what we saw in "`age` as a `String`", above. This is still not ideal. Is there a better way?

### at compile time

Many languages allow _compile-time_ constraints, as well. Usually this is accomplished through macros, which inspect the source code during a compilation phase. These are often referred to as [refined types](https://en.wikipedia.org/wiki/Refinement_type).

Scala has [quite good](https://blog.rockthejvm.com/refined-types/) support for [refined types](https://github.com/Iltotore/iron) across multiple libraries. A solution using the [`refined` library](https://github.com/fthomas/refined) might look something like

```scala
case class Person(age: Int Refined GreaterEqual[0] And Less[150])
```

A limitation of this approach is that the field(s) to be constrained at compile-time must be [_literal_, hard-coded](https://en.wikipedia.org/wiki/Literal_(computer_programming)) values. Compile-time constraints cannot be enforced on, for example, values provided by a user. By that point, the program has already been compiled. In this case, we can always fall back to runtime constraints, which is often what these libraries do.

For now, we'll continue with runtime constraints only, since often that's the best we can do.

## `age` as an `Age` with constraints

From simplest to most complex implementation, we moved left to right in the diagram below

```
String => Int => Int with constraints
```

This increase in complexity directly correlates with the _accuracy_ with which we're modelling this data

> "The problems tackled have inherent complexity, and it takes some effort to model them appropriately." [[source]](https://fasterthanli.me/articles/i-want-off-mr-golangs-wild-ride)

The move left-to-right above should be driven by the requirements of your system. You should not implement compile-time refinements, for example, unless you have lots of hard-coded values to validate at compile time: otherwise [you aren't gonna need it](https://martinfowler.com/bliki/Yagni.html). Every line of code has a cost to implement and maintain. Avoiding premature specification is just as important as avoiding [premature generalization](https://www.codewithjason.com/premature-generalization), though it's **_always easier_ to move from more specific types to less specific types**, so prefer specificity over generalization.

Every bit of data has a _context_, as well. There is no such thing as a "pure" `Int` value floating around in the universe. An age can be modelled as an `Int`, but it's different from a weight, which could also be modelled as an Int. The labels we attach to these raw values are the _context_

```scala
case class Person(age: Int, weight: Int) {
  assert(age >= 0 && age < 150)
  assert(weight >= 0 && weight < 500)
}
```

There is one more problem for us to solve here. Suppose I'm 81kg and 33 years old

```scala
val me = Person(81, 33)
```

That compiles, but... it shouldn't. I swapped my weight and age!

An easy way to avoid this confusion is to define some more types. In this case, [_newtypes_](https://doc.rust-lang.org/rust-by-example/generics/new_types.html)

```scala
case class Age(years: Int) {
  assert(years >= 0 && years < 150)
}

case class Weight(kgs: Int) {
  assert(kgs >= 0 && kgs < 500)
}

case class Person(age: Age, weight: Weight)
```

The name _newtype_ for this pattern [comes from Haskell](https://wiki.haskell.org/Newtype). This is a simple way to ensure that we don't accidentally swap values with the same underlying type. The following, for example, will not compile

```scala
val age = Age(33)
val weight = Weight(81)

val me = Person(weight, age) // does not compile!
```

We could also use [_tagged types_](https://medium.com/iterators/to-tag-a-type-88dc344bb66c). In Scala, the simplest possible example of this looks something like

```scala
trait AgeTag
type Age = Int with AgeTag

object Age {
  def apply(years: Int): Age = {
    assert(years >= 0 && years < 150)
    years.asInstanceOf[Age]
  }
}

trait WeightTag
type Weight = Int with WeightTag

object Weight {
  def apply(kgs: Int): Weight = {
    assert(kgs >= 0 && kgs < 500)
    kgs.asInstanceOf[Weight]
  }
}

case class Person(age: Age, weight: Weight)

val p0 = Person(42, 42) // does not compile -- an Int is not an Age
val p1 = Person(Age(42), 42) // does not compile -- an Int is not a Weight	
val p2 = Person(Age(42), Weight(42)) // compiles!
val p3 = Person(Weight(42), Weight(42)) // does not compile -- a Weight is not an Age
```

This makes use of the fact that function application `f()` is syntactic sugar in Scala for an `apply()` method. So `f()` is equivalent to `f.apply()`.

This approach allows us to model the idea that an `Age` / a `Weight` is an `Int`, but an `Int` is not an `Age` / a `Weight`. This means we can treat an `Age` / a `Weight` _as_ an `Int` and add, subtract, or do whatever other `Int`-like things we want to do.

Mixing these two approaches in one example, you can see the difference between newtypes and tagged types. You must extract the "raw value" from a newtype. You do not need to do this with a tagged type

```scala
// `Age` is a tagged type
trait AgeTag
type Age = Int with AgeTag

object Age {
  def apply(years: Int): Age = {
    assert(years >= 0 && years < 150)
    years.asInstanceOf[Age]
  }
}

// `Weight` is a newtype
case class Weight(kgs: Int) {
  assert(kgs >= 0 && kgs < 500)
}

// `Age`s can be treated as `Int`s, because they _are_ `Int`s
assert(40 == Age(10) + Age(30))

// `Weight`s are not `Int`s, they _contain_ `Int`s
Weight(10) + Weight(30) // does not compile

// To add `Weight`s, we must "unwrap" them
Weight(10).kgs + Weight(30).kgs
```

In some languages, the "unwrapping" of newtypes can be done automatically. This can make newtypes as ergonomic as tagged types. For example, in Scala, this could be done with an _implicit conversion_

```scala
implicit def weightAsInt(weight: Weight): Int = weight.kgs

// `Weight`s are not `Int`s, but they can be _converted_ to `Int`s
Weight(10) + Weight(30) // this now compiles
```

## Further refinements

The important point of the above discussion is that, as much as possible, we want to _make invalid states unrepresentable_.

- "Jeff" is an invalid age. Age isn't a string, it is a number.
- -1 is an invalid age. Age cannot be negative, it should be 0 or positive, and probably less than about 150.
- My age is not 88. An age should be easily distinguishable from other integral values, like weight.

Everything discussed above implemented these refinements on the concept of "age", one at a time.

We can make further refinements if there is a need for those refinements.

For example, suppose we want to send a "Happy Birthday!" email to a `Person` on their birthday. Rather than an `Age`, we now need a date of birth.

```scala
case class Date(year: Year, month: Month, day: Day)

case class Year(value: Int, currentYear: Int) {
  assert(value >= 1900 && value <= currentYear)
}

case class Month(value: Int) {
  assert(value >= 1 && value <= 12)
}

case class Day(value: Int) {
  assert(value >= 1 && value <= 31)
}

case class Person(dateOfBirth: Date, weight: Weight) {
  def age(currentDate: Date): Age = {
    ??? // TODO calculate Age from dateOfBirth
  }
}
```

The amount of information provided by `dateOfBirth` is strictly greater than the amount of information provided by `Age`. We can calculate someone's age from their date of birth, but we cannot do the opposite.

The above implementation leaves much to be desired, though -- there are lots of invalid states. A better way to implement this would be for `Month` to be an enum, and for `Day` validity to depend on the `Month` (February never has 30 days, for example)

```scala
case class Year(value: Int, currentYear: Int) {
  assert(value >= 1900 && value <= currentYear)
}

sealed trait Month

case object January extends Month
case object February extends Month
case object March extends Month
case object April extends Month
case object May extends Month
case object June extends Month
case object July extends Month
case object August extends Month
case object September extends Month
case object October extends Month
case object November extends Month
case object December extends Month

case class Day(value: Int, month: Month) {
  month match {
    case February => assert(value >= 1 && value <= 28)
    case April | June | September | November => assert(value >= 1 && value <= 30)
    case _ => assert(value >= 1 && value <= 31)
  }
}

case class Date(year: Year, month: Month, day: Day)
```

Always prefer low-cardinality types to [high-cardinality types](https://www.seas.upenn.edu/~sweirich/types/archive/1999-2003/msg01233.html), when possible. It limits the number of possible invalid states. In most languages, `enum`s are the way to go here (in Scala 2, an `enum` can be modelled using a `sealed trait`, as shown above). But there are still invalid states hiding above. Can you find them?

In some cases, stringly-typed data validated using regular expressions can be replaced entirely by `enum`s. Could you model [Canadian postal codes](https://www150.statcan.gc.ca/n1/pub/92-153-g/2011002/tech-eng.htm) such that it's impossible to construct an invalid one?

Use the above knowledge to go forth and _make invalid states unrepresentable_.

Make Invalid States Unrepresentable

Building a Rust IoT MVP over the Holidays

What This Senior Developer Learned From His First Big Rust Project

Quickly and Easily Publish Your Scala Code for Use in Other Projects


## Intro

Say what you will about JavaScript, but [npm](https://www.npmjs.com/) has made it incredibly easy for anyone to publish code. In just a few clicks, you can share a library to make others' lives easier, or publish a little snippet like [`left-pad`](https://qz.com/646467/how-one-programmer-broke-the-internet-by-deleting-a-tiny-piece-of-code/) to cause endless headaches.

Java has long had the [Maven repository](https://mvnrepository.com/) (and accompanying build tool of the same name), which serves a similar purpose: sharing packaged code.

But it can be intimidating for a newbie to publish to Maven -- that's where Sonatype comes in.

Sonatype, Inc.'s [Central Repository](https://central.sonatype.org/) (a complement to Apache's Maven repository) makes it super easy to share code. With just a few lines and a few manual steps, you can publish your package to Maven, and others can easily import it and use it.

sbt's [Using Sonatype](https://www.scala-sbt.org/release/docs/Using-Sonatype.html) guide and Sonatype's [Deployment Guide](https://central.sonatype.org/publish/publish-guide/) cover probably 95% of what you need to know for this process, but there are a few _gotchas_ that I'll highlight below using the ❗ exclamation point emoji.

> Note: this guide assumes that your source code is hosted in a GitHub or GitLab repository.

## Steps

### 1. Create a Sonatype Jira issue

Sonatype syncs with Maven, and so to publish to Maven, you can publish via Sonatype. To do so, you'll need a Sonatype Jira account to create an issue. Do that by [clicking here to create an account](https://issues.sonatype.org/secure/Signup!default.jspa) and then [clicking here to create a Jira issue](https://issues.sonatype.org/secure/CreateIssue.jspa?issuetype=21&pid=10134).

As an example, [here is the ticket I opened](https://issues.sonatype.org/browse/OSSRH-71079) to publish a project of mine to Maven. If you want to publish multiple packages to Maven using Sonatype, you'll need to create multiple Jira tickets, but you only need the one user account.

The fields in the Jira are easy to fill out:

#### Summary

This should include the name of the project. In my case, I'm publishing a small project hosted at https://github.com/awwsmm/zepto, so I've included the name "zepto" in the Jira summary.

#### Group Id

If your project is hosted on GitHub, this should be something like `io.github.awwsmm`, or similar for your GitHub username. See [this guide](https://central.sonatype.org/publish/requirements/coordinates/) for more information on package coordinates.

> ❗ **GOTCHA:** this should _not_ be something like `com.github.awwsmm`. `com.github` coordinates are no longer supported. It _can_ be your personal website, like `com.awwsmm`, but [a bot will comment on your Jira asking you to add a DNS record to prove that you own that website, like here on another one of my Jiras](https://issues.sonatype.org/browse/OSSRH-95101).

#### Project URL

This is where users can go on the web to find information about your project. For me, that's `https://github.com/awwsmm/zepto`, but it could be a `github.io` page or a page on your personal website, or whatever.

#### SCM url

This is the URL that your source code is hosted at. For me, that's `https://github.com/awwsmm/zepto.git`.

### 2. Generate and distribute a PGP key

#### On macOS

On macOS, I recommend that you use https://gpgtools.org/ to create and distribute your PGP key. Simply download the installer and follow the instructions there.

To run `sbt publishSigned` later in this guide, you will also need `gpg2` and `pinentry` installed. You can get those on macOS via `brew` with

```bash
$ brew install gpg2
$ brew install pinentry-mac
```

Note that you need `gpg2` here, not `gpg`. You should [check if you have both installed and `brew uninstall gpg` if it exists](https://stackoverflow.com/a/45932796/2925434)

```bash
$ which gpg gpg2
```

Once you've done that, check that `gpg-agent` is correctly configured, and restart it by `--kill`ing it, per [these instructions](https://superuser.com/a/1628810/728488).

#### On Linux

We'll use a [PGP key](https://en.wikipedia.org/wiki/Pretty_Good_Privacy) to sign this package, so users can be confident that you are the person who created it. To do this, you can mostly follow the instructions from [scala-sbt.org](https://www.scala-sbt.org/release/docs/Using-Sonatype.html)...

Install [GnuPG](https://www.gnupg.org/download/) (on macOS, you can do this via `brew install gnupg`) and verify that it's installed by checking the version

```bash
$ gpg --version
gpg (GnuPG) 2.3.1
...
```

Next, generate a key...

```bash
$ gpg --gen-key
```

...list your keys...

```bash
$ gpg --list-keys
gpg: checking the trustdb
gpg: marginals needed: 3  completes needed: 1  trust model: pgp
gpg: depth: 0  valid:   1  signed:   0  trust: 0-, 0q, 0n, 0m, 0f, 1u
gpg: next trustdb check due at 2023-07-11
/Users/andrew.watson/.gnupg/pubring.kbx
---------------------------------------
pub   ed25519 2021-07-11 [SC] [expires: 2023-07-11]
      1234517530FB96F147C6A146A326F592D39AAAAA
uid           [ultimate] Andrew Watson <my@email.com>
sub   cv25519 2021-07-11 [E] [expires: 2023-07-11]
```

...and distribute the key to some GPG keyserver, like

```bash
$ gpg --keyserver keyserver.ubuntu.com --send-keys 
1234517530FB96F147C6A146A326F592D39AAAAA
gpg: sending key A326F592D39AAAAA to hkp://keyserver.ubuntu.com
```

> ❗ **GOTCHA:** If you get an error like `keyserver send failed: No name` from the command above, it means the keyserver you're trying to send to is no longer active. [You can instead use any other major GPG keyserver](https://stackoverflow.com/a/68132500/2925434), like the Ubuntu one I used above.

[Leonard notes here](https://leonard.io/blog/2017/01/an-in-depth-guide-to-deploying-to-maven-central/) that synchronising these keys around the various keyservers can take some time, but for me it worked almost instantly.

Once you publish your key, you can check that it's visible on the public keyservers by searching for it here: https://keys.openpgp.org/.

### 3. Enable `sbt-pgp`

[As outlined here](https://www.scala-sbt.org/release/docs/Using-Sonatype.html), you'll next want to add the `sbt-pgp` plugin to your project. You can add it to your project's `plugin.sbt` file, or you can add it to `~/.sbt/1.0/plugins/gpg.sbt` to enable it for all sbt projects. The line you should add is

```scala
addSbtPlugin("com.github.sbt" % "sbt-pgp" % "2.2.1")
```

The latest version of this plugin can be found [at the GitHub page for the project](https://github.com/sbt/sbt-pgp).

### 4. Add your Sonatype credentials to your project

Next, add your credentials to your project by following the instructions in [Step 3](https://www.scala-sbt.org/release/docs/Using-Sonatype.html) here. I'll reproduce them to be clear...

Create a file at

```bash
$HOME/.sbt/1.0/sonatype.sbt
```

And add the following line to it

```scala
credentials += Credentials(Path.userHome / ".sbt" / "sonatype_credentials")
```

Then, create the credentials file at

```bash
~/.sbt/sonatype_credentials
```

Add the following information to that file

```
realm=Sonatype Nexus Repository Manager
host=s01.oss.sonatype.org
user=<your username>
password=<your password>
```

The `user` and `password` here are the same username and password you used when creating your account on Sonatype's Jira server in Step 1. You don't need to worry about putting anything in quotes here -- spaces are fine. But...

> ❗ **GOTCHA:** ...be careful! Some older guides say to set `host=oss.sonatype.org`, but this has recently changed to `host=s01.oss.sonatype.org`. More information about this change [can be found here](https://central.sonatype.org/publish/release/).

### 5. Set up your `publish.sbt`

The penultimate step in this process is to configure your project for publishing, by adding a bunch of information to either your project's `build.sbt` (or, as I recommend) `publish.sbt` file.

Replace `username` with your GitHub / GitLab username below, `project` with the name of the project (in my case, these are `awwsmm` and `zepto`, respectively), and all the other obvious fill-in bits, and paste it into a `publish.sbt` file at the root of your project

```scala
ThisBuild / organization := "io.github.username"
ThisBuild / organizationName := "username"
ThisBuild / organizationHomepage := Some(url("https://www.yourwebsite.com"))

ThisBuild / scmInfo := Some(
  ScmInfo(
    url("https://github.com/username/project"),
    "scm:git@github.username/project.git"
  )
)

ThisBuild / developers := List(
  Developer(
    id    = "username",
    name  = "Your Name Goes Here",
    email = "youremail@address.com",
    url   = url("https://www.yourwebsite.com")
  )
)

ThisBuild / description := "Describe your project here..."
ThisBuild / licenses := List("The Unlicense" -> new URL("https://unlicense.org/"))
ThisBuild / homepage := Some(url("https://github.com/username/project"))

// Remove all additional repository other than Maven Central from POM
ThisBuild / pomIncludeRepository := { _ => false }

ThisBuild / publishTo := {
  val nexus = "https://s01.oss.sonatype.org/"
  if (isSnapshot.value) Some("snapshots" at nexus + "content/repositories/snapshots")
  else Some("releases" at nexus + "service/local/staging/deploy/maven2")
}

ThisBuild / publishMavenStyle := true

ThisBuild / versionScheme := Some("early-semver")
```

You can set the `licenses` to whatever you feel is appropriate. Here is GitHub's guide to [choosing a license for your repo](https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/creating-a-repository-on-github/licensing-a-repository) -- though be aware that they [probably ignore it](https://news.ycombinator.com/item?id=27736650) anyway.

> ❗ **GOTCHA:** Notice that we've also used `s01.oss.sonatype.org` above!

### 6. Publish!

Once you've got all of that set up, you should be able to publish your project from the `sbt` shell with the `publishSigned` command

```scala
sbt> publishSigned
```

On macOS, this should open up the GPGTools UI, where you can enter the password you associated with your PGP key. The output will then look something like

```bash
[info] Wrote /local/path/to/zepto_2.13-0.4.0.pom
[info] 	published zepto_2.13 to https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/io/github/awwsmm/zepto_2.13/0.4.0/zepto_2.13-0.4.0-sources.jar
[info] 	published zepto_2.13 to https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/io/github/awwsmm/zepto_2.13/0.4.0/zepto_2.13-0.4.0-javadoc.jar
[info] 	published zepto_2.13 to https://s01.oss.sonatype.org/service/local/staging/deploy/maven2/io/github/awwsmm/zepto_2.13/0.4.0/zepto_2.13-0.4.0-javadoc.jar.asc
...
```

You can use your Sonatype username and password to log in to https://s01.oss.sonatype.org/ where you should now see your (pre-release) project!

Note that if you try to click on one of the `.jar` links above, you may get an error like

```
Staging of Repository within profile ID='...' is not yet started!
```

You can [follow along with Sonatype's guide here](https://central.sonatype.org/publish/release/) to release the artifacts to the public.

![Screenshot of my pre-release project listed in my account on sonatype.org](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ue4pc8kh7bmewm41qndl.png)

Select the project, click `Close` from the menu, and then `Release`. You should see your project [on Maven](https://repo1.maven.org/maven2/io/github/awwsmm/zepto_2.13/0.4.0/) almost immediately, though it will take a few hours to show up [in the search engine](https://search.maven.org/artifact/io.github.awwsmm/zepto_2.13).

...and that's it!

You should now be able to import your published library into other projects by adding

```
"io.github.awwsmm" %% "zepto" % "0.4.0"
```

(or whatever is appropriate) to the `libraryDependencies` in the `build.sbt` of another project.

Publish your Scala sbt Project to Maven in 5 Minutes with Sonatype

A Case Study in Building a Website using Next.js


## Trade-Offs in Software Development

> "Where's all my CPU and memory gone?"
>
> -- [thegeomaster](https://news.ycombinator.com/item?id=14868999)

### Software development is -- and has always been -- about trade-offs.

The [CAP theorem](https://en.wikipedia.org/wiki/CAP_theorem) tells us that we need to choose between consistency, availability, and partition tolerance when designing distributed data stores.

[Caches](https://www.indeed.com/career-advice/career-development/cache-pros-and-cons) can make information available more quickly, but it might be out of date, and it will definitely use more storage space than making a fresh request each time.

And, as always, money can be a factor. [SSDs are faster than HDDs](https://reviewed.usatoday.com/laptops/features/ssd-vs-hdd), but they are more expensive per GB. And microservices in a distributed system should each have their own database, but cloud computing costs can mean that shared DBs are more cost-effective.

Weighing the pros and cons of varied solutions to a problem was something I needed to tackle recently when redesigning part of my personal website, [awwsmm.com](https://awwsmm.com). Here's how that went down...

## A Case Study: My Website

### The Setup

My website is a pretty minimal Next.js site, written in TypeScript, hosted [on GitHub](https://github.com/awwsmm/awwsmm.com), built and deployed [by Vercel](https://vercel.com/).

In general, Next.js allows you to have two kinds of pages: static pre-rendered pages, and dynamic pages, rendered "just in time", when a visitor to your website tries to visit that page.

My website has static [blog posts](https://www.awwsmm.com/blog), as well as ["project" pages](https://www.awwsmm.com/projects), which give a quick overview of personal projects that I've been working on lately.

My initial design was for a given project page (for example, [this one for my website itself](https://www.awwsmm.com/projects/awwsmm.com)) to have an up-to-date commit history, interleaved with occasional "log entries", summarizing big sweeping changes made to these projects. (Kind of like release notes, but with more detail behind _why_ certain changes are being made.)

So what was the problem?

### The Problem

The problem was that these project pages were statically rendered; they were built in advance.

I would [request the commit history of a project from GitHub](https://github.com/awwsmm/awwsmm.com/blob/cfec0c66a0cf1ba8cc54459ee77c85b707d21d61/lib/utils/ProjectUtils.ts#L59) and write it to a local cache, saved in the repository.

When [a new PR is opened](https://github.com/awwsmm/awwsmm.com/pull/74) against the repo containing my website, Vercel runs a test deployment. When the test deployment looks good, I hit "merge" and the new changes are added to the repo in a merge commit.

But, because the cache is created only when I'm developing locally, _this merge commit is not a part of the commit history in the cache._ It couldn't possibly be. It would require updating the cache file, which would introduce changes not included in that commit, which would require another commit, ad infinitum.

This means that the commit history for the `awwsmm.com` project page is _always_ at least one commit behind `master`.

This bothered me, and I wanted to see if I could fix it.

### First Attempt: Environment-Aware Caching

[My first attempt at a solution](https://github.com/awwsmm/awwsmm.com/commit/569a6368f7bd115a13aa219b398b4bf23eab04ad#diff-e1998422d9b0a8903593c437a2bf074f2d47e10bede05c333e3a2312797ca486) was (what I'm going to call) "environment-aware caching".

What if, after the deployment passed and the new PR was merged into `master`, Vercel ignored the cache, only using it as a backup? During deployment, we could hit the GitHub API again, which should then have the new commit, right?

This required knowing which environment the build was running on. This is straightforward, as Vercel populates [a `VERCEL_ENV` environment variable](https://vercel.com/docs/concepts/projects/environment-variables/system-environment-variables) to `"production"`, `"preview"`, or `"deployment"`, depending on where the build is running.

But this also required maintaining a cache which we hopefully would never fall back to, which seemed kind of silly.

It also meant that each release would have to be deployed at least twice: once to satisfy the checks before merging the PR, and once after the PR was merged to pick up the new commit on `master`. I'd have to remember to do this "double deployment" to keep things up-to-date.

Finally, there was some human error here in that I tried to use [the same caching mechanism for "last updated" dates on my blog posts](https://github.com/awwsmm/awwsmm.com/commit/cb38f6411859de4586f945eaae6231ebcd87a78f#diff-8779cf4bd89603a2f8a50f4ea24460afbd79e6d8b65ba8575183d02f47cf074b), confusing the issue.

All in all, this solution was pretty complex to maintain (all I want is a static blog), and it put me off of keeping my website up-to-date for a while. When I finally came back here after a few months, I decided that a simpler solution was in order.

### Second Attempt: Server-Side Rendering

So how about trying to [render the project pages "just in time"?](https://github.com/awwsmm/awwsmm.com/commit/5daf1b29cbdcafa7f26f6fed340adbfc6482df2e#diff-dc938814c5900cd48c67f3c5b743ac02c3d0730245e9fa633d02e9fef4cdb801R104)

Vercel's [server-side rendering (SSR)](https://vercel.com/blog/nextjs-server-side-rendering-vs-static-generation) also generates static pages, but it renders them only when the user navigates to the page, not during deployment.

"This is great!" I thought. I could just request the commit history when the page is requested, and it would always be up-to-date.

Unfortunately, requesting and processing 100 commits from GitHub seemed to be too much to ask. I was consistently waiting about 3 seconds for the page to load, which is [really bad](https://web.dev/ttfb/). Project pages with shorter commit histories loaded a bit faster, but there was still a noticeable delay. Sending the request to GitHub, awaiting a response, processing the response, generating the resulting page, and displaying it just took too much time.

This approach would also send a request to GitHub _every time_ a user loaded that page.

This second issue could be solved by [fine-tuning the `Cache-Control`](https://github.com/awwsmm/awwsmm.com/commit/d6eb611eb5d9c746670cceeffc0fecfe32fd0817#diff-dc938814c5900cd48c67f3c5b743ac02c3d0730245e9fa633d02e9fef4cdb801R105-R123) header sent along with the request, such that I could guarantee that the 5000-requests-per-hour (authenticated) limit would never be exceeded.

But the first issue remained a problem.

### Third Attempt: Non-Blocking Server-Side Rendering

"Maybe I'm the problem" I thought.

"Maybe it's the way I (think I) am blocking inside of `getServerSideProps`."

[I rearranged this method](https://github.com/awwsmm/awwsmm.com/commit/ea8ec00ed8905f55d6bb780df5535dcb459af638) to not `await` anywhere, but return a `Promise` which is a result of other `Promise`s chained together with `then`, and nothing else.

I was hoping that, with everything done in a non-blocking way in the return value, Vercel could work some magic to speed up the call. (Maybe it could run the request as soon as the user _hovered over_ the link to the page?)

But it didn't help. I was still stuck at ~3 seconds of loading time. Worth a try, at least.

### Fourth Attempt: GraphQL

"Maybe the response from GitHub is taking so long because it's returning too much data?"

I wasn't using _most_ of the response anyway; all I cared about was the commit hash, the message, and the date. I'd never used GraphQL before, but I knew that it could be used in situations like this, where you wanted to request only _particular_ data from an endpoint.

So I learned enough about GraphQL and GitHub's API to [request only the commit data I cared about](https://github.com/awwsmm/awwsmm.com/commit/f6260bbc9bd1d181fbf3f613a73008f1cfe53e4c#diff-93b7ed2c3a46038993bf9d2c1478cc9dbb3f14e3beeafd30cb21983fbfc75befR67-R79).

This sped up the page a bit _locally_ but not in production on Vercel. It was still taking about 3 seconds.

### Fifth Attempt: Timeout / Fallback to Cache

"Well, if it's loading fast enough locally, but not remotely, maybe I can set a timeout threshold?"

My thought was that I could [cache](https://www.google.com/search?q=time+is+a+flat+circle) a "stale" version of the page to display if the "live" version of the page took too long to load. Maybe longer than 500ms or so.

So I would now [generate a cache when building locally](https://github.com/awwsmm/awwsmm.com/commit/927db29fb8cdfb53402d8a7aeff047503ec36ff1#diff-93b7ed2c3a46038993bf9d2c1478cc9dbb3f14e3beeafd30cb21983fbfc75bef), and save the cache to the repo. In production, I would attempt to request fresh data from GitHub, but if it took longer than 500ms, I would fall back to the cache.

...but it still took 3 seconds to render the page remotely.

"Why?" I thought, pulling my hair out in frustration.

Other project pages loaded quickly... maybe Vercel was just (somwehow?) slow to process this 600-line JSON file, generate the hundreds of components for it, style all of them, and display them.

I found [issues on the `vercel` repo](https://github.com/vercel/vercel/discussions/7961) of people complaining about similar problems. Maybe it just took a few seconds for Vercel to spin up a runner to render the page? (That didn't seem to square with the load times of the other project pages.)

I took solace in the fact that other people on the Internet _also_ thought that [`getServerSideProps` is weird](https://www.youtube.com/watch?v=1er7Zqs_h9k) and unintuitive.

I assessed other possible avenues for investigation: [timing the response?](https://web.dev/custom-metrics/?utm_source=devtools#server-timing-api), [actually reading the docs? (no thanks)](https://vercel.com/blog/nextjs-server-side-rendering-vs-static-generation), incremental static generation?, [edge functions?](https://vercel.com/docs/concepts/functions/edge-functions).

I was beginning to lose the will to live... all I wanted was a blog.

## Regrouping

I took a breather and came back to it all after a little while.

"What is it I actually want?"

- up-to-date commit histories
- fast page loads

The problem was that those two things were -- if not mutually exclusive, at least -- in competition with each other. Up-to-the-second commit histories would require a request to GitHub as soon as the user requested the project page. Which meant that all of the data fetching, and processing, and rendering would have to be done _quickly_.

This didn't seem like too big of an ask to me, but apparently it was. Maybe it was my code, maybe it was the Lambda cold start times, maybe it was something on Vercel's end... whatever it was, it was pinning me to 3-second page load times.

"Maybe I can request and process _most_ of the history in advance, then only request _recent_ history when a user clicks on a page?"

This sounded overcomplicated, though. (Unlikely, but) what if a project had had 100 new commits since I last deployed my website (and generated the cache)? Then I would have the same problem: trying to request and render 100 commits onto this history page.

[So I compromised](https://github.com/awwsmm/awwsmm.com/commit/34ee82581280b28da6041b4966da0d423665b885), and came up with a simpler solution:

1. generate `project/` pages statically

   This solves the load time issue -- by the time we get to production, the pages will have already been generated, so nothing needs to be done except displaying them.
   
   But this means that we will not have an up-to-date commit history.

2. accept that the commit history will not be up-to-date

   I'd already sunk so much time into this, I was ready to compromise.

   Nobody coming to my website will care if my commit histories are slightly out of date. So I decided to just put a disclaimer at the top and bottom of the histories, saying something like "for the most up-to-date commit history, see GitHub".

Now, we can generate pages in advance, keep load times to basically zero, and we don't have to worry about complex caching solutions, or what environment the build is running in, or GitHub rate limits, or anything.

My compromise was that my time and energy were worth more to me than having this one feature on my website be exactly the way I had envisioned it initially.

## Software Development is About Compromise

In my case, I tried and tried to get my website to do what I wanted it to, but my two requirements: up-to-date commit histories and fast loading times, were in direct competition with each other.

Having an up-to-date history will require a request when the user clicks the button, and processing the result of that request, which will take time.

Surely there are more complex solutions which balance these two better, but in the end, having these pages be up-to-date is not critical for my blog. Choosing where to use my time is another trade-off. I learned a lot about Vercel, GraphQL, and Promises in JavaScript / TypeScript during this process, but ultimately, "see GitHub" is good enough for me. And that's a compromise I'm willing to accept.

Blog