Let’s learn how to build end-to-end, type-safe applications with Gleam.

We’ll use a PostgreSQL database to store our data and Gleam for both the frontend and backend.

Project structure

First, let’s setup the project structure. We will need 3 top level folders: server, client and shared. Each of them is a Gleam project:

gleam new shared 
gleam new client 
gleam new server

So as the result we will get this folder structure:

.
├── client
│   ├── gleam.toml
│   ├── src
│   └── test
├── server
│   ├── gleam.toml
│   ├── src
│   └── test
└── shared
    ├── gleam.toml
    ├── src
    └── test

10 directories, 3 files

Note, that both client and server are using local dependency on the shared project. Aslo client is configured with javascript as the target.

target = "javascript"

[dependencies]
shared = { path = "../shared" }

Preparing Postgres Database

Let’s apply a pragmatic Domain-Driven Design (DDD) approach. Suppose we want to store and manage User entities. A sensible first step is to design the database schema, starting with a simple users table that contains a few core fields.

CREATE TABLE users (
    id          INTEGER GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    name        TEXT NOT NULL,
    avatar_url  TEXT,
    email       TEXT NOT NULL,
    created_at  TIMESTAMP NOT NULL DEFAULT now(),
    updated_at  TIMESTAMP
);

Next, let’s spin up a local database in a Docker container so we can use it for further development.

services:
  postgres:
    image: postgres:17-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: gleam
      POSTGRES_PASSWORD: gleam
      POSTGRES_DB: gleam
    ports:
      - "5432:5432"
    volumes:
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d app_db"]
      interval: 10s
      timeout: 5s
      retries: 5

Gleam SQL queries

How do we bring knowledge of the database schema into our Gleam code?

We don’t need a complex ORM or any magical framework. Instead, we can rely on plain SQL combined with a small amount of code generation.

To do this, let’s install the excellent squirrel library:

gleam add --dev squirrel

Notice the --dev flag — this isn’t even a runtime dependency.

Next, let’s add queries, written as plain SQL, to the server/src/sql directory.

└── server
    ├── gleam.toml
    └── src
        └── sql
            ├── insert_users.sql
            ├── search_users_by_name.sql
            ├── select_users_by_email.sql
            ├── select_users_by_id.sql
            └── select_users.sql

So we end up with the following SQL files: select_users.sql, select_users_by_id.sql, select_users_by_email.sql, insert_users.sql

-- select_users.sql
SELECT * FROM users;

-- insert_users.sql
INSERT INTO users(name, email)
VALUES ($1, $2)
RETURNING *;

-- select_users_by_email.sql
SELECT * FROM users WHERE email = $1;

-- select_users_by_id.sql
SELECT * FROM users WHERE id = $1;

These queries are trivial for now, but we’re not limited in any way — each SQL file can contain any SQL that PostgreSQL understands. For example, we can start using full-text search right away:

SELECT * FROM users
WHERE to_tsvector('simple', name) @@ plainto_tsquery('simple', $1);

The SQL file names can be anything — it’s up to you to choose a convention. Keep in mind, however, that the generated code will use the file name both for the Gleam function name and for the generated record types.

I currently follow this pattern: the operation (e.g. select, insert, update, delete), followed by the table name, and then an optional condition.

Before we can use squirrel for code generation, we need to set the DATABASE_URL environment variable.

export DATABASE_URL=postgres://gleam:gleam@localhost:5432/gleam

❯ gleam run -m squirrel
   Compiled in 0.08s
    Running squirrel.main
🐿️  Generated 5 queries!

The squirrel library detected SQL files and generated 5 queries, that looks promising! Now that we have the server/src/sql.gleam file, let’s take a look at what was generated.

For example, here is the select_user_by_id function (most of the decoding logic is omitted for simplicity):

pub fn select_users_by_id(
  db: pog.Connection,
  arg_1: Int,
) -> Result(pog.Returned(SelectUsersByIdRow), pog.QueryError) {
  let decoder = {
    use id <- decode.field(0, decode.int)
    use name <- decode.field(1, decode.string)

    decode.success(SelectUsersByIdRow(id:,name:,))
  }

  "SELECT * FROM users WHERE id = $1; "
  |> pog.query
  |> pog.parameter(pog.int(arg_1))
  |> pog.returning(decoder)
  |> pog.execute(db)
}

And also the custom record type SelectUsersByIdRow:

pub type SelectUsersByIdRow {
  SelectUsersByIdRow(
    id: Int,
    name: String,
    avatar_url: Option(String),
    email: String,
    created_at: Timestamp,
    updated_at: Option(Timestamp),
  )
}

The same code is generated for the other queries as well.

Next, let’s see how we can execute these queries from the main function of our application:

pub fn main() -> Nil {
  let conn = conn()

  let assert Ok(query_result) = sql.insert_users(conn, "Andrey", "email@example.com")
  let assert Ok(inserted_user) = list.first(query_result.rows)

  // InsertUsersRow(2, "Andrey", None, "email@example.com", Timestamp(1768325274, 201229000), None)
}

And also query by id:

pub fn main() -> Nil {
  let conn = conn()

  let assert Ok(query_result) = sql.select_users_by_id(conn, 0)
  // Returned(0, [])

  let assert Ok(query_result) = sql.select_users_by_id(conn, 1)
  // Returned(1, [SelectUsersByIdRow(1, "Andrey", None, "email@example.com", Timestamp(1768303857, 745886000), None)])

  Nil
}

This is already really handy — and pretty cool! It’s also type-safe, since the queries and record types were generated based on the actual database schema; the squirrel library performs schema introspection automatically.

The only downside is that we end up with multiple record types that actually represent the same logical type:

pub type SelectUsersRow {
  SelectUsersRow(
    id: Int,
    name: String,
    avatar_url: Option(String),
    email: String,
    created_at: Timestamp,
    updated_at: Option(Timestamp),
  )
}

pub type InsertUsersRow {}
pub type SearchUsersByNameRow {}
pub type SelectUsersByEmailRow {}
pub type SelectUsersByIdRow {}

The solution to that is to create a domain model in the shared project:

pub type User {
  User(
    id: Int,
    name: String,
    avatar_url: Option(String),
    email: String,
    created_at: Timestamp,
    updated_at: Option(Timestamp),
  )
}

Now we can create a set of mappers to convert the SQL-generated types into our domain types:

pub fn select_users_row_to_user(row: sql.SelectUsersRow) -> users.User {
  users.User(
    id: row.id,
    name: row.name,
    avatar_url: row.avatar_url,
    email: row.email,
    created_at: row.created_at,
    updated_at: row.updated_at,
  )
}

pub fn insert_users_row_to_user(row: sql.InsertUsersRow) {}

pub fn select_users_by_email_row_to_user(row: sql.SelectUsersByEmailRow) {}

pub fn select_users_by_id_row_to_user(row: sql.SelectUsersByIdRow) {}

pub fn search_users_by_name_row_to_user(row: sql.SearchUsersByNameRow) {}

Yes, this approach introduces some code duplication, and when there’s a one-to-one mapping with the domain model it can feel redundant. But, in real-world applications it’s often much more useful, as it adds a layer of abstraction that can hide the underlying database structure, allow remapping of fields, and more.

Once we have a shared User type as our domain model, we can use it to send data over the wire — for example, in a JSON REST API.

There are also language server (LSP) commands that can automatically generate _to_json and _from_json functions for us:

pub fn user_to_json(user: User) -> json.Json {
  let User(id:, name:, avatar_url:, email:, created_at:, updated_at:) = user
  json.object([
    #("id", json.int(id)),
    #("name", json.string(name)),
    #("avatar_url", case avatar_url {
      option.None -> json.null()
      option.Some(value) -> json.string(value)
    }),
    #("email", json.string(email)),
    #("created_at", timestamp_to_json(created_at)),
    #("updated_at", case updated_at {
      option.None -> json.null()
      option.Some(value) -> timestamp_to_json(value)
    }),
  ])
}

And a decoder:

pub fn user_decoder() -> decode.Decoder(User) {
  use id <- decode.field("id", decode.int)
  use name <- decode.field("name", decode.string)
  use avatar_url <- decode.field("avatar_url", decode.optional(decode.string))
  use email <- decode.field("email", decode.string)
  use created_at <- decode.field("created_at", timestamp_decoder())
  use updated_at <- decode.field(
    "updated_at",
    decode.optional(timestamp_decoder()),
  )
  decode.success(User(id:, name:, avatar_url:, email:, created_at:, updated_at:))
}

Don’t forget that we place this domain type and its encode/decode functions into a shared project. This means both the server and client will work with the same entities, which reduces friction and the possibility of errors.

Let’s look at an example from the server, where we fetch users from the database, map them to the domain model type, and encode them to JSON — making them ready to be sent in an API response:

pub fn main() -> Nil {
  let assert Ok(query_result) = sql.select_users(conn())

  let users: List(users.User) =
    query_result.rows
    |> list.map(mappers.select_users_row_to_user)

  let users_json: String =
    users
    |> json.array(users.user_to_json)
    |> json.to_string

  // "[{\"id\":1,\"name\":\"Andrey\",\"avatar_url\":null,...}]"
}

Moving to the client side

Finally, it’s time to move to the frontend. Here’s a function that simulates the JSON response from our backend:

fn random_users_json_from_api() {
  list.range(0, 10)
  |> list.map(fn(id) {
    let name = "user_" <> int.to_string(id)
    users.User(
      id: id,
      name: name,
      email: name <> "@example.com",
      avatar_url: option.Some(
        "https://api.dicebear.com/9.x/glass/svg?seed=" <> name,
      ),
      created_at: timestamp.from_unix_seconds(0),
      updated_at: option.None,
    )
  })
  |> json.array(users.user_to_json)
  |> json.to_string
}

We can define a user_view helper function in the client app, which specifies how to render our User type:

fn user_view(user: users.User) {
  div([class("flex gap-2 border border-gray-200 p-2 rounded-3xl")], [
    div([class("flex gap-2 items-center")], [
      div([], [
        html.img([
          class("w-20 h-auto rounded-full"),
          src(option.unwrap(user.avatar_url, "")),
        ]),
      ]),
      div([], [
        p([class("font-bold")], [text(user.name)]),
        p([class("text-gray-700")], [text(user.email)]),
      ]),
    ]),
  ])
}

So, if we put everything together, this is what our complete client app looks like:

fn random_users_json_from_api() {}
fn user_view(user: users.User) {}

pub fn main() {
  let assert Ok(users) =
    random_users_json_from_api()
    |> json.parse(decode.list(users.user_decoder()))

  let app =
    lustre.element(div(
      [attribute.class("max-w-xl mx-auto space-y-2")],
      users |> list.map(user_view),
    ))
    
  let assert Ok(_) = lustre.start(app, "#app", Nil)
}

That’s it! We now have type-safety all the way from the database to the frontend, and everything lives in the same repository.

If the database schema changes in an incompatible way and we regenerate sql.gleam, the compiler will immediately show errors in the project that need to be fixed.

The same applies to field renames or similar changes. Since the shared module defines the types, the compiler checks both client and server code, so we avoid unnoticed issues and runtime exceptions.

Plus, the compiler is extremely fast, so running both server and client builds in watch mode gives us instant feedback!

Subscribe now

If you want to see a full example, you can check out the project on my GitHub: https://github.com/andfadeev/learn_gleam/tree/master/type_safety

Not long ago, I started my Gleam learning journey and have been enjoying it a lot so far. In this article, I’d like to share what I’ve learned about web development with Gleam.

We’ll build a REST web application step by step, backed by a PostgreSQL database and complemented with a small HTML interface. Along the way, we’ll use the following packages:

• wisp and mist for the web server

• squirrel for the type-safe SQL queries

• our app will talk JSON

• lustre for building server-side HTML

The entire code for this blog post could be found here: https://github.com/andfadeev/learn_gleam_todo

Subscribe now

Setting up the development environment

Setting up the development environment on macOS is simple. I was using homebrew but it’s also possible with version managers like asdf or mise.

brew install gleam
brew install erlang
brew install rebar3

Let’s confirm that everything is ready and good to go:

❯ gleam --version
gleam 1.13.0

❯ erl --version
Erlang/OTP 28

❯ rebar3 --version
rebar 3.25.1 on Erlang/OTP 28 Erts 16.1.2

Since I’m currently using my wife’s MacBook Air with only 8 GB of RAM, system resources are limited, and my usual IDE (JetBrains IntelliJ IDEA) is painfully slow. As a result, I decided to use Zed (https://zed.dev) as the editor for this project.

Zed is extremely fast, and I haven’t experienced any delays or slowdowns on this machine. It’s also extensible, with support for language servers and Tree-sitter.

When you open a Gleam file, Zed automatically detects it and prompts you to install the appropriate extension. After that, it connects to the Gleam Language Server, so the entire setup was essentially automatic.

Gleam language

Gleam is a statically typed, functional programming language focused on simplicity, correctness, and developer experience. It features a strong type system with powerful type inference, immutable data, and an expressive standard library, and it compiles to both the Erlang VM (BEAM) and JavaScript.

I wanted to become more familiar with the BEAM ecosystem and was looking for a language to start with. Over the past decade, I’ve mostly worked with Clojure, which is dynamically typed. While Elixir is more mature, I was looking for a different experience—specifically one with stronger typing, which I’ve started to miss.

Here are some of the language features I’d like to highlight:

• Immutability

• A concise yet powerful standard library

• An emphasis on one clear way of doing things

• No nulls and forced error handling

• Built-in Result and Option types

The goal of this blog is to provide a step-by-step guide to building a web application with a database backend. Because of that, I won’t go deep into the Gleam language itself. The best way to get familiar with Gleam is to follow the official language tour.

Gleam CLI tool

Once Gleam is installed on your machine, you gain access to the gleam CLI tool. This tool will be our main entry point for all common tasks — creating a new project, building and running it, managing dependencies, formatting code, and running tests.

You can see the list of commands by running:

gleam help

Let’s start with creating a new project:

gleam new learn_gleam_todo

Now we can move to the newly created folder and run tests and the project itself:

cd learn_gleam_todo

gleam test
1 passed, no failures

gleam run 
Hello from learn_gleam_todo!

The CLI tool is also used to add dependencies (that are recorded in the gleam.toml file). Let’s add the dependency for the wisp framework (https://github.com/gleam-wisp/wisp) that we are going to use later:

gleam add wisp

cat gleam.toml
name = "learn_gleam_todo"
version = "1.0.0"

[dependencies]
gleam_stdlib = ">= 0.44.0 and < 2.0.0"
wisp = ">= 2.1.1 and < 3.0.0"

[dev-dependencies]
gleeunit = ">= 1.0.0 and < 2.0.0"

Later, it could be used to keep our dependencies up-to-date by running:

gleam update

Create a web server

In this section, we will create a web server and a REST API for our Todo application.

Let’s add dependencies that will be needed to get started:

gleam add wisp gleam_erlang mist envoy

Let’s finally write some Gleam code and edit the src/learn_gleam_todo.gleam file:

import envoy
import gleam/erlang/process
import gleam/result
import mist
import wisp.{type Request, type Response}
import wisp/wisp_mist

fn middleware(req: Request, handler: fn(Request) -> Response) -> Response {
  use <- wisp.log_request(req)
  use <- wisp.rescue_crashes
  use req <- wisp.handle_head(req)
  use req <- wisp.csrf_known_header_protection(req)

  handler(req)
}

fn handler(req: Request) -> Response {
  use _ <- middleware(req)

  wisp.ok()
  |> wisp.string_body("Hellow from Wisp & Gleam")
}

pub fn main() -> Nil {
  wisp.configure_logger()

  let secret =
    result.unwrap(envoy.get("SECRET_KEY_BASE"), "wisp_secret_fallback")

  let assert Ok(_) =
    wisp_mist.handler(handler, secret)
    |> mist.new
    |> mist.port(8080)
    |> mist.start

  process.sleep_forever()
}

A lot is going on here, so let’s break it down. In the main function, we create a web server and configure it to listen on port 8080. Wisp requires a unique secret, so we use the envoy package to read it from an environment variable, with a fallback to a hardcoded value. If you prefer, you can instead require the environment variable to be set and let the application fail fast if it’s missing.

Our handler is just a function that accepts wisp.Request and returns wisp.Response. It will catch all the routes and all HTTP methods for now and will always return 200 with a text response.

We also used a subset of available middleware to have basic logging and error handling.

Now let’s run our application and validate that it’s working. I’m going to use curl but you can just open the URL in the browser:

gleam run
Listening on http://127.0.0.1:8080

curl http://127.0.0.1:8080
Hellow from Wisp & Gleam

You can find the result of this step in this commit: https://github.com/andfadeev/learn_gleam_todo/commit/4e4dc6ce896dc6bda46f8cfb8641922df5132a65

Adding REST CRUD API

In this section, we will add some routes to our web handler, so we will need:

• GET /todos — to list all todo items

• POST /todos — to create a new todo item

• GET /todos/:id — get todo item by id

• DELETE /todos/:id — delete todo item by id

Interestingly, there are no abstractions or special syntax to define the route table. We will use Gleam functions and pattern matching to define the logic we need.

Let’s first add a couple of additional dependencies that we will need further:

gleam add gleam_http gleam_json

Now it’s time to define routes, not that we are just using pattern matching on the URL segment first and on the HTTP method (utilising some helper functions provided by wisp):

fn delete_todo_handler(_id: String) {
  wisp.no_content()
}

fn get_todo_handler(id: String) {
  wisp.string_body(wisp.ok(), "Todo item " <> id)
}

fn todo_handler(req: Request, id: String) -> Response {
  case req.method {
    http.Get -> get_todo_handler(id)
    http.Delete -> delete_todo_handler(id)
    _ -> wisp.method_not_allowed([http.Get, http.Delete])
  }
}

fn get_todos_hander() {
  wisp.string_body(wisp.ok(), "Todo items")
}

fn post_todos_hander() {
  wisp.created()
}

fn todos_handler(req: Request) -> Response {
  case req.method {
    http.Get -> get_todos_hander()
    http.Post -> post_todos_hander()
    _ -> wisp.method_not_allowed([http.Get, http.Post])
  }
}

fn handler(req: Request) -> Response {
  use req <- middleware(req)

  case wisp.path_segments(req) {
    ["todos"] -> todos_handler(req)
    ["todos", id] -> todo_handler(req, id)
    _ -> wisp.not_found()
  }
}

Time to restart the application and do some testing:

gleam run
Listening on http://127.0.0.1:8080

curl http://127.0.0.1:8080/todos
Todo items

curl -X POST http://127.0.0.1:8080/todos
Created

curl http://127.0.0.1:8080/todos/1
Todo item 1

curl -i -X DELETE http://127.0.0.1:8080/todos/1
HTTP/1.1 204 No Content

Code changes could be found in this commit: https://github.com/andfadeev/learn_gleam_todo/commit/1bceaabf134befc0e7ba9013e82e983d30b5f01a

Adding JSON support

Now let’s add a model for your todo items, support JSON in responses and requests. Again, we will need some extra packages:

gleam add gleam_time youid

So we are going to create the type for our todo items, JSON encoder and generate some random items for the GET /todos response:

type TodoItem {
  TodoItem(
    id: uuid.Uuid,
    title: String,
    description: option.Option(String),
    status: String,
    created_at: timestamp.Timestamp,
    updated_at: timestamp.Timestamp,
  )
}

fn timestamp_to_json(ts: timestamp.Timestamp) {
  json.string(timestamp.to_rfc3339(ts, calendar.utc_offset))
}

fn todo_item_to_json(item: TodoItem) {
  json.object([
    #("id", json.string(uuid.to_string(item.id))),
    #("title", json.string(item.title)),
    #("description", json.string(option.unwrap(item.description, ""))),
    #("status", json.string(item.status)),
    #("created_at", timestamp_to_json(item.created_at)),
    #("updated_at", timestamp_to_json(item.updated_at)),
  ])
}

fn get_todos_hander() {
  let todo_items = [
    TodoItem(
      uuid.v4(),
      "todoitem1",
      option.Some("description 1"),
      "pending",
      timestamp.from_unix_seconds(1_766_689_000),
      timestamp.from_unix_seconds(1_766_689_000),
    ),
    TodoItem(
      uuid.v4(),
      "todoitem2",
      option.None,
      "completed",
      timestamp.from_unix_seconds(1_766_689_000),
      timestamp.from_unix_seconds(1_766_689_000),
    ),
  ]

  json.array(todo_items, todo_item_to_json)
  |> json.to_string
  |> wisp.json_response(200)
}

So now everything should be in place, and we can test the endpoint. I’m going to use jq for pretty printing in the terminal:

gleam run 

curl http://127.0.0.1:8080/todos | jq
[
  {
    "id": "2d2238c3-631b-49d9-962b-6aa251a5a34f",
    "title": "todoitem1",
    "description": "description 1",
    "status": "pending",
    "created_at": "2025-12-25T18:56:40Z",
    "updated_at": "2025-12-25T18:56:40Z"
  },
  {
    "id": "ec68c70f-0214-4fe6-84a0-b84535e682b0",
    "title": "todoitem2",
    "description": "",
    "status": "completed",
    "created_at": "2025-12-25T18:56:40Z",
    "updated_at": "2025-12-25T18:56:40Z"
  }
]

Commit for this change could be found by this link: https://github.com/andfadeev/learn_gleam_todo/commit/5fccd22bc1694efb3f1af9e02c90c94a68c2351a

Handling JSON request body

Finally, we need to support parsing the incoming request body for the POST /todos route:

import gleam/dynamic/decode

fn post_todos_handler(req: Request) {
  use json <- wisp.require_json(req)

  let result = {
    let decoder = {
      use title <- decode.field("title", decode.string)
      use description <- decode.optional_field("description", "", decode.string)
      decode.success(#(title, description))
    }
    use #(title, description) <- result.try(decode.run(json, decoder))

    let todo_item =
      TodoItem(
        uuid.v4(),
        title,
        option.Some(description),
        "pending",
        timestamp.from_unix_seconds(1_766_689_000),
        timestamp.from_unix_seconds(1_766_689_000),
      )

    Ok(
      todo_item_to_json(todo_item)
      |> json.to_string
      |> wisp.json_response(200),
    )
  }

  case result {
    Ok(resp) -> resp
    Error(_) -> wisp.unprocessable_content()
  }
}

Let’s test that the POST endpoint is working now:

curl -X POST http://127.0.0.1:8080/todos \
  -H "Content-Type: application/json" \
  -d '{
    "title": "mytodo",
    "description": "something here"
  }' | jq

{
  "id": "0390b077-4c9e-4cd9-982a-672250f4d707",
  "title": "mytodo",
  "description": "something here",
  "status": "pending",
  "created_at": "2025-12-25T18:56:40Z",
  "updated_at": "2025-12-25T18:56:40Z"
}

Code change could be found here: https://github.com/andfadeev/learn_gleam_todo/commit/40fdef1445aeeaa3d08787e1e97b6a739947f2ea

Adding a database layer

In this section, we are going to add a PostgreSQL database backend to store our todo items.

Let’s define a simple Docker Compose file to have a way to start the database. We will also point it to an initialisation script so we will have a database table created automatically:

Create docker-compose.yml in the root of the project:

services:
  postgres:
    image: postgres:17-alpine
    restart: unless-stopped
    environment:
      POSTGRES_USER: gleam
      POSTGRES_PASSWORD: gleam
      POSTGRES_DB: gleamdb
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d app_db"]
      interval: 10s
      timeout: 5s
      retries: 5

volumes:
  postgres_data:

And also the init.sql in the root as well:

CREATE TABLE todo_items (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    title TEXT NOT NULL,
    description TEXT,
    status TEXT NOT NULL DEFAULT 'pending',
    created_at TIMESTAMP NOT NULL DEFAULT now(),
    updated_at TIMESTAMP NOT NULL DEFAULT now()
);

Now we can start the database by running:

docker compose up -d

psql -h localhost -p 5432 -U gleam -d gleamdb
Password: gleam

gleamdb=# select * from todo_items ;
 id | title | description | status | created_at | updated_at
----+-------+-------------+--------+------------+------------
(0 rows)

We are going to use squirrel library (https://github.com/giacomocavalieri/squirrel) to generate type-safe Gleam code from plain SQL queries. Let’s create a bunch of SQL files in the src/sql folder:

❯ tree src/sql
src/sql
├── delete_todo_item.sql
├── find_todo_item.sql
├── find_todo_items.sql
└── insert_todo_item.sql

1 directory, 4 files

Each file will contain an SQL query:

-- delete_todo_item.sql

delete from todo_items
where id = $1;

-- find_todo_item.sql

select * from todo_items
where id = $1;

-- find_todo_items.sql

select * from todo_items;

-- insert_todo_item.sql

insert into todo_items (title, description, status)
values ($1, $2, $3)
returning *;

Now let’s add a new package, note that we are using --dev as it will be only used for code generation:

gleam add pog
gleam add squirrel --dev

We also need to expose an env var so we can run a code generation command that will actually read our database table definition to generate type-safe Gleam code:

export DATABASE_URL=postgres://gleam:gleam@localhost:5432/gleamdb

❯ gleam run -m squirrel
🐿️  Generated 4 queries!

You can explore a new Gleam file src/sql.gleam with generated code, for example:

/// A row you get from running the `find_todo_item` query
/// defined in `./src/sql/find_todo_item.sql`.
///
/// > 🐿️ This type definition was generated automatically using v4.6.0 of the
/// > [squirrel package](https://github.com/giacomocavalieri/squirrel).
///
pub type FindTodoItemRow {
  FindTodoItemRow(
    id: Uuid,
    title: String,
    description: Option(String),
    status: String,
    created_at: Timestamp,
    updated_at: Timestamp,
  )
}

/// Runs the `find_todo_item` query
/// defined in `./src/sql/find_todo_item.sql`.
///
/// > 🐿️ This function was generated automatically using v4.6.0 of
/// > the [squirrel package](https://github.com/giacomocavalieri/squirrel).
///
pub fn find_todo_item(
  db: pog.Connection,
  arg_1: Uuid,
) -> Result(pog.Returned(FindTodoItemRow), pog.QueryError) {
  let decoder = {
    use id <- decode.field(0, uuid_decoder())
    use title <- decode.field(1, decode.string)
    use description <- decode.field(2, decode.optional(decode.string))
    use status <- decode.field(3, decode.string)
    use created_at <- decode.field(4, pog.timestamp_decoder())
    use updated_at <- decode.field(5, pog.timestamp_decoder())
    decode.success(FindTodoItemRow(
      id:,
      title:,
      description:,
      status:,
      created_at:,
      updated_at:,
    ))
  }

  "select * from todo_items
where id = $1;
"
  |> pog.query
  |> pog.parameter(pog.text(uuid.to_string(arg_1)))
  |> pog.returning(decoder)
  |> pog.execute(db)
}

Now we can create a connection pool and execute some queries:

import pog
import sql

pub fn main() -> Nil {

  let db_pool_name = process.new_name("db_pool")
  let assert Ok(database_url) = envoy.get("DATABASE_URL")
  let assert Ok(pog_config) = pog.url_config(db_pool_name, database_url)
  let assert Ok(_) =
    pog_config
    |> pog.pool_size(10)
    |> pog.start

  let con = pog.named_connection(db_pool_name)

  case sql.find_todo_items(con) {
    Ok(todo_items) -> {
      echo "Got todo items"
      echo todo_items
      echo "OK"
    }
    Error(_) -> {
      echo "Failed to get todo items"
    }
  }
}

Let’s test, don’t forget that we now need the DATABASE_URL env var:

export DATABASE_URL=postgres://gleam:gleam@localhost:5432/gleamdb

gleam run
"Got todo items"
Returned(0, [])
"OK"

Code change could be found here: https://github.com/andfadeev/learn_gleam_todo/commit/143fdc20a477cf0727ae0446772136f13db07582

Integrating the database into the web application

So we created the connection pool in the main function, but we need to have access to the database connection from handlers. The way of doing it is to create a new Context type that will hold a database connection and pass it to handlers:

pub type Context {
  Context(db: pog.Connection)
}

fn handler(req: Request, ctx: Context) -> Response {
  use req <- middleware(req)

  case wisp.path_segments(req) {
    ["todos"] -> todos_handler(req, ctx)
    ["todos", id] -> todo_handler(req, ctx, id)
    _ -> wisp.not_found()
  }
}

pub fn main() -> Nil {

  // rest of the main fn

  let con = pog.named_connection(db_pool_name)

  let context = Context(con)
  let handler = handler(_, context)

  let assert Ok(_) =
    wisp_mist.handler(handler, secret)
    |> mist.new
    |> mist.port(8080)
    |> mist.start

  process.sleep_forever()
}

Note that we will also need to pass the new context object to all the handlers, see full change here: https://github.com/andfadeev/learn_gleam_todo/commit/1f4f578a3f48af39597c7b3b98c7b933c7aca8e2

Connecting the dots

Now that we have database access and our REST handlers in place, it’s time to make things real. We’ll start persisting data by inserting records into the database in the POST request and using proper queries in the other handlers.

We’ll work with database types, map them to our custom TodoItem type, and JSON-encode the results. We’ll also make sure that the id passed in the request path is a valid UUID:

fn delete_todo_handler(ctx: Context, id: String) {
  case uuid.from_string(id) {
    Ok(id) -> {
      case sql.delete_todo_item(ctx.db, id) {
        Ok(_) -> wisp.no_content()
        Error(_) -> wisp.internal_server_error()
      }
    }
    Error(_) -> wisp.bad_request("Invalid id")
  }
}

fn get_todo_handler(ctx: Context, id: String) {
  case uuid.from_string(id) {
    Ok(id) -> {
      case sql.find_todo_item(ctx.db, id) {
        Ok(todo_item) -> {
          case todo_item.rows {
            [] -> wisp.not_found()
            [row] -> {
              let todo_item =
                TodoItem(
                  row.id,
                  row.title,
                  row.description,
                  row.status,
                  row.created_at,
                  row.updated_at,
                )

              todo_item_to_json(todo_item)
              |> json.to_string
              |> wisp.json_response(200)
            }
            _ -> {
              wisp.internal_server_error()
            }
          }
        }
        Error(_) -> wisp.internal_server_error()
      }
    }
    Error(_) -> wisp.bad_request("Invalid id")
  }
}

fn get_todos_hander(ctx: Context) {
  case sql.find_todo_items(ctx.db) {
    Ok(todo_items) -> {
      todo_items.rows
      |> list.map(fn(row: sql.FindTodoItemsRow) {
        TodoItem(
          row.id,
          row.title,
          row.description,
          row.status,
          row.created_at,
          row.updated_at,
        )
      })
      |> json.array(todo_item_to_json)
      |> json.to_string
      |> wisp.json_response(200)
    }
    Error(_) -> {
      wisp.internal_server_error()
    }
  }
}

fn post_todos_handler(req: Request, ctx: Context) {
  use json <- wisp.require_json(req)

  let result = {
    let decoder = {
      use title <- decode.field("title", decode.string)
      use description <- decode.optional_field("description", "", decode.string)
      decode.success(#(title, description))
    }
    use #(title, description) <- result.try(decode.run(json, decoder))

    case sql.insert_todo_item(ctx.db, title, description, "pending") {
      Ok(r) -> {
        case r.rows {
          [row] -> {
            TodoItem(
              row.id,
              row.title,
              row.description,
              row.status,
              row.created_at,
              row.updated_at,
            )
            |> todo_item_to_json()
            |> json.to_string
            |> wisp.json_response(200)
            |> Ok()
          }
          _ -> Ok(wisp.internal_server_error())
        }
      }
      Error(_) -> {
        Ok(wisp.internal_server_error())
      }
    }
  }

  case result {
    Ok(resp) -> resp
    Error(_) -> wisp.unprocessable_content()
  }
}

Of course, error handling and error messaging could be better, but you got the idea. Full change for this section you can find in this commit: https://github.com/andfadeev/learn_gleam_todo/commit/5b06e5280d8450ff210afd176039c5733a1611bd

Lustre

Finally, let’s cover the UI. Gleam could be compiled to JavaScript, and there is a library to build complex frontend applications (Elm-inspired): https://github.com/lustre-labs/lustre

In this example, we will just use it as an HTML DSL, so we will be able to create HTML views without templates, but directly with Gleam code. A bit more verbose compared to Clojure Hiccup, but it’s type-safe on the other hand.

gleam add lustre

The goal is to create an index handler for the GET / and show the list of todo items, but as an HTML page:

import lustre/attribute as attr
import lustre/element
import lustre/element/html
import gleam/int

fn todo_item_component(item: TodoItem) {
  html.div([attr.class("rounded border mt-4 p-4")], [
    html.h2([], [html.text(item.title)]),
    html.p([], [html.text(option.unwrap(item.description, ""))]),
  ])
}

fn get_index_handler(req: Request, context: Context) -> Response {
  use <- wisp.require_method(req, http.Get)

  case sql.find_todo_items(context.db) {
    Ok(todo_items) -> {
      let todo_items_html =
        todo_items.rows
        |> list.map(fn(i: sql.FindTodoItemsRow) {
          TodoItem(
            i.id,
            i.title,
            i.description,
            i.status,
            i.created_at,
            i.updated_at,
          )
        })
        |> list.map(todo_item_component)
      let html =
        html.html([], [
          html.head([], [
            html.title([], "Gleam todo items"),
            html.script([attr.src("https://cdn.tailwindcss.com")], ""),
          ]),
          html.body([attr.class("max-w-2xl mx-auto")], [
            html.h1([attr.class("text-red-800 font-bold")], [
              html.text("Todo: " <> int.to_string(list.length(todo_items_html))),
            ]),
            html.div([attr.class("text-blue-800")], todo_items_html),
          ]),
        ])

      wisp.ok()
      |> wisp.html_body(element.to_document_string(html))
    }
    Error(_) -> {
      wisp.internal_server_error()
    }
  }
}

fn handler(req: Request, ctx: Context) -> Response {
  use req <- middleware(req)

  case wisp.path_segments(req) {
    [] -> get_index_handler(req, ctx)
    ["todos"] -> todos_handler(req, ctx)
    ["todos", id] -> todo_handler(req, ctx, id)
    _ -> wisp.not_found()
  }
}

You can open the index page in your browser and see the result:

curl http://127.0.0.1:8080

<!doctype html>
<html>
   <head>
      <title>Gleam todo items</title>
      <script src="https://cdn.tailwindcss.com"></script>
   </head>
   <body class="max-w-2xl mx-auto">
      <h1 class="text-red-800 font-bold">Todo: 3</h1>
      <div class="text-blue-800">
         <div class="rounded border mt-4 p-4">
            <h2>mytodo</h2>
            <p>something here</p>
         </div>
         <div class="rounded border mt-4 p-4">
            <h2>mytodo</h2>
            <p>something here</p>
         </div>
         <div class="rounded border mt-4 p-4">
            <h2>mytodo</h2>
            <p>something here</p>
         </div>
      </div>
   </body>
</html>

Full commit for this change could be found by this link: https://github.com/andfadeev/learn_gleam_todo/commit/d00461ea30516fba5321d1b441b1bd98fb4a0d84

I think that’s it, the whole example could be found by this link: https://github.com/andfadeev/learn_gleam_todo

Final thoughts

I’ve had a blast learning Gleam so far, and I’ll definitely continue exploring it. That said, I’ll need to work on a larger project to better understand where the rough edges are and what might be missing.

While Gleam has been gaining more traction recently, it’s still early days. I wouldn’t fully bet on it yet, but it’s absolutely worth learning if you want to broaden your horizons and explore a different part of the BEAM ecosystem.

A few thoughts on areas for improvement—please note that I’m still a Gleam beginner, so I may be missing existing or emerging solutions:

• Proper live reload support for web development

• Stronger documentation

• Ecosystem maturity (need to explore using external Elixir or Erlang libraries)

Subscribe now

I've been using Substack for my tech blog and newsletter for a while.

As a developer, I truly believe that sharing knowledge through a newsletter is one of the best ways to grow an audience — and eventually monetise your efforts.

Subscribe now

Substack is built for a general audience, not devs. I was missing code highlighting and minimalist design.

I wanted to build something dedicated for developers:

• open-source, free to use, self-hosted

• beautiful code highlighting

• markdown-first writing

• clean, minimalist design

• dark mode support

I believe that an open-source model is a great way to build a product. IndiePubStack is:

• fully open-source and free to use (MIT licence)

• designed to be self-hosted (Docker image distribution)

• built on top of reliable services (Kinde for auth, Resend for email delivery)

All you need is an inexpensive VPS and a domain name, free Kinde and Resend accounts (that will be enough for a long time). For example with Resend you can grow your newsletter up to 1000 subscribers for free (with unlimited emails), Kinde free tier includes 10500 MAU (monthly active users).

Kinde also recently released billing feature, so I’m planning to add support for premium tiers soon, so you can start monetiziation of your newsletter with paid content.

I'm already using IndiePubStack for my personal blog and newsletter, I have a couple of blogs posts recently published here:

• https://indiepubstack.andreyfadeev.com/posts/5/getting-started-with-leiningen-a-beginners-guide

• https://indiepubstack.andreyfadeev.com/posts/3/expressive-clojure-testing-with-the-matcher-combinators-library

The future depends on the community feedback, if the open-source project gets some traction, the next logical step will be to build a SaaS platform on top of it.

If you want to support the project (or even contribute) please checkout the GitHub repository: https://github.com/IndiePubStack/IndiePubStack, stars will help a lot!

There is dedicated X (Twitter) account for the project: https://x.com/IndiePubStack

Any form of feedback will be extremely helpful, I’m working hard my evenings preparing for the ProductHunt launched that planned on 3rd September 2025: https://www.producthunt.com/products/indiepubstack

Subscribe now

If you're a developer, there's a good chance you're thinking about launching a project of your own. But what’s the best tech stack? How do you deploy it? What should you focus on? In this article, I’ll give you some practical advice to help you get started quickly—and keep a clear path for scaling later.

Infrastructure and Deployment

Keep it really simple. Get a cheap Hetzner instance and install Coolify on it. Stick with a single-instance deployment. Provide a Dockerfile for your app—Coolify will build and deploy it for you. Once you’ve got your domain name, configure it in Coolify to get HTTPS with free SSL certificates.

For storage, just use Postgres and manage it with Coolify. It takes just a couple of clicks to get it running on the same Hetzner instance. One thing I’d still recommend: configure automatic database backups. Choose a frequency that works for you—it’s also handled by Coolify, and your snapshots will be stored in AWS S3 (or any compatible object storage).

Coolify also gives you built-in CI/CD. Push your code to a private GitHub repo, connect it with Coolify, and it will build and deploy from the main branch automatically.

Language Choice

This one’s easy: pick what you're most comfortable with. You can package almost anything into a Docker image and ship it.

Personally, I’d pick Clojure for the backend. A few reasons why:

• Dynamic types – Controversial, but for a solo developer it’s a no-brainer. You’ll iterate much faster.

• Hiccup – Hands-down the best way to work with HTML templates, in my opinion.

• JVM performance – Don’t stress about it. It’s more than good enough, and if your project takes off, you can optimize later.

If I’m not planning rich frontend functionality, I stick with HTMX as long as possible. But if it starts leaning toward an SPA, I’d go with React + JavaScript to keep things straightforward.

Write High-Level Tests

I highly recommend spending time writing tests. Be pragmatic—don’t go too deep. Focus on core scenarios: happy paths, failures, edge cases.

For web apps, API-level testing usually hits the sweet spot. Set up test data, call your API, and validate both the response and the resulting DB state. You'll thank yourself later—especially when it’s time to refactor.

Monitoring and Observability

Coolify gives you basic log viewing, but it’s not enough. Set up host metrics alerts (CPU and memory usage). Also, add a simple health check that emails you if the app goes down. For exceptions, use something like Sentry to get visibility and alerts.

Use Managed Services for the Hard Stuff

Some problems are deceptively hard to solve well—authentication and payments are at the top of that list. Don’t build your own auth system unless you absolutely have to. Use a managed service like Clerk, Kinde, or Auth0. They handle password security, social logins, multi-factor auth, and session management—saving you a lot of time and potential vulnerabilities.

Same goes for payments: just use Stripe. It’s incredibly well-documented, widely supported, and gives you features like subscriptions, invoices, refunds, and webhook events out of the box. You’ll also learn a lot by integrating these services—it’s valuable experience, and it keeps your focus on building your product, not re-inventing the wheel.

Future Scaling Path

Don’t worry about scaling too early. You can always scale vertically by upgrading your instance. But if things go well and your traffic grows, you still have a clear path forward:

• Move Postgres to a managed solution (AWS RDS, Aurora, Neon, etc.)

• Use the same Dockerfile, but deploy it with a container orchestrator (AWS ECS Fargate, GCP, DigitalOcean, etc.)

• Add a load balancer in front

That’s it. Keep it simple. Avoid premature optimizations. Happy coding!

Subscribe now

Artist concept of Deep Space 1 firing its ion thrusters. The image is in public domain.

Imagine you wrote a piece of software, it was deployed, and then it turned out there is a bug in it. You want to fix it, but there is a problem: the hardware your software runs on is located at a distance of 60 million miles.

It would be a miracle if this bug could be fixed. Indeed, it was fixed, and the miracle is called Lisp. According to the developer of that software Ron Garrett, it was the features of Lisp that allowed him to rescue the Deep Space 1 probe.

I am a regular software developer in the banking and telecom sectors. But on several occasions, I faced a problem that seemed similar to that of Ron Garrett.

The problem

Sometimes you have a large application which takes a long time to build and deploy. In one bank, I had a situation where a Maven build took thirty minutes and a local deployment another twenty.

In a European telecom company, the full build and deployment took about two hours to complete.

This causes various problems.

One of them is this: If you have a large Spring-based application (tens of thousands of beans) and you need to figure out how something works, you need to:

1. modify the code,

2. rebuild and launch the application,

3. look how it behaves,

4. modify the code again, and

5. repeat steps 2–4 until you've achieved your goal.

If rebuilding and launching the application takes hours, it means you can make fewer experiments.

You could say that modifying a small piece of code in a running application takes as much time as sending a software update to a space probe at the other end of the solar system.

Alternative solutions

Sometimes you can isolate the problem in a JUnit test. Then you don't need to rebuild the entire application.

But in my case, this workaround was not feasible, because I needed to access the Spring context at runtime and inspect the data inside some of the beans.

Another approach is to use a tool like JRebel. But

• in big companies it takes months to buy a licence, and you need to fix your issue yesterday, plus

• there is no guarantee that a tool like this will work in the environment of the customer (e. g. because of firewalls and other security measures).

If you only want to change one particular Java class (and the JAR files are not signed), you could also

1. build the project locally, and

2. then replace the class file on the server.

I know at least one company which did this in production. This was of the reasons I left it.

Clojure to the rescue

One day, I was waiting for the rebuild and redeployment to complete. I remembered the story about Lisp in space.

It got me thinking: Could it be that my problem was somewhat similar to Ron Garrett's? In both cases, there was the issue of waiting time. NASA developers needed to wait for the signal to reach the probe or come back to Earth, and I was waiting for builds.

They solved the problem using Lisp. So, couldn't I solve my problem using the modern incarnation of Lisp – Clojure?

It turned out that I could. The process looks like this:

1. Create a Spring Bean with a Clojure REPL inside.

2. Put some initial code into a Clojure file inside the application.

3. Create a REST endpoint which calls the Clojure function whenever I call this endpoint in a browser.

4. Build and deploy the application.

5. Connect to the REPL from Visual Studio Code.

6. Call the endpoint and look at the results in the log file or its response.

7. Modify the Clojure code in Visual Studio Code and send the changes to the REPL.

8. Call the endpoint again.

9. Repeat steps 7 through 8 until the Clojure code behaves as you want it to.

10. Rewrite the Clojure code in Java.

In this scenario, the lengthy operation of rebuilding and redeployment takes place only once (step 4).

The changes to the Clojure code (step 7) occur at fractions of a second.

Below you can find a step-by-step tutorial on how to do this.

Tutorial

Step 1

Create a simple Spring project using the Spring Initialzr.

You can also git clone my sample project.

Step 2

Add Clojure and nREPL dependencies to the pom.xml file:

<dependency>
  <groupId>org.clojure</groupId>
  <artifactId>clojure</artifactId> 
  <version>1.10.3</version>
</dependency>

<dependency>
  <groupId>nrepl</groupId>
  <artifactId>nrepl</artifactId> 
  <version>1.0.0</version>
</dependency>

Step 3

Add a Spring bean which will launch the REPL.

package com.pravles.guestpost;

import clojure.java.api.Clojure;
import clojure.lang.IFn;
import org.springframework.stereotype.Service;

import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;

@Servicepublic
class ClojureRepl {
  @PostConstruct
  public void init() {
    final IFn require = Clojure.var("clojure.core", "require");
    require.invoke(Clojure.read("com.pravles.guestpost"));
    require.invoke(Clojure.read("nrepl.server"));
    final IFn start = Clojure.var("nrepl.server", "start-server");
    int port = 5555;
    start.invoke(Clojure.read(":port"), Clojure.read(Integer.toString(port)));
  }

  @PreDestroy
  public void shutDown() {
    Clojure.var("clojure.core.server", "stop-server").invoke(Clojure.read("{:name spring-repl}"));
  }
}

Step 4

Create a REST endpoint which calls a Clojure function and passes the Spring context as an argument.

package com.pravles.guestpost;

import clojure.java.api.Clojure;
import lombok.extern.sl4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.ApplicationContext;
import org.springframework.web.bind.annotations.GetMapping;
import org.springframework.web.bind.annotations.RequestMapping;
import org.springframework.web.bind.annotations.RestController;

@Slf4j
@RestController("newDebugController")
@RequestMapping("debug")
public class DebugController {
  @Autowired
  private ApplicationContext ctx;

  @GetMapping("run")
  public Object run() {
    return Clojure.var("com.pravles.guestpost", "debug").invoke(ctx);
  }
}

Step 5

Create a file src/main/resources/com/pravles/clojureguestpost.clj with the initial Clojure code.

#_{:clj-kondo/ignore [:namespace-name-mismatch]}

(ns com.pravles.guestpost
  (:require [clojure.pprint :refer [pprint]])
  (:require [clojure.string])
  (:require [clojure.java.io]))

(import javax.servlet.http.HttpServletResponse)

(defn debug [ctx] "debug return value")

Step 6

Add a bean with one method that capitalizes a string:

@Componentpublic
class UpperCaseBean {
  public String upperCase(final String input) {
    if (input == null) {
      return "";
    }
    return input.toUpperCase();
  }
}

Step 7

Build and run the application.

Step 8

Open the URL http://localhost:8080/debug/run in the browser. You should see the text "debug return value" there.

Step 9

Install VS Code, if necessary.

Step 10

Install the VS Code extension Calva.

Step 11

Create a new project using Leiningen (lein new app) and open it in VS Code.

To do, select the menu item File -> Open folder.

There, select the directory with the application created by Leiningen.

Step 12

Paste the contents of file from step 5 into the file src/app/core.clj in VS Code.

Copy the file from step 4 into one of the folders of the project from the previous step.

Step 13

Connect VS Code to the REPL inside your Spring project.

To do, select the menu item View -> Command Palette.

A little text window will appear. Type Calva there, then select the item Connect to a running REPL server....

In the next window, select the Generic REPL type.

In the next window, enter localhost:5555 as the address of the REPL server.

After a few seconds you should see the message: Connected session: clj on the right side of the window.

Step 14

Modify the file from step 12 as follows.

#_{:clj-kondo/ignore [:namespace-name-mismatch]} 

(ns com.pravles.guestpost
  (:require [clojure.pprint :refer [pprint]])
  (:require [clojure.string])
  (:require [clojure.java.io]))

(import javax.servlet.http.HttpServletResponse)

(defn debug [ctx] (-> java.util.Date (new) (.toString)))

Press Ctrl-A to select the contents of the file. Press Ctrl-Enter. This will send the changes to the function debug to the REPL in your Spring application.

You should see a text like => #'com.pravles.clojureguestpost/debug at the bottom of the window with the code upon successful update.

Step 15

Reload the page from step 6. Now you should see the current date and time in your browser.

This means that you've changed the code of your Spring application without rebuilding and redeploying it.

Step 16

Now let's retrieve a reference to our bean from step 6 from within Clojure. An instance of org.springframework.context.ApplicationContext is passed into our function. We can use its getBean method to retrieve the reference to the bean via its name.

bean (-> ctx (.getBean "upperCaseBean"))

Then we can call its upperCase method and return the result:

#_{:clj-kondo/ignore [:namespace-name-mismatch]}

(ns com.pravles.clojureguestpost
  (:require [clojure.pprint :refer [pprint]])
  (:require [clojure.string])
  (:require [clojure.java.io]))

(defn debug [ctx]
  (let [date (-> java.util.Date (new) (.toString)) 
        bean (-> ctx (.getBean "upperCaseBean"))]
    (str "bean: " (-> bean (.upperCase "foo")))))

Press Ctrl-A and Ctrl-Enter to send the update to the Spring application.

Then, reload the URL http://localhost:8080/debug/run in the browser. You should see an output like this:

Conclusion

Now you know how to harness the power of Clojure in legacy projects. Did you use Clojure to work with legacy software in a better way? Share your thoughts in the comments.

About me

I work as a software developer during the day so I can work on my novel during the evenings. Visit pravles.substack.com to read completed parts of my novel for free. If you are interested in how writing fiction helped me become a better software developer, please leave a comment.

class Example {
  
  private int x = 0;
  
  public int increment() {
    this.x = this.x + 1;
    return x;
  }
}

It’s okay until you try to use it in concurrent environment, from different threads. In that case we have a classic race condition, increment is not atomic, two threads can read value at the same time and one one increment will be written.

There are ways to improve it, for example, we can use synchronized keyword on the method name, which will always attempt to acquire a lock on the object and force a single-thread execution of the method.

public synchronized int increment() {
    this.x = this.x + 1;
    return x;
}

Also, we can use explicit locks, like ReentrantLock:

 class X {
   private final ReentrantLock lock = new ReentrantLock();
   // ...

   public void m() {
     lock.lock();  // block until condition holds
     try {
       // ... method body
     } finally {
       lock.unlock()
     }
   }
 }

Or alternatively, if we want a thread-safe way to work with an Integer we can completely swap our own implementation AtomicInteger:

import java.util.concurrent.atomic.AtomicInteger;
  
AtomicInteger counter = new AtomicInteger(0);
counter.incrementAndGet();

There are other classes available in the import java.util.concurrent.atomic.* package. The backbone of that is the Compare And Set (CAS) operation: Java will rely on the underlying platform processor instructions, for example, CMPXCHG on x86/x64 to atomically swap the variable value.

So how is this applicable to our topic, Clojure Atoms?

Let’s check Atom semantics, we can define an atom containing any value type, from primitives to data structures like vectors, sets and maps, to update those values we use swap! function, it is guaranteed to be thread-safe.

(def x (atom 0))
(swap! x inc) 
=> 1
(deref x) 
=> 1
@x 
=> 1

Note that to get value from an atom there is deref function, @ is the short way of writing deref.

As I’ve said we can use data structures inside atom container as well:

(def x (atom [1 2 3]))

(swap! x (fn [v] (mapv inc v)))

=> [2 3 4]

Let’s look inside of the atom swap! implementation:

(defn swap!
  "Atomically swaps the value of atom to be:
  (apply f current-value-of-atom args). Note that f may be called
  multiple times, and thus should be free of side effects.  Returns
  the value that was swapped in."
  {:added "1.0"
   :static true}
  ([^clojure.lang.IAtom atom f] (.swap atom f))
  ([^clojure.lang.IAtom atom f x] (.swap atom f x))
  ([^clojure.lang.IAtom atom f x y] (.swap atom f x y))
  ([^clojure.lang.IAtom atom f x y & args] (.swap atom f x y args)))

import java.util.concurrent.atomic.AtomicReference;

public final class Atom extends ARef implements IAtom2 {
  final AtomicReference state;

  public Object deref() {
    return this.state.get();
  }

  public Object swap(IFn f) {
    Object v;
    Object newv;
    do {
      v = this.deref();
      newv = f.invoke(v);
      this.validate(newv);
    } while(!this.state.compareAndSet(v, newv));

    this.notifyWatches(v, newv);
    return newv;
  }

We can see the usage of AtomicReference, that’s basically how our object is stored in the atom. Deref is simple, we just call get on the AtomicReference:

public Object deref() {
    return this.state.get();
}

In the swap we see a common pattern: Compare And Set Loop. The idea is simple:

• get current value

• calculate the next value (the invoke call of user-provided function with the current value as the input)

• try to apply changes with compareAndSet

But why do we need the loop? CompareAndSet will return true in case of success, but there is still a possibility that other threads changed the value in between! That’s why we need to retry that until we get lucky and our update is applied:

    /**
     * Atomically sets the value to {@code newValue}
     * if the current value {@code == expectedValue},
     * with memory effects as specified by {@link VarHandle#compareAndSet}.
     *
     * @param expectedValue the expected value
     * @param newValue the new value
     * @return {@code true} if successful. False return indicates that
     * the actual value was not equal to the expected value.
     */
    public final boolean compareAndSet(V expectedValue, V newValue) {
        return VALUE.compareAndSet(this, expectedValue, newValue);
    }

Due to this retry logic - it’s clear that you shouldn’t add side-effects into the swap fn, as you can see there is no guarantee that it will be executed once. It’s only guaranteed that the result will be updated in a thread-safe way.

Also, note that we have this line:

this.notifyWatches(v, newv);

It’s a watcher feature of the Atom, not commonly used or known but pretty cool.

(def a (atom {}))

(add-watch a :watcher
  (fn [key atom old-state new-state]
    (println "Atom value changed)))

Now the watcher fn will be triggered each time the value inside the atom is changed, it’s possible to add multiple watchers as well.

Hope that was useful!

Coming from Java, I was finally free from OOP, classes, interfaces and inheritance. It was like a miracle to realize that I only needed a vector, a map and a set to model everything.

Lists are everywhere in Clojure, it’s basically a building block of the LISP program. They could be used as a data structure as well, under the hood it’s just a linked list, but usually, a vector is a better choice for data modelling.

Vector []

In simple terms, you can think about a vector as an array. Same properties: fast access by index, but requires shifting elements if inserted in the middle. Technically all Clojure data structures are trees (to implement persistent immutable data structures), so instead of O(1) for index access it is O(log32n) but realistically could be neglected.

Here we have a simple vector with elements of the same type:

[1 2 3]
["a" "b" "c"]

It doesn’t have to be the case all the time, here is a vector with mixed types and also a nested vector:

[1 "a" [2 3 nil]]

Read more

I’ve been using Tailwind for multiple years now and it’s my go-to way of styling web applications. Tiny, composable classes are great for both React components and for Clojure Hiccup components. The issue that in Clojure environment I was really lazy and just used CDN link to add Tailwind to my hobby projects. That’s worked but it’s not ideal, size of the file is large and it’s hard to configure.

I’ve recently explored a proper way of using Tailwind and was surprised, it’s not much harder to configure it to work with Clojure backend compared to React and Javascript.

First of all, as I usually don’t have any Node in my Clojure projects, I didn’t really want to install it just for Tailwind. Luckily, there is a standalone Tailwind CLI available: https://tailwindcss.com/blog/standalone-cli

I’ve used Homebrew to install:

brew install tailwindcss

The next step is to run init command in the root of the project:

tailwindcss init

The result of this command is a new file `tailwind.config.js` that will be used for additional customisations.

The idea of Tailwind build step is to search source files for known Tailwind CSS classes and it’s language agnostic, it will basically search for a string like literals that match class names.

Let’s configure CLI to search for classes in Clojure source files:

content: [
    "./src/**/*.{clj,cljs,cljc}"
]

In addition to that we will need an `input.css` file with a couple of imports:

@tailwind base;
@tailwind components;
@tailwind utilities;

That’s basically all we need, let’s run the build:

tailwindcss -i input.css -o ./resources/public/css/output.css

It should discover all Tailwind classes and generate much smaller result CSS file.

I usually use Babashka as my task runner, so there is an example of adding tasks to build, minify or watch Tailwind, `bb.edn`:

{:tasks
 {:init (do (defn sh [& args]
              (binding [*out* *err*]
                (apply println "+" args))
              (apply shell args))
            (defn tailwindcss
              [cmd]
              (sh (str "tailwindcss -i input.css -o ./resources/public/css/output.css " cmd))))
  
  tailwind:watch {:doc "Build TailwindCSS in watch mode"
                  :task (tailwindcss "--watch")}
  tailwind:build {:doc "Build TailwindCSS output"
                  :task (tailwindcss nil)}
  tailwind:minify {:doc "Build & minify TailwindCSS"
                   :task (tailwindcss "--minify")}}}

Now, for example, in development we can just start watch mode:

bb tailwind:watch

Other customisations

I’ve tried a couple of things, as setting custom container width and adding other fonts.

For the font, the easiest is to add it as import into the `input.css` file, I’ve used one from Google Fonts.

module.exports = {
  content: [
    "./src/**/*.{clj,cljs,cljc}"
  ],
  theme: {
    extend: {
      container: {
        center: true,
        padding: '1rem',
        screens: {
          DEFAULT: '600px',
        },
      },
      fontFamily: {
        sans: ['Titillium Web',  'sans-serif'],
      },
    },
  },
  plugins: [],
};

Dynamic classes

I think it’s best to avoid dynamic class names, but if you have one of those, for example some string concatenation to generate class name — one way to solve it is to use `safelist` configuration:

module.exports = {
  safelist: ["bg-red-500"]
};

If you are looking for a full example, I’ve added this Tailwind integration into my Clojure service template: https://github.com/andfadeev/clojure-service-template

Subscribe now

I was looking for a cheap and convenient way to improve deployment for my hobby projects. My current setup is a DigitalOcean Droplet, which costs me ~8$/month. Inside I have Docker and Docker Compose that run: a Clojure application, a PostgreSQL database, an additional container that does database S3 backups on a schedule and Nginx as reverse proxy which also is configured to generate and renew LetsEncrypt certificates. That actually worked for me for many years, but there are a couple of things I don’t like:

• The cost of DigitalOcean Droplet is quite high for the resources.

• My deployment process is quite manual: GitHub Actions pipeline is building a docker image for me and after I need to manually pull the repo on the host with `docker-compose.yaml` and restart the container.

On the other hand, the Hetzner Cloud offering is appealing. They have the CX22 plan: 2 vCPU, 4GB RAM, 40TB for €3.79/month. Also, you will probably what to pay for IPv4 on top of that.

But I didn’t really want to replicate my poor deployment pipeline, so I’ve looked for alternatives.

Coolify was on my radar for quite a while, but the thing I didn’t understand is how many features it has. I saw it advertised as a self-hosted Vercel alternative and without looking into much detail I assumed that it’s Javascript-focused - I was wrong.

You can literally run anything - I was able to deploy a Clojure application without any modifications.

To build and deploy your application you can still use Docker, but by default, it’s using Nixpacks. It was new to me and better read official docs (https://nixpacks.com/docs), but It has Clojure support: https://nixpacks.com/docs/providers/clojure by just detecting a `project.clj` file in your repo.

Let’s talk about installing Coolify - it is really simple and takes just a couple of minutes and a single command to run:

curl -fsSL https://cdn.coollabs.io/coolify/install.sh | bash

You need to run it from your Hetzner host and it’s probably the last time you need to SSH there. At the end of the installation, you’ll get a URL for the Coolify UI running on the host, everything else is done from there.

Running PostgreSQL

My application requires a PostgreSQL database, so that was the first thing I’ve tried. You can basically deploy a PostgreSQL instance from the UI with a couple of clicks.

My instance was up and running in no time. Extra features I really liked:

• I was able to expose my database port to the outside (make it publically available for a short period), so I could easily transfer data from my old database with pg_dump and the psql restore.

• You can configure database backups from Coolify, so you don’t need extra tools: the destination could be most of the S3-compatible object stores.

Deploying the Clojure application

I have a private GitHub repo with my Clojure application. The way you configure the CI pipeline for it in Coolify is by creating a GitHub application with permission to read the repo, it will also automatically receive webhooks for new commits in the main branch and trigger new deployments.

The other killer feature is the preview branches on PRs, I don’t really use it in my project, but I definitely see the value.

So the build process as I’ve mentioned is using Nixpacks by default and it actually worked without problems for the Clojure application, it detected the environment like so:

One thing to mention, I got an issue with the Java runtime version in another app, but the fix was to set the build environment variable to configure Nixpack Clojure:

NIXPACKS_JDK_VERSION=latest

Coolify is also solving the problem of storing secrets, instead of setting them manually on the host or storing them in GitHub private repo (as I used to do) — you just set them in UI (similar to Vercel):

Also, you get a nice you to view logs and you can configure log drain to a 3rd party (like NewRelic):

Other features I’ve tried are:

• Configuring custom domain

• Basic built-in host monitoring (CPU, Memory, DiscUsage)

• Email notifications (you need a Resend API key for that)

• Push notification to Telegram Bot

The most impressive thing really is that during my experiments I encountered 0 issues (except that one with Java runtime version, which I solved in 10 minutes by reading Nixpacks docs and setting one extra var).

Well done, Coolify!

P.S.

I’ve not switched my project to run on the Coolify Hetzner host, but I’ll run a replica of it for a while, check that everything is stable — and hopefully switch!

That should save me a couple of USD per month and improve the developer experience :)

Subscribe now

In August 2023 I got an idea to start a YouTube channel.

Time flies, the whole year passed and it’s time to celebrate the first anniversary and sum up the results.

During the year I’ve learned quite a lot of new skills:

• how to plan and prepare videos

• how to set the environment (recording tools like OBS), camera settings, lightning

• basic video editing (thanks for a great free DaVinci Resolve software)

• making video thumbnails with Canva

• video recording process and talking to the camera (there is a lot to improve here)

The journey was extremely exciting and I’ve learnt a lot. I think the most important lesson is to be consistent and optimize the process. Videos are obviously not my main job so I need to fit that somehow into my already quite busy life. The lesson I’ve learnt is not to pursue perfection, and not to spend a lot of time editing. It’s really hard to stay motivated when you put a lot of effort into a video and still get low numbers of views, but it’s a marathon, not a sprint.

In the end, I’ve really streamlined the recording process and spent minimum time editing — that really helped to stay consistent and release content.

To be honest, consistency could be better at some periods — that’s something I’ll try to fix this year.

Let’s move to current stats:

• 3012 subscribers

• 154000 total views

• 9500 watch hours

• 75 uploads

In addition, I’ve got a Substack publication with 103 subscribers, grown my LinkedIn to 900 followers and my Telegram channel is growing as well!

Subscribe now

Path to monetization

One of the goals I had when started was to try to get to a YouTube partnership and achieve monetization on the channel. If you are not familiar with the terms — your channel needs to reach some milestones to be able to get part of the revenue from ads. Frankly speaking, the requirements are not easy:

• 1000 subscribers

• 4000 watch hours (total time your video was watched on YouTube)

So it was quite challenging. I was reading some stories about getting to monetization on the internet and depending on a niche and channel-specific both cases are valid: for some people, it’s easier to get 4000 watch hours (some entertainment or viral content) but it’s really hard to gain subscribers, for others it’s opposite. In my case, I’ve reached 1000 subscribers much faster than 4000 watch hours.

I got 1000 subscribers in November, but 4000 watch hours only in mid-February. So the entire journey to monetisation took 5 months (from mid-August to mid-February).

For most people, the total revenue will be an interesting topic I believe and I would say you won’t get rich with a small channel and videos that are not getting tons of views and not going viral.

The total from ads is £151 since monetization was enabled and it’s less than £1 a day.

To be honest, YouTube ads revenue wasn’t the only source of income from my blogging this year. I’ve got a bunch of donations on the Buy Me A Coffee resource and a paid subscription here on Substack. Those two together are a bit more successful compared to YouTube gains. It’s also a great feeling to receive donations with kind words and see that people are grateful for some videos. I really hope some of the topics were really useful!

The most unexpected experience was a request for a sponsored video.

Final thoughts

I’d like to say thank you to everyone who watched my videos, subscribed to the channel, left comments and made donations! That was a constant source of inspiration during the year.

Also if you were thinking of starting a blog or a channel, I highly recommend just starting it, it’s not that scary and you don’t need any planning or preparation! It definitely has a lot of benefits and is a constant source of motivation to learn new things.

Looking forward to the second year on YouTube!

Subscribe now

It feels like the right time to make yourself familiar with Java Virtual Threads and here are a couple of reasons:

• In JDK 21, the Virtual Threads feature is out of preview and enabled by default

• Looks like we will have a Clojure 1.12 release soon (let’s see why it is important later in this article)

• Ring release supports Jetty 12 (which has support for Virtual Threads)

Subscribe now

Virtual Threads vs Platform Threads

If you want to learn more about virtual threads and all their whys and hows, I highly recommend this article: https://blog.rockthejvm.com/ultimate-guide-to-java-virtual-threads/

Virtual threads allow us to follow a simple pattern: to create a new thread for every concurrent task. This model is called one task per thread.

But why it’s not desired with old Java threads (platform threads) — the answer is that they are expensive in many ways. One way to see it is to try to create multiple platform threads in a loop:

(defn with-platform-threads
  []
  (dotimes [i 100000]
    (.start (Thread. (fn []
                       (Thread/sleep 1000)))))
  :done)

(with-platform-threads)

It’s quite easy to reach a message like this:

Execution error (OutOfMemoryError) at java.lang.Thread/start0 (Thread.java:-2).
unable to create native thread: possibly out of memory or process/resource limits reached

On JDK 21 (in previous releases we need to enable the preview feature) we can use virtual threads and try the same:

(defn with-virtual-threads
  []
  (dotimes [i 100000]
    (.start (.name (Thread/ofVirtual) "virtual-thread-")
            (fn [] (Thread/sleep 1000))))
  :done)

(with-virtual-threads)
:done

The other benefit is that if a virtual thread is blocked on IO, it will be unmounted from the platform (carrier) thread — and a pending virtual thread could reuse the same platform thread. As you can see we can efficiently reuse a limited pool of platform threads while keeping the virtual threads number high!

Ring and Jetty 12

Jetty 12 could be configured with a pool of virtual threads to handle requests, and it can significantly improve the server performance in case of lots of IO operations in handlers.

;; project.clj 

(defproject jetty-ring-virtual-threads "0.1.0-SNAPSHOT"
  :dependencies [[org.clojure/clojure "1.11.1"]
                 [ring/ring "1.12.2"]]
  :main ^:skip-aot jetty-ring-virtual-threads.core
  :target-path "target/%s")

;; jetty-ring-virtual-threads.core

(defn handler [_]
  (Thread/sleep 50)
  {:status 200
   :headers {"Content-Type" "text/plain"}
   :body "Hello World"})

(defn -main []
  (println "Starting Jetty Server")
  (jetty/run-jetty handler {:port 3000}))

;; and let's run it!
(-main)

To do a simple benchmarking of the server I’ll use `wrk` tool https://github.com/wg/wrk:

> wrk -t12 -c400 -d30s http://localhost:3000

Running 30s test @ http://localhost:3000
12 threads and 400 connections
25625 requests in 30.06s, 3.79MB read
Socket errors: connect 0, read 1303, write 0, timeout 0
Requests/sec: 852.39
Transfer/sec: 129.02KB

We got 850 Request per second with Jetty 12 on platform threads, now let’s configure it to use virtual threads:

(defn -main
  []
  (println "Starting Jetty Server (Virtual Threads)")
  (let [thread-pool (QueuedThreadPool.)]
    (.setVirtualThreadsExecutor
      thread-pool (Executors/newVirtualThreadPerTaskExecutor))
    (jetty/run-jetty handler {:port 3000
                              :thread-pool thread-pool})))

Everything else stays the same — time to rerun the benchmark:

> wrk -t12 -c400 -d30s http://localhost:3000

Running 30s test @ http://localhost:3000
12 threads and 400 connections
214960 requests in 30.10s, 31.78MB read
Socket errors: connect 0, read 1251, write 0, timeout 0
Requests/sec: 7140.46
Transfer/sec: 1.06MB

So we have almost 10x improvement — 7100 RPS is impressive!

Pinned virtual threads

So now let’s understand why future Clojure 1.12 release is important — and the reason is to avoid virtual threads pinning.

There are some cases where a blocking operation doesn’t unmount the virtual thread from the carrier thread, blocking the underlying carrier thread. In such cases, we say the virtual is pinned to the carrier thread. It’s not an error but a behavior that limits the application’s scalability. Note that if a carrier thread is pinned, the JVM can always add a new platform thread to the carrier pool if the configurations of the carrier pool allow it.

Fortunately, there are only two cases in which a virtual thread is pinned to the carrier thread:

• When it executes code inside a synchronized block or method;

• When it calls a native method or a foreign function (i.e., a call to a native library using JNI).

In Clojure delay and lazy-seq are using synchronized blocks, let’s check clojure.lang.Delay internals:

  public Object deref() {
    if (this.fn != null) {
      // this is the problem
      synchronized(this) {
        if (this.fn != null) {
          try {
            this.val = this.fn.invoke();
          } catch (Throwable var4) {
            Throwable t = var4;
            this.exception = t;
          }

          this.fn = null;
        }
      }
    }

    if (this.exception != null) {
      throw Util.sneakyThrow(this.exception);
    } else {
      return this.val;
    }
  }

Let’s change our handler to demonstrate the problem:

(defn handler [_]
  @(delay
    (Thread/sleep 50))
  {:status 200
   :headers {"Content-Type" "text/plain"}
   :body "Hello World"})

Technically nothing changed, we still sleep for 50ms but it’s wrapped in a delay that immediately derefed, but now we have a blocking call inside synchronized block — let’s run the benchmark:

> wrk -t12 -c400 -d30s http://localhost:3000

Running 30s test @ http://localhost:3000
12 threads and 400 connections
12534 requests in 30.06s, 1.85MB read
Socket errors: connect 0, read 1272, write 0, timeout 2844
Requests/sec: 416.90
Transfer/sec: 63.10KB

As you can see the result is much worse, but it’s on a first run after the server start, the sequential result is giving similar results (7000 RPS). I think it’s because the internal platform pool is scaled to max allowed size during the first test run — but I need to investigate more to be sure.

Also by enabling -Djdk.tracePinnedThreads=full I was able to get some errors about virtual thread pinning:

Thread[#148,ForkJoinPool-1-worker-3,5,CarrierThreads]
    java.base/java.lang.VirtualThread$VThreadContinuation.onPinned(VirtualThread.java:183)
    java.base/jdk.internal.vm.Continuation.onPinned0(Continuation.java:393)
    java.base/java.lang.VirtualThread.parkNanos(VirtualThread.java:621)
    java.base/java.lang.VirtualThread.sleepNanos(VirtualThread.java:791)
    java.base/java.lang.Thread.sleep(Thread.java:507)

Clojure 1.12

In future Clojure release there will be changes related to delays and lazy-seqs: https://github.com/clojure/clojure/blob/master/changes.md

1.2 Java 21 - Virtual thread pinning from user code under synchronized

Clojure users want to use virtual threads on JDK 21. Prior to 1.12, Clojure lazy-seqs and delays, in order to enforce run-once behavior, ran user code under synchronized blocks, which as of JDK 21 don't yet participate in cooperative blocking. Thus if that code did e.g. blocking I/O it would pin a real thread. JDK 21 may emit warnings for this when using -Djdk.tracePinnedThreads=full.

To avoid this pinning, in 1.12 lazy-seq and delay use locks instead of synchronized blocks.

Let’s update the version in the project.clj:

[org.clojure/clojure "1.12.0-rc1"]

Now if we go to Delay implementation — we will see the usage of locks instead of synchronized blocks:

this.lock = new ReentrantLock();

And let’s rerun our benchmark again:

> wrk -t12 -c400 -d30s http://localhost:3000

Running 30s test @ http://localhost:3000
12 threads and 400 connections
214040 requests in 30.11s, 31.64MB read
Socket errors: connect 0, read 1240, write 0, timeout 0
Requests/sec: 7109.61
Transfer/sec: 1.05MB

Alghtouth I still have some concerns about the benchmarking results (especially in the case when pinning should occur), clearly virtual threads in Jetty are definitely worth exploring — I’m looking forward to try that on some real applications soon!

Subscribe now

Read more

Welcome to Chapter II of the Clojure Getting Started Roadmap series. Before diving into technical details, the core of the language, and libraries, let’s discuss why you might choose Clojure as your next programming language and consider any potential downsides.

It's not just about the technology; we'll also explore the impact of picking a niche language like Clojure on your software career progression.

Subscribe now

As of 2024, Clojure remains a niche language, but it's far from new. Its first public release was in 2007, marking 17 years since its debut. It's also not just a "hipster" language; it has proven its worth across many domains and attracts developers seeking better alternatives to mainstream languages. The Nubank story is impressive—Clojure-based neo-bank that acquired Cognitect, the company behind Clojure.

Why Learn Clojure in 2024?

Read more

The goal of this roadmap is to provide step-by-step instructions on how to start with Clojure in 2024, highlighting the most important areas to focus on.

We’ll cover the core aspects of the language as well as the essential tools and libraries you need to become familiar with. Clojure's unique nature requires a mindset shift to be productive and to work harmoniously with the language, rather than against it.

Clojure is a hosted language that can run on multiple platforms. While we’ll primarily focus on the JVM and scripting with Babashka, much of the content will also be applicable to front-end development with ClojureScript.

To clarify, this is not intended to be an exhaustive book about Clojure. Instead, it’s a practical guide on what to focus on when starting out. With seven years of full-time professional experience in Clojure, I've gained extensive hands-on knowledge of the language and its ecosystem.

This guide is what I wish I had when I was starting out, and I hope it will be incredibly useful to everyone who reads it!

This is what to expect in the following chapters:

• Why Clojure?

• Getting Started with Clojure

  • Setting Up Your Development Environment

  • Your First Clojure Program

• Clojure Fundamentals

  • Basic Syntax

  • Data Types (Numbers, Strings, Collections)

  • Functions

Most of the chapters will be behind a paywall, accessible to all paid subscribers on Substack. Additionally, all paid subscribers will receive the final version of the book in EPUB or PDF format once it’s completed.

The final version will also be available as a paid digital download on various platforms, so stay tuned for updates!

I plan to release a chapter each week, though it’s challenging to predict the exact schedule. I promise to do my best!

Starting with basics

It’s important to start with the basics and make yourself comfortable with the language and most importantly with the workflow. The goal is to create an environment where you can quickly iterate and try new code.

Subscribe now

To be clear, I’m not suggesting spending much time or reading a huge book. I think it should be the opposite, you should minimize the time until you write your first line of code and make sure you can easily iterate and learn more.

Steps here:

• Setup the editor so it’s helpful (for most people it will probably be VSCode, but I was exploring Zed recently and found it’s good enough to work with Go with `gopls` LSP)

• Create your first module and make sure you can run it. Classic “Hello World” from the main function is what we need.

• Trying code from `main` could be challenging as you go, so we need a better way. So here we should learn how to create unit tests, so we can test any function that we write.

And here was the first discovery, built-in features of the language regarding tests and not that great, so you probably will need a 3rd party library for assertions. Feels like the `testify` package is a popular one.

import (
  "testing"
  "github.com/stretchr/testify/assert"
)

func TestSomething(t *testing.T) {
  assert.Equal(t, "expected", YourFnToTest(1), "should work!")
}

Once we are here, you can spend a bit of time trying different things, solving some problems, etc. But to get a real feel of the language we need to check those areas:

• create a web server

• define routes

• working with JSON (serialize and deserialize)

• querying relational database

• making HTTP requests

Of course in reality there is more, but those are the most commonly used things in back-end development, so by exploring all those areas - you’ll get a good understanding of the language ecosystem.

Web server

Being able to create a web server, and register some endpoints will be a huge leap forward. Now our application will start to feel real!

The first step is to check how it looks to create a web server without any 3rd party libraries, and due to the `net/http` package, it’s really easy in Go!

package main

import (
    "fmt"
    "net/http"
)

func hello(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, "hello\n")
}

func main() {
    http.HandleFunc("/hello", hello)
    http.ListenAndServe(":8090", nil)
}

It’s already something! The next question is how we create a more complex application, here we will need a router.

I’ve done a bit of research on what are popular libraries and frameworks in this area, here are some: Gin, Echo, Fiber, Chi, Gorilla.

As I come from a Clojure background, I really hate heavy and overcomplicated frameworks with lots of magic.

So my choice here so far (and remember I’m only starting my Go journey) is Chi: https://github.com/go-chi/chi. It is small, basically just a router, but at the same time is flexible, by allowing different sets of middleware for groups of routes.

package main

import (
    "net/http"

    "github.com/go-chi/chi/v5"
    "github.com/go-chi/chi/v5/middleware"
)

func main() {
    r := chi.NewRouter()
    r.Use(middleware.Logger)
    r.Get("/", func(w http.ResponseWriter, r *http.Request) {
        w.Write([]byte("Hello World!"))
    })
    http.ListenAndServe(":3000", r)
}

OK, that’s something! Unfortunately, nowadays we don’t talk plain text, so the next area to focus on is working with JSON, how to serialize and deserialize data.

Working with JSON

Again, Go has a standard package: encoding/json (https://gobyexample.com/json)

type User struct {
  ID   int64  `json:"id"`
  Name string `json:"name"`
}

user := User{
  ID: 1, 
  Name: "Andrey",
}

jsonText, _ := json.Marshal(user)

fmt.Println("As JSON", string(jsonText))

As I’ve picked Chi as a web component library, it makes sense, what is the recommended way to return JSON from handlers, there is a render package available: https://github.com/go-chi/render.

func (u *User) Render(w http.ResponseWriter, r *http.Request) error {
 return nil
}

func GetUser(w http.ResponseWriter, r *http.Request) {
  user := User{
  ID:   1,
  Name: "Andrey",
}
  render.Status(r, http.StatusOK)
  render.Render(w, r, &user)
}

func main() {
  r := chi.NewRouter()
  r.Use(middleware.Logger)

  r.Get("/user", GetUser)
  
  http.ListenAndServe(":8787", r)
}

Working with SQL

SQL and relational databases are still the most commonly used persistent storage nowadays, so it’s crucial to know how to work with them from any language.

I’ll be using PostgreSQL for all examples. To work with it we will need an SQL package https://pkg.go.dev/database/sql and a driver https://github.com/lib/pq.

And while this should work for simple applications:

err := pool.QueryRowContext(ctx, "select p.name from people as p where p.id = :id;", sql.Named("id", id)).Scan(&name)

We really need a better way. But, I hate ORMs! I think this is a trauma from my old Java days with Hibernate. In my opinion, ORMs just bring so much magic and complexity without much value, it’s usually an application that is too small and simple for an ORM or already too complex.

I was looking for some kind of SQL query builder library and found https://github.com/go-jet/jet. It really reminded me of a type-safe Java library jOOQ, which I really enjoyed many years ago!

It’s not an ORM but it relies on code generation, it will generate tables and model definitions in Go based on your database schema. After that, you can write type-safe queries and benefit from auto-completion and additional checks from the compiler.

stmt := SELECT(
    Actor.ActorID, Actor.FirstName, Actor.LastName, Actor.LastUpdate,  
    Film.AllColumns,                                                  
    Language.AllColumns.Except(Language.LastUpdate),
    Category.AllColumns,
).FROM(
    Actor.
        INNER_JOIN(FilmActor, Actor.ActorID.EQ(FilmActor.ActorID)).  
        INNER_JOIN(Film, Film.FilmID.EQ(FilmActor.FilmID)).          
        INNER_JOIN(Language, Language.LanguageID.EQ(Film.LanguageID)).
        INNER_JOIN(FilmCategory, FilmCategory.FilmID.EQ(Film.FilmID)).
        INNER_JOIN(Category, Category.CategoryID.EQ(FilmCategory.CategoryID)),
).WHERE(
    Language.Name.EQ(String("English")).             
        AND(Category.Name.NOT_EQ(String("Action"))).  
        AND(Film.Length.GT(Int(180))),               
).ORDER_BY(
    Actor.ActorID.ASC(),
    Film.FilmID.ASC(),
)

Looks nice!

I’ve used a database schema provided (https://github.com/go-jet/jet-test-data/blob/master/init/postgres/dvds.sql) to create my local database. And generated Go code from it:

jet -dsn=postgresql://user:pass@localhost:5432/jetdb?sslmode=disable -schema=dvds -path=./.gen

Finally, it’s time to link our web application with the database, so a handler should make a call to the database and return data as JSON.

A question I usually have at this point is how to pass dependencies to handlers, it is just global state or any dependency injection patterns. I’ve picked in my example just a global var to store my database connection handler, but I’ve found this link really useful to get the feel of other approaches: https://www.alexedwards.net/blog/organising-database-access

var DB *sql.DB

type FilmResponse struct {
	*model.Film
}

func (f *FilmResponse) Render(w http.ResponseWriter, r *http.Request) error {
	return nil
}

func GetFilm(w http.ResponseWriter, r *http.Request) {
	stmt := SELECT(
		Film.AllColumns,
	).FROM(
		Film,
	).WHERE(
		Film.Length.GT(Int(150)),
	).ORDER_BY(
		Film.Title.ASC(),
	)

	var films []struct {
		model.Film
	}

	err := stmt.Query(DB, &films)
	if err != nil {
		log.Fatal(err)
	}

	firstFilm := films[1]

	render.Status(r, http.StatusOK)
	render.Render(w, r, &FilmResponse{&firstFilm.Film})
}


func main() {

         // I'm using TestContainers to start PostgreSQL
         // see the code snippet at the end of the blog post

	DB, err = sql.Open("postgres", connStr)

	if err != nil {
		log.Fatal(err)
	}

	r := chi.NewRouter()
	r.Use(middleware.Logger)

	r.Get("/film", GetFilm)

	http.ListenAndServe(":8787", r)
}

Making HTTP requests

The final bit for the first round of learning is to figure out how to make HTTP requests. Again, looks like we have a standard package, so it’s simple!

Also, I’ve decided to wrap that in a new handler:

r.Get("/proxy", func(w http.ResponseWriter, r *http.Request) {
	resp, err := http.Get("https://pokeapi.co/api/v2/pokemon/ditto")
	if err != nil {
		log.Fatal(err)
	}

	body, err := io.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}

	w.Write([]byte(string(body)))
})

Final thoughts

I think an exercise like this will give you a good overview of the language and the ecosystem as well as hands-on experience. Of course, there is more to learn: concurrency, non-relational databases, working with AWS, etc. Also usually I prefer to read one or two books just to have more structured knowledge.

Bonus

I’m really enjoying working with TestContainers and Go is supported (https://golang.testcontainers.org/). I wasn’t using TestContainers for tests here, but just to avoid writing Docker Compose files to start the local database, I’ve done it from code in the main function:

ctx := context.Background()

dbName := "jetdb"
dbUser := "jet"
dbPassword := "jet"

postgresContainer, err := postgres.RunContainer(ctx, testcontainers.WithImage("docker.io/postgres:16-alpine"),
	postgres.WithInitScripts("dvds.sql"),
	postgres.WithDatabase(dbName),
	postgres.WithUsername(dbUser),
	postgres.WithPassword(dbPassword),
	testcontainers.WithWaitStrategy(
		wait.ForLog("database system is ready to accept connections").
		WithOccurrence(2).
		WithStartupTimeout(30*time.Second)),
	)

if err != nil {
	log.Fatalf("failed to start container: %s", err)
}

connStr, err := postgresContainer.ConnectionString(ctx, "sslmode=disable", "application_name=test")
fmt.Println("PostgreSQL connection string", connStr)

DB, err = sql.Open("postgres", connStr)

I’ve recently seen some posts on LinkedIn about managing multiple Java versions with things like sdkman or jenv. Don’t get me wrong, those tools are doing the job well and really useful. But I just thought about how many people are still using different version managers from different tools (sdkman, nvm, rbenv, that list can go on and on).

asdf

https://asdf-vm.com/

This is just a reminder that things like asdf exist and it’s doing what it’s promising: a single tool to manage all runtimes. It is also really extensible as you can easily add your tool as well by writing a simple plugin, nothing too scary, it’s just a bit of scripts that define how you can list versions of your tool and how you can download it to install. In reality, it’s a rare case when you need to write your own plugin due to a huge list of community plugins available.

I have a video about using asdf on the channel, so it’s a good starting point:

mise (rtx previously)

https://mise.jdx.dev/

Also in the comments for that video, there was a suggestion to try a different tool called mise (known as rtx before).

It is the same idea (fully compatible with asdf) but written in Rust and it is trying to solve some known issues of asdf (performance, CLI UX, etc). More information can be found here: https://mise.jdx.dev/dev-tools/comparison-to-asdf.html

A short demo of using mise:

If you were using rtx previously, check this link: https://mise.jdx.dev/rtx.html

The cool thing is that you can also manage your global versions of tools. I already see a great use case for the future. Next time when I have to set up a new laptop, I can just install mise, copy ~/.config/mise/config.toml file and run mise install. I’ll instantly get the same working environment as before. If a project requires different versions, they will be defined in a local .mise.toml file, and mise will automatically install the required versions when you go into those directories.

I highly recommend you at least try those tools as they are convenient in practice (not just on paper), mise has great docs and it’s a great starting point!

Subscribe now

Hi, a quick one! I’ve just realized that the New UI in IntelliJ supports a background image - so I decided to give it a try.

I’m pretty sure it wasn’t possible in IDEA for a while (probably before the New UI was released) - but I’m not sure now.

Anyway, it’s possible now - so you can upgrade the look of your editor!

I’ve recently been to the Seven Sisters and Eastbourne and took a picture there and it works really well as the background.

It works in all sidebars (project tree, REPL, terminal) and looks fantastic to my taste.

Here is what you need to do:

• Enable the New UI in IDEA: Settings → Appearance & Behaviour → New UI

• Set background image: Settings → Appearance & Behaviour → Appearance → UI Options

  Subscribe now

Have a nice day!

Entering the AWS Console for the first time could be really overwhelming. Once you click on the search tab you will see a dropdown containing a huge list.

Subscribe now

And how do you suppose to know that EC2 (standing for Amazon Elastic Compute Cloud) is your virtual machine?

So the purpose of this guide is to highlight the most commonly used AWS Cloud services so you are not lost when you start your AWS journey.

VPC and networking

Amazon Virtual Private Cloud (Amazon VPC) is a service provided by Amazon Web Services (AWS) that allows users to create isolated virtual networks within the AWS cloud, enabling them to launch and manage resources with control over network configuration and security settings.

VPC is definitely worth mentioning, you’ll need to create one so you have a place where you can put other AWS resources (like EC2, databases, etc.) - but most likely if you are joining a company that already using AWS - it should have VPC and networking setup so you won’t need to configure it yourself and probably won’t need to modify it too much.

But once you are more familiar with AWS in general I highly recommend setting up your personal AWS account and going through the VPC setup yourself. It will be definitely useful long-term and will learn a lot of new things (for most people), like CIDR blocks, private and public subnets, how to use VPC flow logs, etc.

If you want to learn more - start here: https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html

EC2 (Amazon Elastic Compute Cloud)

Now let’s move to something more used in day-to-day work: EC2. So it’s just a fancy name for you well-known virtual machines, you can request an instance (in most cases some sort of Linux) with some amount of CPU and RAM then log into it (via SSH from the local machine or from AWS Console) and do whatever you need. Install tools, run your applications, etc.

AWS Containers services

In many recent years, I don’t remember seeing anything that wasn’t eventually deployed as a container, so this is probably the most important section that you need to learn about AWS.

Of course, before you deploy something as a container you need to build it and publish it. So once you’ve built your Docker image (how and where you are doing it is up to you, that could be locally in the simplest case or as part of your CI pipeline) - it’s time to publish your image.

Amazon Elastic Container Registry (Amazon ECR) is a fully managed container registry offering high-performance hosting, so you can reliably deploy application images and artifacts anywhere.

ECR is the service to store your Docker images (privately), so you can use them in other AWS services, like ECS or EKS. A simple way of thinking about ECR is as a Docker Hub that is private to your AWS account.

The next bit is more interesting - how and where do you actually run your containers? And there are a couple of options.

Amazon Elastic Kubernetes Service (Amazon EKS)

Amazon Elastic Kubernetes Service (Amazon EKS) is a managed Kubernetes service to run Kubernetes in the AWS cloud and on-premises data centers. In the cloud, Amazon EKS automatically manages the availability and scalability of the Kubernetes control plane nodes responsible for scheduling containers, managing application availability, storing cluster data, and other key tasks.

So one option is to use an AWS-managed Kubernetes cluster (EKS) - it is a really powerful option as you basically have all Kubernetes features available, but it’s still quite a lot to manage. Yes the cluster itself is managed by AWS, but everything on top (e.g. a tool to deploy your Helm charts, etc) is up to the operating team in your company.

Honestly, I prefer a simpler approach to running containers in AWS - and it is ECS.

Amazon Elastic Container Service (ECS)

Amazon Elastic Container Service (ECS) is a fully managed container orchestration service that helps you to more efficiently deploy, manage, and scale containerized applications

So a couple of things to understand about ECS, first of all, new a need to create an ECS cluster - this will be the place where you can put your services.

The next thing is to define the underlying infrastructure, and again same as with EKS you have two options: EC2 and Fargate.

On EC2

In the case of EC2, you are responsible for creating instances first (that could be just single EC2 instances or they could live inside an auto-scaling group) - then you can add those instances to the ECS cluster, so they will be available for ECS to place your ECS services and tasks.

On Fargate

AWS Fargate is a serverless, pay-as-you-go compute engine that lets you focus on building applications without managing servers. Moving tasks such as server management, resource allocation, and scaling to AWS does not only improve your operational posture, but also accelerates the process of going from idea to production on the cloud, and lowers the total cost of ownership. 

In the case of Fargate, it’s even simpler - you don’t have to manage underlying EC2 at all.

Regarding costs, on paper Fargate is a bit more expensive, in reality, to make EC2-based deployment cost-efficient you need to spend a lot of time optimizing and balancing your tasks and underlying EC2 infrastructure and it’s really hard to fully automate. So you will end up with not fully utilized EC2 instances at some point. If your company don’t have a dedicated infrastructure team - I would suggest just using Fargate and focusing on the applications, not the infrastructure.

ECS services and tasks

In ECS you define your deployments and ECS services and tasks. The service will contain one or multiple tasks and you can define things like deployment strategy, instance count, etc. ECS tasks on the other hand describe which containers to run as part of the task, the core here is container definitions - JSON data that defines your deployment. I really like the simplicity of the format and that you can just use Terraform without any additional tools to provision your ECS services.

A simple task definition will look something like this:

{
  "family": "my-service",
  "containerDefinitions": [
    {
      "name": "web",
      "image": "my/image:latest",
      "cpu": 128,
      "memory": 512,
      "portMappings": [
        {
          "containerPort": 8080,
          "protocol": "tcp"
        }
      ]
      "essential": true
    }
  ]
}

Load balancers (ALB, NLB)

If you decide to use ECS, most likely you will need a load balancer in front of your ECS service, so you will have a single DNS name for your service and the load will be distributed across your ECS service instances. The integration between ECS and ALB (Application load balancer) is really common and easy to configure. There are other use cases for load balancers as well (not related to ECS).

S3 (Amazon Simple Storage Service)

The next crucial and really popular AWS service is S3 (it’s a triple S in the service name, that’s why it’s called S3). But it’s also really easy to describe - you can store any files or objects you want. The granularity is the S3 bucket where you can put your objects in a nested way (different S3 paths).

Databases

It’s almost a 100% chance that you’ll use some kind of database in your system. Depending on your architecture it will be most likely relational (SQL) or NoSQL database.

Amazon Relational Database Service (RDS)

If you want a PostgreSQL or MySQL you’ll use an RDS with the required engine type. The other thing to consider is Aurora which is the same RDS in a nutshell but with additional options like high availability and easier to scale.

NoSQL

A popular NoSQL database solution is DynamoDB. There is also a managed Cassandra (called Amazon Keyspaces).

CloudWatch

CloudWatch is a service where you will be able to find your logs and metrics. Most AWS services have a way to configure CloudWatch integration to save logs and additional metrics. There are some metrics available by default - so you can just explore what’s available. There is also an option to configure custom metrics and set up alerts and dashboards so you have better visibility of your system.

Messaging

There are different options regarding messaging and queues so you can pick something depending on your needs.

Amazon Simple Queue Service (SQS)

Amazon SQS provides queues for high-throughput, system-to-system messaging. You can use queues to decouple heavyweight processes and to buffer and batch work. Amazon SQS stores messages until microservices and serverless applications process them.

Kinesis

Amazon Kinesis cost-effectively processes and analyzes streaming data at any scale as a fully managed service. With Kinesis, you can ingest real-time data, such as video, audio, application logs, website clickstreams, and IoT telemetry data, for machine learning (ML), analytics, and other applications.

Managed Kafka (MSK)

Securely stream data with a fully managed, highly available Apache Kafka service

It’s out of the scope of the article to discuss differences and where to use what - but hopefully you won’t be surprised now when you hear SQS, Kinesis or MSK.

Serverless (AWS Lambda)

AWS Lambda is a compute service that runs your code in response to events and automatically manages the compute resources, making it the fastest way to turn an idea into a modern, production, serverless applications.

Some people use AWS Lambda quite heavily, and some do not. AWS Lambda is a service that will allow you to run your serverless functions (in supported languages) without provisioning the infrastructure (like EC2s).

Analytics

If you work with data and analytics, most likely you will end up working with Athena (a query engine, for example on top of data stored in S3), AWS Glue (a tool to manage schemas for your Athena tables)

AWS Costs

It’s also worth mentioning the AWS Costs Explorer - a place where you can check your costs (hopefully they are not skyrocketing). A useful feature there is that you can split it by service, resource or tag (it’s really important to tag our resources when you create them).

Subscribe now

I hope that was a useful overview of AWS Cloud. There are a lot of nuances in all of the services and it just takes time to become familiar - so don’t worry too much!

Although things like Datomic and XTDB exist - sometimes you are just not ready to commit to a completely different storage type. So what if you want to keep using a well-known relational database (PostgreSQL in our case) but still try to implement an event-sourcing pattern?

Subscribe now

What is the event-sourcing pattern?

ChatGPT is quite good at giving short summaries and definitions, so here it is:

Event sourcing is a software design pattern where the state of a system is determined by a sequence of immutable events, allowing for a complete audit trail and the ability to reconstruct the system's state at any point in time. It involves storing and replaying events to represent changes in the application state rather than maintaining the current state directly.

Check out this article as well: https://learn.microsoft.com/en-us/azure/architecture/patterns/event-sourcing

PostgreSQL and Clojure implementation

The database table structure will be really simple, to store all the data we will need just a single table:

create table events
(
    id             uuid primary key   default gen_random_uuid(),
    type           text      not null,
    aggregate_id   uuid      not null,
    aggregate_type text      not null,
    payload        jsonb     not null,
    created_at     timestamp not null default current_timestamp
);

So `aggregate_type` is the type of the entity (or resource): `Card`, `Order`, `Customer` or whatever your application needs, and `aggregate_id` is the ID of the resource.

The `type` field is the type of individual event inside the resource, the naming could be different, but I prefer to include the resource name so it’s more readable: OrderCreated, OrderPaid, CardDispatched, CustomerBlocked - again, it could be any event that happens in your system.

The most interesting part is of course the `payload` field, it is the JSON payload and here we have the flexibility of NoSQL inside the relational database - so you just put any data related to the event in the payload.

SQL queries

Let’s take a look now at a couple of SQL queries that will be most commonly used:

Starting with the simplest one, getting a single event by ID:

select * from events where id = :id

Getting all event for the resource ID:

select * from events 
where aggregate_id = :aggregate_id 
order by created_at

In this case, it is usually useful to apply ordering by `created_at` so we can combine events in order later in the code (you can of course do the sorting in code as well) - we will see that in a bit!

Things get a bit tricky if you want to search for multiple resources where a condition based on the data of the payload is true, in this case, we need to find all aggregate ids first and then we need to find all other events for those aggregate ids. The most natural way of doing it is using an `IN` statement, like this:

SELECT * FROM events
WHERE aggregate_id IN (
    SELECT aggregate_id from events
    WHERE payload ->> 'customer_id' = :customer_id
      AND aggregate_type = 'customer'
)

However, we found out that using a `JOIN` works better in PostgreSQL for most cases, so the query becomes:

SELECT DISTINCT e1.* FROM events e1
  INNER JOIN events e2 using (aggregate_id)
  WHERE e2.payload ->> 'customer_id' = :customer_id 
    AND aggregate_type = 'customer'

That’s it for the DB side, let’s take a look at how to combine those events in code in Clojure!

Show me some Clojure code

The core of the Clojure logic will be the `apply-event` function, we are going to use Clojure multi-methods here:

(defmulti apply-event
          (fn [_state event]
            (keyword
              (str (:aggregate-type event)
                   "/"
                   (:type event)))))

After that we need to register this function for all our event types, for example:

(defmethod apply-event :order/order-created
  [_state event]
  (merge
    {:resource-type (:aggregate-type event)
     :order-id (:aggregate-id event)
     :created-at (:created-at event)}
    (:payload event)))

(defmethod apply-event :order/order-paid
  [state event]
  (merge state
         (:payload event)
         {:updated-at (:created-at event)}))

(defmethod apply-event :order/order-dispatched
  [state event]
  (merge state
         (:payload event)
         {:updated-at (:created-at event)}))

Note that sometimes we just ignore the state, `_state`, as we know this event should always be the first one for a particular resource type, so here we just building the initial state. In all other event types, we are using the passed state and updating it.

Finally, we need to apply this `apply-event` function to the list of events, in this case, let’s introduce a new helper function:

(defn project
  ([events]
   (project {} events))
  ([state events]
   (reduce apply-event state events)))

And the final code after we get events from DB will look like this:

(defn get-by-aggregate-id
  [{:keys [datasource]} aggregate-id]
  (let [select-query (-> {:select :*
                          :from :events
                          :where [:= :aggregate-id aggregate-id]
                          :order-by [:created-at]}
                         (sql/format))
        events (jdbc/execute!
                 (datasource)
                 select-query
                 {:builder-fn rs/as-unqualified-kebab-maps})]
    (->> events
         (map (fn [event]
                (update event :payload <-jsonb)))
         (project))))

Subscribe now

Final thoughts

Of course, this pattern is not coming for free (compared to mutable state in the database), some queries are much more complex or even not possible, so it always depends on the use case.

Also, a common problem is that now we are calculating the state of the resource from events on the fly each time - which can lead to issues with the performance. A common solution to that is to build a separate view which is updated based on the sequence of events - so now in addition to the event log we will have our normal tables with the current state of the resource, but it should be used only for the read operations.

The link to the source code for this example

For those who prefer the video format here is the link to my YouTube:

Subscribe now

Hello, Clojure enthusiasts!

I'm Andrey, and in today's blog post, I want to share my thoughts on the libraries and tools we use in production when working with Clojure.

Whether you're a seasoned developer or just starting your journey with Clojure, I hope this information proves valuable.

Libraries vs Frameworks

Before delving into specifics, let's address a crucial point. While there are web frameworks in the Clojure ecosystem, I recommend, especially for beginners, to start by combining libraries instead. This approach provides a deeper understanding of how different pieces work together. Later, if you find frameworks appealing, feel free to explore them. However, in my experience with real production code, I've rarely (never) seen frameworks used.

As a framework - `biff` (https://biffweb.com) looks interesting, maybe for beginners, it will be a quick way to get up and running.

Anyway in this blog post, we will focus on libraries!

Managing Dependencies

Now, let's talk about managing dependencies. Traditionally, leiningen has been the go-to tool, but `deps.edn` is gaining traction. While Leningen still works fine, `deps.edn` is becoming a standard in new projects.

• https://leiningen.org

• https://clojure.org/guides/deps_and_cli

Managing Components

Handling components in a Clojure system is crucial. While you can roll your solution, using a library can add structure and make it easier to manage start and stop logic. Two solid options that I can recommend here are:

• `component` - https://github.com/stuartsierra/component

• `integrant` - https://github.com/weavejester/integrant

System creation with `component` will look like this:

(defn example-system [config-options]
  (let [{:keys [host port]} config-options]
    (component/system-map
      :db (new-database host port)
      :scheduler (new-scheduler)
      :app (component/using
             (example-component config-options)
             {:database  :db
              :scheduler :scheduler}))))

With `integrant` you’ll define your system as a plain Clojure map (or an EDN file) with custom tags:

(def config
  {:adapter/jetty {:port 8080, :handler (ig/ref :handler/greet)}
   :handler/greet {:name "Alice"}})

(defmethod ig/init-key :adapter/jetty [_ {:keys [handler] :as opts}]
  (jetty/run-jetty handler (-> opts (dissoc :handler) (assoc :join? false))))

(defmethod ig/init-key :handler/greet [_ {:keys [name]}]
  (fn [_] (resp/response (str "Hello " name))))

Integrant could work quite well with the `aero` library, but again there are two possible approaches, so I recommend reading both links to learn more:

• https://www.pixelated-noise.com/blog/2022/04/28/integrant-and-aero/index.html

• https://robjohnson.dev/posts/aero-and-integrant/

I'd advise against using `mount` (https://github.com/tolitius/mount) due to its potential to encourage bad coding practices (IMO) - sharing global state and making functions impure but using global components directly (not passed through input arguments). That could be overcome with some best practices (not allowing this pattern) - the issue that it is almost impossible to enforce.

Managing configuration

This is pretty simple for me, a good option is `aero` (https://github.com/juxt/aero), the configuration file is defined in EDN and it’s really easy to read:

{:input-topic #env KAFKA_INPUT_TOPIC
 :output-topic #env KAFKA_OUTPUT_TOPIC
 :kafka {:bootstrap.servers #env KAFKA_BOOTSTRAP_SERVERS
                 :application.id #env KAFKA_APPLICATION_ID
                 :security.protocol "SSL"
                 :ssl.keystore.type "PKCS12"
                 :ssl.truststore.type "JKS"
                 :auto.offset.reset #env KAFKA_AUTO_OFFSET_RESET
                 :producer.compression.type #or [#env KAFKA_PRODUCER_COMPRESSION_TYPE "zstd"]
                 :processing.guarantee #or [#env KAFKA_PROCESSING_GUARANTEE "exactly_once_v2"]
                 :producer.acks "all"}
 :server {:host "0.0.0.0"
          :port 10124}
 :service-version #env SERVICE_VERSION}

So on startup of your application, you just render it (so it will fill the placeholders with values from environment variables) and pass it to your application system. A smart idea would be to use one of the schema validation libraries and create a schema in your application configuration - so the application won’t start if it’s not valid (that will usually mean that you forgot to pass the required environment variable).

If you want to keep it simple use `(System/getenv "ENV_VAR")`.

Schemas

For schemas and validation, there are three solid options:

• `schema` - https://github.com/plumatic/schema

• `clojure.spec` - https://github.com/clojure/spec.alpha

• `malli` - https://github.com/metosin/malli

I prefer to use `malli` most of the time, an example of schema will look like this:

(def kafka-config-base-schema
  (m/schema
    [:map
     [:bootstrap.servers :string]
     [:application.id :string]
     [:auto.offset.reset [:enum "earliest" "latest" "none"]]
     [:producer.compression.type [:enum "none" "zstd"]]
     [:processing.guarantee [:enum "at_least_once" "exactly_once_v2"]]
     [:producer.acks [:enum "0" "1" "all"]]]))

HTTP Server

For spinning up HTTP servers, we've primarily used Jetty as the base. On top of Jetty, you can use Ring or Pedestal - both are fine!

• https://github.com/ring-clojure/ring

• http://pedestal.io/pedestal/0.7-pre/index.html

There was a period when `http-kit` was popular, I’m not a huge fan TBH, mainly because there is a lack of support if you will finally want some advanced monitoring for your application: APM and distributed tracing from the OpenTelementry and auto-instrumentation (Datadog, NewRelic, etc).

HTTP Router

In addition to the base HTTP server and HTTP abstraction, you’ll need a way to route your request to correct handlers, one of the oldest options is Compojure, it’s simple to learn and use but has its limitation being macros-based. People tend to use data-based approaches, so there are libraries on the plate like:

• https://github.com/juxt/bidi

• https://github.com/metosin/reitit

If you’ve decided to use Pedestal not Ring it comes with its own data-based way to define routes - and it’s good enough for most use cases (also Reitit can work on top of Pedestal as well).

HTTP Requests

When it comes to handling HTTP requests, `clj-http` is a reliable choice, utilizing the Apache HTTP library.

If you're on Java 11 or higher, the built-in Java HTTP client is also worth exploring, for example, the https://github.com/babashka/http-client

Working with JSON

Using JSON is popular nowadays (I don’t even remember working with anything else), a couple of libraries to work with JSON in Clojure:

• https://github.com/dakrone/cheshire

• https://github.com/metosin/jsonista

• https://github.com/clojure/data.json

• https://github.com/cnuernber/charred

Check this link - it’s a good overview of the JSON libraries available: https://www.juxt.pro/blog/json-in-clojure/

Working with Relational Databases

For working with relational databases, `next.jdbc` (https://github.com/seancorfield/next-jdbc) is the current standard, and HikariCP works well for connection pools.

Regarding SQL query libraries, consider either HoneySQL or HugSQL, both of which we've used in production.

Other Libraries

• `core.cache` or `core.memoize` if you’d like a bit more advanced caching library

• `core.async` if you are big on the idea of building non-blocking code (be careful it’s really easy to overengineer here)

• `claypoole` - a great library to work with Java thread pools in Clojure: https://github.com/clj-commons/claypoole

Other tools

• `babashka` (https://babashka.org) - great tool to write scripts in Clojure, could sound crazy at first, but give it a try if you haven’t before! We have all our scripting for CI written in babashka now!

• `cljfmt` - my go-to code formatter

• `clj-kondo` - great static linter for Clojure code

We use both `cljfmt` and `clj-kondo` as pipeline steps in the CI to keep the Clojure code tidy!

Tests

Although Clojure developers rely on REPL a lot to debug and write code, it’s still crucial to eventually produce tests for your code. There are different levels of testing available, the most common groups I’ve seen so far are:

• `unit` - simple tests that do not require any complex setup or starting your system (or parts of the system), call your functions or chains of functions and assert the result

• `persistence` - it is slightly more high-level, in those tests we are talking to the relational database (or other type of persistent storage) and checking for side-effects (e.g. a database row was created as the result of the test), but we still don’t usually start our application system

• `component` - it is testing the full flow that requires spinning your application system, here are a couple of examples: call the HTTP endpoint of your test system and expect correct side-effects (change in the database, event published to the queue, a call to the third party system was made, etc)

Regarding writing tests, I just suggest you stick with the simplest option: `clojure.test` that comes out-of-the-box with Clojure.

The slightly more interesting topic is which test runner to use. Here we have a bunch of options:

• `lein test` - in case you are using Leininen, that comes out-of-the-box

• `eftest` - has more configuration options and better reporting, and could be used for both `leiningen` and `deps.edn`: https://github.com/weavejester/eftest

• `cognitect-labs/test-runner` - a simple (but good enough) option if you are using `deps.edn` and missing a built-in option to run tests: https://github.com/cognitect-labs/test-runner

• `kaocha` - a test runner with a lot of features and great docs, for both Leiningen and `deps.edn`: https://github.com/lambdaisland/kaocha

Although there are a bunch of options, I don’t suggest you spend too much time picking one, test runner is doing a simple job: search and run your tests - so you will be fine in any case. Better focus on writing more robust and reliable tests so you worry less after refactoring your code!

Conclusion

And there you have it – a comprehensive overview of the libraries and tools we use in production Clojure code (or at least I’ve tried). I hope this guide helps you navigate the rich Clojure ecosystem.

If you prefer the video content, you can find almost the same as the video on my channel: