From Scala to Migration
As I mentioned in the initial Rust post, I’m migrating a Scala project to Rust to learn the language. I’ve made a lot of progress, and I’m at a stage where I’m thinking about the deployment right now. In this post, I’ll go over some of the highlights of this project for me and my learning of Rust.
Original project
Before we look at specific things, let’s do a quick overview of the project so you can have an idea of what it is that I’m migrating. You can think of this application as a simple CRM for a small bridal dress retailer. Today, I realize I should probably not have rolled a custom-made application, but I was younger and so excited about learning programming that I did.
The original implementation is written in Scala. On the backend I used Http4s with cats-effect and fs2 as core libraries. The frontend is a simple HTML + CSS + JS application. The backend generates the HTML through a Scala templating engine called Twirl. All the data is stored into Airtable.
The feature set of the application is small:
- CRUD like interfaces for things like clients / events / invoices / items and products, and payments (we record the transaction but the money does not flow through the system)
- invoices are the central piece tying most of the other entities together
- printing of invoices to PDF (all generated PDFs are also stored inside the cloud (GCP Storage))
- printing is delegated to a third party API over HTTPs - PDF Rocket (amazing service)
- admin panel to quickly retrieve all invoices, payment, etc as CSV for reporting and accounting
- somewhat complex state and transition for items of an invoice and a cumulative state for the invoice itself
Before starting the migration, I used Claude to build a plan of how I’d do that. My ambitions for this rewrite were not only about learning Rust, but also to move away from Airtable. And with my plan, I started.
Templating
It’s unfortunate, but I think the bulk of the complexity for this application resides in the Twirl templates, and that’s why I decided to start migrating the templates first.
I decided to use maud after getting a recommendation by a fellow rustacean from reddit. I went with maud because I wanted my views to be compiled, as they were with Twirl.
Overall, I’m really happy with maud, it works great, even if implemented through a macro.
The only caveat I have so far is the way Option are handled. In Twirl, if you have val value = Some("string"), you can just @value and Twirl will render what you’d expect: string. Nothing if the value is None.
With maud, you need to explicitly handle the Some case. For instance, if let x = Some("test");, then html! { (x) } will not compile. You will need to do something like:
html! {
@if let Some(s) = x {
(s)
}
}
Initially, I was worried that Twirls template, which are compiled to functions, would be hard to replicate in maud, but it turns out that it was not.
If you had a template at number_input.scala.html containing:
@(value: Option[Int])
<input type="number" value="@value">
You can use it from another template:
@import number_input
@()
<form>
<label>Number</label>
@number_input(Some(5))
</form>
In Rust, with maud, you’d just do the natural thing:
fn number_input(value: Option<i64>) -> Markup {
html! {
input type="number" [value]=(value)
}
}
fn form() -> Markup {
html! {
form {
label { "Number" }
(number_input(Some(5)))
}
}
}
Overall, kudos to the maud team, and the html2maud cli and library, which I used to programmatically convert most of my templates automatically.
maud code.
Backend
The original backend is pretty simple. As I mentioned above, it contains a handful of CRUD like endpoints for some entities and porting that over to Rust and axum was simple.
Example for a simple CRUD for a Client entity
For example a simple /clients endpoint, looked like:
val clientService = HttpService[IO] {
case GET -> Root / "clients" / "new" => ...
case sr @ POST -> Root / "clients" / clientId / "update" => ...
case GET -> Root / "clients" / clientId => ...
case sr @ POST -> Root / "clients" / "new" => ...
case GET -> Root / "clients" => ...
}
Converted to Rust, it would look like a clients.rs file with the following code:
async fn list_clients(State(pool): State<SqlitePool>) -> Result<Markup, AppError> {
...
}
async fn new_client() -> Result<Markup, AppError> {
...
}
async fn one_client(
State(pool): State<SqlitePool>,
Path(client_id): Path<i64>,
) -> Result<Markup, AppError> {
...
}
async fn update_one_client(
State(pool): State<SqlitePool>,
Path(client_id): Path<i64>,
Form(update): Form<ClientForm>,
) -> Result<Redirect, AppError> {
...
}
async fn create_one_client(
State(pool): State<SqlitePool>,
Form(create): Form<ClientForm>,
) -> Result<Redirect, AppError> {
...
}
pub fn client_router() -> Router<AppState> {
Router::new()
.route("/clients", get(list_clients))
.route("/clients/new", get(new_client))
.route("/clients/{client_id}", get(one_client))
.route("/clients/new", post(create_one_client))
.route("/clients/{client_id}/update", post(update_one_client))
}
The more interesting pieces were related to invoices. The original implementation can turn HTML invoices into PDF (usually to print in-store) and when doing so, we store the generated PDF on a GCP Storage bucket.
The implementation for this is pretty interesting and I think I’ll have a separate blog entry for it, but if you want to see it, this commit contains most of it. In practice, we generate the HTML (through maud), send it to PDF Rocket and we’ll stream the response bytes directly into the Google Cloud Storage object.
Similarly, the original implementation has an admin endpoint to help with reporting and accounting. And because I was just finishing working with the /generate-print endpoint and had used streaming, I wondered if I could stream database results into a CSV directly. I was able to pull it off, using sqlx fetch and axum-streams-rs but it was tricky. And here again, my issues were related to lifetimes, and probably a bug in sqlx fetch.
SQLx
The other interesting thing about the backend with regard to the re-write is the move to SQLite. Storing the data in an external service like Airtable, meant that I lost the ability to have transaction and this led to some issues.
By using SQLite and sqlx, we can enjoy ACID guarantees through transactions, which is great. Furthermore, having a local database is so much faster than reaching out to a third party service over HTTP.
The effects on speed are even more dramatic for some endpoints where I fetch all records of a specific kind (like invoices or payments) for a dashboard or a CSV. This is because Airtable has a hard limit on the number of records you can retrieve in one request, so you have to do multiple requests to retrieve the data you need.
As for the experience with SQLx, it was great. Pretty straight forward, and a little “boilerplaty” for simple CRUD things, but that’s fine.
I liked that you can derive FromRow on your struct and query_as will return the right type. SQLx is also heavily using type parameters and lifetime parameters, so you can use that do define functions that work for both transactions and the SQLite pool directly.
As an example, here is how I implemented select_by_id for my config table:
/// Database row structure for config table
#[derive(Debug, FromRow)]
pub struct ConfigRow {
pub id: i64,
pub key: String,
pub value: String,
pub config_type: String,
pub created_at: String,
pub updated_at: String,
}
async fn select_one<'c, E>(id: i64, e: E) -> Result<Option<ConfigRow>>
where
E: Executor<'c, Database = Sqlite>,
{
let result: Option<ConfigRow> = sqlx::query_as(
"SELECT * FROM config WHERE id = ?"
)
.bind(id)
.fetch_optional(e)
.await
.context(format!("Failed to retrieve one config with id {}", id))?;
Ok(result)
}
The function has one lifetime parameter: 'c and one type parameter E. Both are tied together through the constraints where we require that E has a Executor implementation available with the lifetime 'c for the Sqlite database. I found that writing this function was straightforward but that might be because I am familiar with generic functions from Scala.
You can then use that function in one of the following ways:
async fn a_query(pool: &SqlitePool) {
// directly with the pool
select_one(1, pool);
// or with a transaction
let mut tx = pool.begin().await.context("Failed to begin transaction")?;
// ...
select_one(1, &mut *tx);
// ...
tx.commit().await.context("Failed to commit transaction")?;
}
Follow-ups
In general, I’m really happy with Rust and how this whole project is going well and was a lot of fun.
I’ll write at least two follow-up posts, one where I focus more on the deployment of the app, using Nix and NixOS (the code is already available if you want to see), and one related to the /generate-print endpoint, as mentioned above.