Service Kit

I just merged in a pull request to service-kit, which converts a boilerplate project of mine into a cargo template for rust projects that combines my pet architecture with a lot of preconfigured tools. This is something I immediately used to yarf out a few gizmos for kicking the tires on some exploratory ideas, but it’s out there and now anyone can use it if you want.

You can get started with it like so:

cargo generate esmevane/service-kit

From there, it’ll ask you what you want to name your project, and then it’ll attempt to scaffold out the project for you.

Table of Contents

What’s in it?

A bunch of stuff, really.

• A documentation builder that uses oranda and mdbook to create documentation sites (and a ready-to-go action if you’re on Github).
• A cargo-dist based release process, made to work with cargo-release’s version/release workflows (also preconfigured if you’re on Github).
• A minimal CLI tooling setup with clap.rs that has stuff like dialoguer and indicatif ready to go for you to leverage as you like.
• A hierarchical config setup, driven by config.rs, which allows you to build your configuration around environment variables, and your pick of JSON/YAML/TOML config files.
• A minimal setup for a component-based terminal UI made with ratatui and crossterm, for anyone who wants to write nice terminal programs that go a bit beyond CLI commands.
• An axum webserver and two reqwest companion clients, one for browsers and one for rust projects (such as your CLI or TUI), all built around shared types defined with protocol buffers using prost. This webserver comes preconfigured with sqlite-based file storage, and the ability to embed assets in the built binary.
• Preconfigured telemetry with tracing, and logging on by default.
• A service-manager configuration that makes it installable as a service on Windows, macOS, or Linux in webserver mode.

The components

The whole thing builds out a few components for you, all in a virtual workspace managed by cargo, and already wired up as much as makes sense.

• A binary crate. This ships the CLI itself, which is the entrypoint to all the functionality that the kit offers.
• A book crate. This is a markdown book that gets put into the doc sites that oranda generates.
• A core crate. Where most of the code lies, in reusable library form if you need to leverage it.
• A protocol crate. This is where the service definitions sit. It is decoupled from the core crate but re-exported from it if you need single imports. This structure was necessary because of…
• The web crate. This creates a WASM export of the protocol so you can use it in browsers. The front-end embedded in the server leverages this client, which you can see by calling cargo run --bin [your project name] server full and looking at the browser console for localhost:8080.
• An xtask crate. This is a rust ecosystem trick that lets you put complex build processes in a manageable place. It’s common to leverage this pattern instead of makefiles. Right now it has one command, web-build, which builds the WASM client and puts it into the dist directory for the core crate.
• Preconfigured Github actions. If you’re on Github, all of this will (mostly) set up release tooling and documentation publication. As of the time of writing, you’ll have to uncomment some stuff in .github/workflows/web.yml to get it going.

Why?

This is more or less my “blessed” architecture, or in other words it’s how I set stuff up when I’m building it. The boilerplate for this setup takes a while for me, every time. Having this template lets me kick things off faster and copy stuff around less.

Rough edges

There are a lot of rough edges, so I’m calling this service-kit-v0, and if I need to do any updates I’ll use a copy-and-update approach to make sure published templates never change unless it’s to fix a bug, add documentation, and any non-breaking changes.

It’s just not done, you know? The documentation isn’t there at all, plus I haven’t moved all the names into liquid template variables, stuff like that. The readme needs a lot of polish, the ecosystem of components could use some improvement, the TUI isn’t wired up to observe the API, etc.

Next steps

I’ve got a few things I want to put in v1 that didn’t make it into v0 yet, but would probably not constitute breaking changes. These are things I decided to punt on for now, and circle back around to later.

• A little bit of durable queue worker / application messaging. I wanted to do this with sqlx, but, unfortunately, I haven’t been able to figure out how to solve a problem with vacuum and that held me up for a bit. I’m hoping to resolve this at some point, but for now that probably means I’ll have to do this with rusqlite.
• A vite dashboard that looks a bit nicer than the empty page we embed in the webserver right now.
• A setup process in the xtask crate that lets new installs bootstrap my favorite dev environment quicker.
• Container definitions for folks to use, to help people get started with smaller, lighter weight images that use less memory.
• Testing mechanisms and test watchers, and some CI plumbing that leverages them for PR checks and release checks.

Ambitions

Down the road, I’ve got some other stuff I want to incorporate

• Some way to test drive templates. Does this even exist? The feedback loop for templates is a bit cumbersome right now. I’m not sure what I would do to resolve this but it feels like there’s a lot of room to improve. That is, unless I’m missing it entirely.
• Support libraries. Right now all the behavior is baked directly into the template, and obviously that’s not the best approach. In order to make things more configurable down the road, it will probably make sense for me to start shipping service-kit crates that offer common functionality.
• Better Protobuf support. Websockets, a formal channel metaphor, and things of that nature. For example, prost_build wants you to hand-roll your own service generator implementation, and so what I’ve done now is a very minimal version of that. I can do better with syn, and use quotes to test-drive traits, but that’s probably something I should do with the support library vs. directly in the template.
• WASI support. One thing I really wanted to do was incorporate WASI support. I’m actually working a little bit on the underlying browser shims for this, but it didn’t seem like a “right now” thing, so I’ve punted on it. Hopefully it’s not too far off though, now that I’m clearing my free time a bit by publishing my WIP here. I’d like to say that this kit can help you ship quick WASI clients for your services, because that’d rock.
• Magic handlers. This is something I’ve built for myself and is one of the driving reasons why I was looking into baking in durable work queues. My own projects rely on a tool ecosystem I’ve been working on that lets me use the magic handler pattern for my own domain logic, and get Tower services out of it. Because they work so well for async logic, I’ve been able to build actor systems, message handlers, work queues, caching systems, socket exchanges, etc., with “logic bundlers”. I use it a lot myself. But, I need to ship the logic as a crate first, so I can circle back around to this when that’s at a point where I feel comfortable doing so.
• Database support. Obviously sqlite kicks ass, but it’s not everyone’s first choice. So, I’d like to add support for mysql and postgres at least. Doing that can help downstream users (me; I’m the downstream user) do cluster setup easier. I can do this manually for now whenever I need to, but it’d be better if I just shipped it as a configuration option.
• A full-fledged SPA kit. Again, something I often build and copy around for myself. It works, it’s great, I use it a lot. It’s React + xstate, and provides common setup and easy metaphors for localization, screens, dialogs, form validations, and request handling, all with a pretty nice and extensible API, and a preinstalled Mantine setup. I always move these from project to project as well, it just hasn’t been put into the template yet.

Past these obvious big bits, I’d like to bake in mailer support, payment processing support, and maybe some authz/authn boilerplate that I always wind up cooking up, as well as some way to decide what of these someone does and doesn’t want, and then maybe some tutorials on how to use it all in addition to reference docs.

For now, though, I’m done! I’m gonna go play video games with my son or something.

The link again if you’re curious.