What I Learnt Building a Lobsters TUI in Rust - WezM.net by Wesley Moore

What I Learnt Building a Lobsters TUI in Rust

Published on

As a learning and practice exercise I built a crate for interacting with the Lobsters programming community website. It’s built on the asynchronous Rust ecosystem. To demonstrate the crate I also built a terminal user interface (TUI).

Screenshot of Lobsters TUI
A screenshot of the TUI in Alacritty

Try It


Pre-built binaries with no runtime dependencies are available for:

Downloads Source Code

The TUI uses the following key bindings:

As mentioned in the introduction the motivation for starting the client was to practice using the async Rust ecosystem and it kind of spiralled from there. The resulting TUI is functional but not especially useful, since it just opens links in your browser. I can imagine it being slightly more useful if you could also view and reply to comments without leaving the UI.

Building It

The client proved to be an interesting challenge, mostly because Lobsters doesn’t have a full API. This meant I had to learn how to set up and use a cookie jar along side reqwest in order to make authenticated requests. Logging in requires supplying a cross-site request forgery token, which Rails uses to prevent CSRF attacks. To handle this I need to first fetch the login page, note the token, then POST to the login endpoint. I could have tried to extract the token from the markup with a regex or substring matching but instead used kuchiki to parse the HTML and then match on the meta element in the head.

Once I added support for writing with the client (posting comments), not just reading, I thought I best not test against the real site. Fortunately the site’s code is open source. I took this as an opportunity to use my new-found Docker knowledge and run it with Docker Compose. That turned out pretty easy since I was able to base it on one of the Dockerfiles for a Rails app I run. If you’re curious the Alpine Linux based Dockerfile and docker-compose.yml can be viewed in this paste.

After I had the basics of the client worked out I thought it would be neat to fetch the front page stories and render them in the terminal in a style similar to the site itself. I initially did this with ansi_term. It looked good but lacked interactivity so I looked into ways to build a TUI along the lines of tig. I built it several times with different crates, switching each time I hit a limitation. I tried:

Finally I ended up going a bit lower-level and used termion. It does everything itself but at the same time you lose the conveniences ncurses provides. It also doesn’t support Windows, so my plans of supporting that were thwarted. Some time after I had the termion version working I revisited tui-rs, which I had initially dismissed as unsuitable for my task. In hindsight it would probably have been perfect, but we’re here now.

In addition to async and TUI I also learned more about:

Whilst the library and UI aren’t especially useful the exercise was worth it. I got to practice a bunch of things and learn some new ones at the same time.

Comments on Lobsters

Previous Post: Cross Compiling Rust for FreeBSD With Docker
Next Post: An Illustrated Guide to Useful Command Line Tools

Comment icon Stay in touch!

Follow me on Twitter or Mastodon, subscribe to the feed, or send me an email.