In this work, we introduce OpenBench, a benchmark for evaluating the spatial reasoning ability of multimodal large language models in realistic open-world scenes. Built from pedestrian-perspective multimodal data with metric 3D cues, OpenBench covers question types ranging from qualitative relations to quantitative and kinematic reasoning. Our results show that performance gains reported in indoor settings do not transfer directly to open-world scenarios, highlighting an important gap in grounded spatial intelligence.

Congratulations to Mingrui and the team on sharing this work with the CVPR 2026 community!

Discover how we did it in our paper: AlignDrive: Aligned Lateral-Longitudinal Planning for End-to-End Autonomous Driving (arXiv). For a deeper dive into the methodology and to view our qualitative results, please visit our official project homepage.

AlignDrive takes aim at a subtle but important planning mismatch in autonomous driving: where to go and how fast to move should be decided together. The method conditions longitudinal prediction on the planned drive path, so speed reasoning becomes tied to the vehicle’s lateral choices and surrounding-agent interactions. It also uses planning-focused augmentation to expose the model to rare safety-critical cases, improving robustness on challenging closed-loop benchmarks.

During the A2 workshop, World Model: A New Paradigm for Autonomous Driving Research from High-Quality Data Generation to Reinforcement Learning Training, Zhaozhi delivered a talk on behalf of Prof. Zhang’s team. The presentation, titled Exploring Safety for Autonomous Driving: From Scene Generation to Planning, discussed how scenario generation and planning methods can support safer autonomous-driving systems. More information about the conference is available on the CICV official website.

Congratulations to Prof. Zhang, our team member Zhaozhi, and the team on sharing our autonomous-driving research with the CICV community.

In this work, we identify that multimodal large language models often struggle with visual-spatial reasoning because visual tokens are overshadowed by language tokens in attention. To address this issue, we propose VideoAnchor, a plug-and-play module that reinforces shared visual cues across frames without retraining, leading to more coherent visual grounding and stronger performance on spatial reasoning benchmarks.

Congratulations to Zhaozhi and the group on this exciting presentation at ICLR 2026!

Since the AI-assisted coding hype began, I have been an outspoken skeptic of these technologies. In large part, this was due to my own experience with the early models. While I am a techie, I am very much deliberate in how I adopt and use technology. I love useful tech that makes my life easier, and lets me accomplish more of my goals. However, I hate bad tech; technology that wastes my time, demonstrates bad design, or diminishes rather than enriches enjoyment (yes, I know, I was quite active on Twitter for a while …. maybe that’s a post for another day).

When I first tried generative coding models, they simply did not produce good code. They would sometimes produce functional code (but often not), and their solutions were almost always both messier and less efficient than what I would have done by hand. I tasked them with implementing the projects in the classes I teach, and they either failed, or succeeded with implementations that I would hope an undergraduate at UMD would find embarrassing. One early success, that I filed away in the back of my mind, however, was having ChatGPT help me write some highly vectorized AVX-256 code for longest common prefix computation that ended up being useful in our CAPS-SA work. Nonetheless, I was incredibly skeptical of people talking up (and using) these tools, and was quite outspoken about the types of mistakes they made and the limitations they had.

A small series of successes

You can try to ignore the world, but you do so at your own peril. Increasingly, colleagues and experts whose opinions I truly respect (some of whom had previously been very outspoken AI skeptics) began to express some very positive opinions. To be clear, I think that in the tech world, there is still a massive overstatement of the case for and future of some of these technology. However, even some of the calmest and most judicious tech folks I follow, who had previously expressed skepticism similar to mine, began to express much more optimistic, albeit nuanced, opinions. In particular, Jon Gjengset, whose technical opinions I have come to value immensely, and whose overall judgments (and taste) I find align highly with my own, has made some very keen observations about the use of these tools, and demonstrated their utility in a way that I find that I would share. So, I decided to give it another go.

Very quickly, I learned that, while I believe my opinions on the early models was correct, in that they were useful for helping those with little to no coding experience to cook up somewhat functional, but often ill-designed things, and they were often of very little use (or even a net negative) for seasoned developers. However, the current state of things is entirely different. In rapid succession, I was able to obtain a series of quick and, critically, practically useful successes with the newer class of models. I mention a few below just to give the flavor of how I was using these tools.

mim

We recently pre-printed a method (and associated tool) called mim, which is a semantically aware auxiliary index for gzip compressed (and block gzip compressed) FASTA and FASTQ files. We demonstrate that with a very small auxiliary index, one can massively speed up the parallel decompression and parsing of these files, eliminating a bottleneck in many tasks in high throughput sequencing analysis. These indices are semantically aware (i.e. know about the structure of the read records) and so can e.g. properly synchronize between paired-end files during decompression. mim is built, at a basic level, around the ideas of zran, and the initial implementation of mim was written in a mix of C and C++, with part of the indexer as a direct modification of the zran code. While this was useful and functional as a way to show the benefits of mim, I have come to greatly value keeping my software ecosystem in Rust as much as possible. It is the language that I prefer to write, and to maintain.

So, I tasked Claude (Sonnet, on the free plan!) with porting the core indexing functionality of mim from C++ to Rust. This was scoped to be a relatively straightforward translation, but nonetheless is something that would probably have taken me a few days to do by hand. More critically, it would have taken me large, uninterrupted, meeting-free blocks of time. Something of which I, as a professor, have precious little.

With the C++ source code (uploaded as an attachment), and a few hours worth of conversation, I was able to obtain a fully-functional Rust implementation of the mim indexing functionality in Rust. It was not a “one shot” solution; there were some hiccups in how multi-part archives were handled. But each time I ran into a problem, so long as I was able to explain to Claude the technical details of the issue that I was encountering, we were able to overcome the problem. As a result, we ended up with a mim implementation in Rust in short order, which has since become the core reference implementation of the idea. The current code has been polished (and parts of it rewritten by 1337 coders, like Ragnar), but getting to a functional and respectable Rust implementation, in a day, via the free tier of Sonnet, more than piqued my interest.

In memory tile collation in Cuttlefish 1

Our Cuttlefish (version 1) tool still acts as the basis for building the core information that is indexed by our (SSHash-based) piscem tool — the mapper for single-cell RNA-seq and single-cell ATAC-seq data upstream of alevin-fry.
Cuttlefish 1 builds a compacted colored de Bruijn graph very efficiently on reference sequences. It outputs both a set of maximal unitigs and, critically, a tiling, that describes precisely how each reference is spelled out by an ordered sequence of unitigs (each in a specific orientation), and possibly with gaps of N nucleotides. When we build a piscem index, we build an SSHash index over the unitig sequences, and a packed, inverted index over the tiling information. In this inverted index, we store, for each unitig, the sequence of references, positions and orientations in which it occurs.

However, when we (primarily Jamshed) wrote Cuttlefish 1, we had in mind the construction over massive collections of large references (e.g. large bacterial pangenomes, plant genomes, or human genomes / pangenomes). Specifically, we had in mind reference collections where, on average, each reference sequence was long (e.g. at least millions to tens of millions of nucleotides long).

This informed the parallelization strategy we adopted. Each input reference was cut into a number of bins equal to the thread count, each thread wrote out the tiling information for its assigned interval to an intermediate file. Then, these intermediate tilings were stitched together in order to yield the tiling information for the entire reference sequence before we moved on to the next reference sequence (e.g. the next chromosome). This worked brilliantly for genomes, but quite horribly for transcriptomes. When each reference sequence is short (1-2 kilobases), each thread ends up being assigned a small interval of sequence (a few dozen to a few hundred bases long, and typically tens of unitigs), writing tiny intermediate tilings to file, forcing the flushing of tiny buffers, and then reading those all in again to stitch together the overall tiling.

All of these tiny I/Os just made this step horrifically slower than it needed to be, and this was especially a problem on networked file systems, where millions or billions of tiny flushed I/Os absolutely tanked performance. An index build that should have taken 10s of minutes would end up taking many hours or, sometimes, even tens of hours. There were quite a few GitHub issues about this, and even a friendly how-to guide written by one of our users.

The solution was clear; avoid the intermediate files altogether. As the references are short, they aren’t needed. Likewise, it’s silly to partition a reference into many bins between threads when each thread ends up writing only a few symbols in the tiling. Instead, it makes much more sense to parallelize across reference sequences — to let each thread pull the next available reference sequence and create its tiling information in memory.

Conceptually, all of this was clear, but it was buried in a quite large C++ code base, and this specific code hadn’t been touched in several years. Jamshed and I discussed the fix, and it was on his radar, but he always had more important things to tackle, and then he graduated and moved on to his postdoc at Northeastern (where he’s doing awesome stuff, btw).

So, again, and based on my earlier success, I decided to give an AI coding agent a hand. This time, I didn’t really want to have to paste bits and pieces of code into the Claude website. While I was curious about these agents, I still wasn’t curious enough to commit to paying real, hard, money for one.

Luckily, as part of our academic-affiliated organization (COMBINE-lab) on GitHub, we get the cheap tier of co-pilot (the $10 / month version). So, I hopped on the co-pilot interface on GitHub and started chatting with it to create the feature. There was a bit of back and forth conversation, but over the course of a few hours, co-pilot and I worked on this PR that implemented the feature. This made the bottleneck step of Cuttlefish (on transcriptomes) a lot faster. It was about 5 times faster, even when running on a reasonably fast local disk, and much more than that when running on a cluster with a networked file system (though it still makes sense to do a build and write the output to local scratch and then move it to a shared location). This experience was a huge success. The AI models (mostly Claude Sonnet and Opus) “understood” the C++ codebase, my instructions, and our goal. It went off and implemented the feature piece-by-piece, coming back to me when expert input was required.

Not everything was perfect, however. For example, when implementing the inter-reference parallelization strategy, we reached a point where the agent introduced a concurrency bug. I manually investigated the code and immediately noticed the source of the error, and why it was happening. I pointed out where the error was, and let co-pilot take a shot. Once I pointed out where the problem was, it did come up with a solution, but it was a sub-optimal one. I explained my solution, and it agreed the solution I proposed was better.

Nonetheless, this was a minor hiccup in what was otherwise a big win. Even when interacting with the agent in this way, a key ability that it had was to inspect the feedback I provided critically, and reason about how to address the relevant issues. At one point, when we encountered a segmentation fault, I popped open the executable in lldb, ran it, and provided the backtrace to co-pilot. Based on the backtrace, it reasoned about, diagnosed, and fixed the bug.

This experience led me to attempt a bigger lift.

Two major translations in < 1 week (actully just about 4 days)

SSHash

SSHash is an incredibly efficient sequence dictionary for k-mer lookup in the context of genomics. The original data structure is described in this paper by Giulio Ermanno Pibiri, the reading of which, actually, initially sparked my collaboration with Giulio (so thank goodness for this paper!). Giulio and I recently pre-printed a set of data structural and algorithmic updates to SSHash aimed at theoretically and practically improving the caching behavior.

I’ve wanted a version of sshash in Rust now for a long time. In fact, I asked my PhD student Jason Fan (now at Fulcrum Genomics) to try to implement it about 2 years ago! Jason made some great progress, but it was a heavy lift, and not directly on the path of his dissertation, so this project was never finished. Further, sshash is on the core path of our lightweight read mapping tool, piscem, that is, itself, used upstream of alevin-fry for single-cell RNA-seq and single-cell ATAC-seq processing, so it was a perfect candidate for a “large scale” project to convert from C++ to Rust.

I didn’t want to attempt this through the co-pilot web interface, so I downloaded the co-pilot CLI. I was ready to go! And the co-pilot CLI … immediately segfaulted (another story for another post). So, I threw up my hands and decided to work through the VSCode co-pilot plugin. It works well, but I don’t use VSCode, I’m a neovim guy. Anyway, I wasn’t to be doing most of the coding (just the chatting and guiding, with some minimal manual fixes), so I went with it.

What happened next was a transformative experience for me. While converting an approximately single-file dependency for mim was very useful, and while directly modifying our Cuttlefish C++ codebase and adding a long-awaited feature was impressive, actually, what happened with sshash massively updated my prior about these tools.

I started with describing what SSHash is, giving the agent access to the full C++ SSHash codebase, and describing the design requirements for the Rust version. Claude derived a detailed implementation plan, broken down into major phases and minor sub-phases and steps within each phase. It described what the requirements were for each phase, how correctness was to be assessed, and when it would be OK to move on to the next step.

Over the next approximately 2 days, in a few focused sessions, and sometimes in the background (between meetings, etc.), I guided Claude, through co-pilot, into a complete, functional and approximately performance equivalent version of sshash written in Rust. Now, certainly, there was a lot of great (non-AI-created) infrastructure that we could rely on. SSHash is largely about succinct data structures, and we were able to build directly on sux-rs for bit-packed integer vectors and Elias Fano vectors, and on bsuccinct for the incredible PHast minimal perfect hash function (it was a real win that so many of the leaders in the succinct data structure space are “Rust forward”).

Nonetheless, this implementation feat absolutely blew me away. We went from 0 to a fully-functional and performance equivalent version of this very sophisticated index over 2 days; a goal that had eluded me for 2 years. Granted, if I was in a position to set aside a month or so for nothing but crack coding sessions, I think I could have accomplished this, but I don’t have a month or two to set aside like this. The fact that, in such a short span, I now had access to SSHash in Rust was simply mind-blowing.

Again, as with the Cuttlefish feature. Not everything was perfect the first time. In one step, Claude replaced what was an efficiently encoded Elias Fano sequence with O(1) select support with a Vec<u64> with a binary search. This was a silly and unnecessary mistake from both the size and speed perspective, and I have no idea why it did this. But, because I was watching this process in amazement, I noticed this, pointed out the issue, and it was promptly fixed.

Also, very interestingly, despite being given explicit and repeated instructions to directly follow the set of data structures used in the C++ implementation, Claude initially came up with a novel encoding scheme for offsets into light, medium, and heavy minimizer buckets, deriving an approach that was distinct from (but in practice competitive with) what was done in the C++ code. For the time being, I’ve left that design on a branch of the sshash-rs repository, as I think it was a quite interesting development.

Yet, again, with some cajoling and oversight, Claude produced a semantically equivalent index to what exists in C++. This index now exists in the sshash-rs repository, and it was critical to me in my most recent project (yes, there was one after sshash-rs which was started and had the initial version completed this week). I hope that having SSHash available in Rust also proves useful to other folks working in this space. It’s also astounding to me that all of this (modulo some subsequent optimizations for which I used Claude Code) was accomplished on a $10 / month co-pilot plan!

piscem-rs, the magnum Opus (nerd pun intended)

With the sshash Rust implementation in place, and enough evidence, finally, to pay for an actual Claude Max subscription. I booted it up, and started an ~1.5 day session that would ultimately result in the complete translation of our piscem-cpp mapping tool from C++ (with a submodule dependency on the C++ SSHash) to Rust (with a proper crate dependency on sshash-rs).

Like the sshash translation, this is a complex and interrelated codebase, implementing many features. Piscem supports mapping bulk RNA-seq data, single-cell (and single-nucleus) RNA-seq data, and single-cell ATAC-seq data, with an array of different algorithms and variants. It supports output in a custom binary format (the RAD format), consumed downstream by our alevin-fry tool for single-cell quantification. It implements necessary supporting features, like parsing fragment geometries using a custom parsing expression grammar (PEG). It has a host of different detailed optimizations, like a concurrent shared cache to retain lookup information about the k-mers bridging the ends of unitigs and the unitigs that most frequently follow them in the observed reads (the unitig end cache from our paper “Alevin-fry-atac enables rapid and memory frugal mapping of single-cell ATAC-seq data using virtual colors for accurate genomic pseudoalignment”).

All of this was created, in about a day and a half, by me driving Claude Code (with Opus 4.6). The result is astonishing. The current version supports all of the features of the C++ codebase, is slightly faster than the corresponding C++ code, is more cleanly organized, and produces semantically-equivalent RAD format output (e.g. the order of things can be different because of multithreading). On non-trivial test data, across all assay types, there is 100% concordance between the results of the C++ and Rust versions of piscem.

What’s even more amazing, perhaps, and something that truly sets the experience of using Claude Code apart from something like the web interfaces or even co-pilot in VSCode, is Claude Code’s tremendous ability to interact with the system, plan what it is doing, and introspect on the results of the code it is generating. During this process, when there were performance pitfalls or subtle differences in the algorithm. Claude Code instrumented the code with debug information and inspected the output to understand what was going wrong. Claude Code wrote the test harness to compare the binary RAD format files for semantic equivalence between the C++ and Rust versions of piscem. Claude Code wrote the unit tests to ensure that each abstraction worked in isolation and that implementation changes didn’t affect the output. Claude Code instrumented the index build on an ATAC-seq index (which is larger than that used for transcriptome mapping, because it indexes the whole genome) when I complained that it took too long, discovered the performance issue arising from a pathologically cache incoherent memory access pattern, fixed the pattern in the dependent codebase, validated the performance and correctness of this fix, and then asked me if I wanted to commit and push this change (I did).

In the series of translating the C++ code to Rust, we got down to a case where there was one particular sequencing read being mapped by the C++ program that was not being mapped by the Rust program. Claude suggested this was a very minor difference, but I insisted we fix it. Claude isolated the read, tracked down the difference, and explained why the read mapped in the C++ code and not the Rust code. It turns out, it was the result of a (very rare) integer overflow case in masking the bitpacked representation of a length 31 k-mer in a 64-bit integer. Claude pointed out what this bug in the C++ code was, why it happened, and offered to reproduce the buggy behavior in Rust to achieve perfect parity. Of course, I chose to simply fix the C++ code instead, but just going on that journey with Claude Code, and seeing it track down a rare corner case and reason about the algorithmic flow, was eye-opening. A huge part of what makes Claude Code so powerful, in my opinion, is the way that it can (and often expertly does) close the loop on developing, debugging and optimizing.

As a result of this development push, in about 2 days, we now have a complete implementation of piscem in Rust. This will be shortly replacing our C++ implementation, making it easier to add new features, making it easier for my students to contribute, and making maintenance and deployment much simpler!

In the future, I hope to apply these tools to both novel development, as well as some further translations to Rust that I’ve long desired (I’m looking at you, salmon). Also, along with a team of interested individuals, we have a super-secret (well, maybe not so secret if you follow me on BlueSky) Claude assisted rewrite planned, and I’m super psyched about this.

Non-technical caveats

I’ve substantially revised my perspective on the capabilities and utility of these AI coding models, and I think that they are and will continue to be an incredibly powerful tools, and I am excited to continue to explore how they can help achieve technical (and specifically software) goals. Nonetheless, there are several very real, and very serious caveats to these AI models in general that I think are quite concerning, and for which I don’t currently have a solution. I mention a few below. However, I do not think that simply refusing to engage with these tools is a productive way to address these caveats. I do not think that, even if we wanted, the genie could be put back in the bottle.

Here’s a super-short (and non-exhaustive) list of issues with these tools, how they have come to be, and the effects that will have that keep me up at night. I may likely continue to update this, or even turn it into its own post.

• Training by violating copyright and replicating intellectual property with zero traceability or citation. Despite the legal rulings on fair use exemptions, it seems to me quite clear that these models, in their training, have massively violated relevant copyright and licensing terms. They have consumed, with almost complete disregard for any protection, the intellectual property of countless individuals. Further, by virtue of the way these models work, at least as far as I am aware, when they recapitulate or even completely reproduce this subsumed IP, they provide no citation nor attribution. I do not know how to hold accountable the companies who have built these models for engaging in practices where they certainly know better, or even if that is possible. But I do hope, that going forward, we can develop both social and technological solutions to this serious problem.

• Brainrot: My experience with these models has given me a bit of a “black mirror” moment. What the models do, the capabilities they have, and the quality of the content they produce is, to a large extent, a reflection of the user. They can be an incredible tool for a serious an seasoned developer building software, or for a mathematician working through a complex open conjecture, and even for an ambitious and dedicated teenager looking to master a new skill. However, they can also, absolutely, act as an “easy” button to generate zero-to-little effort content; output that may be passable, but is of no real value. It seems clear that we are well into the regime where a student can choose to put in almost no effort, have one of these models complete a (e.g. programming) assignment for them, and completely miss out on the educational goal. While there is a lot of thought going into how to incorporate AI tools into curricula, and while that may be an important line of inquiry, I think it is incomplete. The real problem that we need to tackle, that predates the challenges raised by AI, but that is newly-magnified by their capabilities, is the very human and social one. How do we motivate actual learning? How do we convince people that learning, itself, is the goal, and to put in real effort when there may be easy ways to game the evaluation system (i.e. grades)? Motivated and truly curious people will learn with or without AI tools, but these tools make it much easier for folks not so-inclined to move through the educational system without learning or exercising critical thinking skills, and perhaps even fooling themselves into thinking they are.

• Widening the gap: The more I experiment with these tools and the more I see how capable they are, the more I also see how savant like they can be. Their capabilities are incredible, but their mistakes are often absolutely silly (to a seasoned expert). Building on the “black mirror” comment above, I think that it may be the case that, rather than having a “rising tide” that “raises all ships”, these tools may further widen the divide between the most effective, capable and knowledgeable experts, and everyone else. It is the seasoned greybeard, who has spent months of cumulative time tracking down and squashing subtle heisenbugs, who can immediately see and correct the silly but obvious (to them) mistake that Claude just made. It is the accomplished algorithm engineer who knows exactly how a specific data structure needs to be laid out to maximize cache efficiency and therefore performance. It is that algorithm engineer, and the knowledge they bring to the tool, that lets them guide the model to the right solution, and not just a solution. Ultimately, I fear that if we are not able to effectively teach and instill that knowledge and experience in those who are now undergoing that critical stage of their development, we may be creating, in some ways, an expertise cliff. For those who have that expertise today, it has often come through hard-earned knowledge, manual construction of sophisticated systems from first principles, and a lot of persistence and banging their heads against the wall. There may be ways to create that level of expertise and knowledge without all of the associated “manual” exercise, but those ways are as yet, to me, unclear.

What is Cargo?

Cargo is the package manager for Rust and, more than that, it’s essentially the build system and project management system. Cargo can be used to initialize the skeleton for a new project, to build your program’s executables or libraries, to run your unit or integration tests, to generate the documentation for your program, to run benchmarks, and to perform a host of other useful actions.

In fact, Cargo does so much that I’m not going to attempt to cover it all in this post. There is entire online book dedicated to Cargo, its use, and its capabilities. Instead, I’m going to focus mostly on Cargo’s function as a depndency / package manager.

So, before I go into details, the TLDR is that Cargo is an amazing, easy-to-use, powerful, and intuitive build system that makes pulling in dependencies trivial, provides mechanisms for semantic versioning-based dependency resolution, reproducible builds, and automatic upgrading of dependency versions. More than build systems I’ve encountered in any other language, Cargo “just works”, and it makes building projects in Rust, even those with substantial sets of dependencies, quick and easy.

Declaring dependencies with Cargo

Cargo relies on a TOML format file called Cargo.toml that describes certain metadata about your project, including its developers, what it does, how it is structured, its dependencies and its relevant compiler options. At a high level, Cargo is a declarative system (it is possible to construct “imperative” build scripts — so-called build.rs files — but they are not needed for many projects), and building your project is as simple as declaring what type of project it is, listing your dependencies and preferred compiler options, and running cargo build --release.

As a non-trivial running example, I’ll be using the Cargo.toml file from our alevin-fry tool for single-cell and single-nucleus RNA-seq processing. The first part of the file describes the package, including metadata like the package name, version, authors, etc. Now, not all of these fields are strictly required, but it’s nice to populate your Cargo.toml with relevant metadata that will make tracking and organizing it easier in the context of other packages.

[package]
name = "alevin-fry"
version = "0.8.0"
authors = [
  "Avi Srivastava <avi.srivastava@nyu.edu>",
  "Hirak Sarkar <hirak_sarkar@hms.harvard.edu>",
  "Dongze He <dhe17@umd.edu>",
  "Mohsen Zakeri <mzakeri@cs.umd.edu>",
  "Rob Patro <rob@cs.umd.edu>",
]
edition = "2021"
description = "A suite of tools for the rapid, accurate and memory-frugal processing single-cell and single-nucleus sequencing data."
license-file = "LICENSE"
readme = "README.md"
repository = "https://github.com/COMBINE-lab/alevin-fry"
homepage = "https://github.com/COMBINE-lab/alevin-fry"
documentation = "https://alevin-fry.readthedocs.io/en/latest/"
include = [
  "/libradicl/src/*.rs",
  "/src/*.rs",
  "/Cargo.toml",
  "/README.md",
  "/LICENSE",
  "/CONTRIBUTING.md",
  "/CODE_OF_CONDUCT.md",
]
keywords = [
  "single-cell",
  "preprocessing",
  "RNA-seq",
  "single-nucleus",
  "RNA-velocity",
]
categories = ["command-line-utilities", "science"]

Most of these fields are self-explanatory, and the syntax is quite straightforward. The entries are a series of key-value pairs, where the value can be a string, a list, or (as we’ll see below) a nested key-value store. Perhaps the only non self-explanatory key here is the edition key. The idea of rust editions are described here, and they essentially describe small backwards incompatible language changes as well as certain default behaviors. Currently “2021” is the most recent edition of rust, and that is what we set here.

The actual dependencies are declared in a section labeled — unexpectedly — as “dependencies”. A short excerpt is below:

[dependencies]
# for local development, look in the libradicl git repository
# but when published, pull the specified version
libradicl = { git = "https://github.com/COMBINE-lab/libradicl", version = "0.4.6" }
anyhow = "1.0.59"
arrayvec = "0.7.2"
ahash = "0.7.6"
bincode = "1.3.3"
bstr = "0.2.17"

This demonstrates several important details about the dependency management system exposed by Cargo. The first thing is the simple manner in which dependencies are declared. Each dependency is the name of a crate (the terminology that Cargo uses for dependencies), followed by a version constraint. In general, Cargo crates follow semantic versioning, and the default syntax for declaring a compatible versions “X.Y.Z” means that you are willing to accept any version compatible with “X.Y.Z”. So this would match, for example, “X.Y.(Z+1)” or “X.Y.(Z+2)”, but not “X.(Y+1).Z”. You can also declare a constraint as “X.Y” which would allow “X.(Y+1).Z” but not “(X+1).Y.Z” etc. You can even declare dependencies as “X”, which allows any version >=X and <(X+1). The full syntax for specifying dependency constraints is quite powerful and flexible, and you can read more about it here.

The second thing to note is that some dependencies have a more complex description. For example, the first dependency is libradicl, a library that we also developed that is hosted on GitHub as well as on crates.io. You’ll note that the declaration of the dependency lists two different sources, a git source and a version source. This is a nice feature of Cargo. When the program is built locally, it will pull the relevant dependency from the GitHub repository listed in the URL. This allows one to develop coupled packages with ease, by allowing a program to always pull in its dependency with the latest commits directly from a remote (or local) repository. Yet, when your package is later published (more on that when we talk about crates.io), it can’t rely on dependencies tracked in Git. For that, you must instead declare a dependency on another crate that is hosted on crates.io — here, we rely on version 0.4.6 of the libradicl crate (or any version compatible with this declaration). This leads to a fairly fluid development experience, where, when working on the alevin-fry tool, if we need to make a change to libradicl, we first make the changes upstream in GitHub (pulling directly from the repo during development). Then, when we are satisfied with the changes that we wish to make, we push a new version to crates.io, and bump the version string in the libradicl dependency to this new version. It’s also worth noting the ease with which the release-please GitHub action and rust action allows tagging a new version and automatically pushing the resulting release to crates.io.

If you look farther down in the Cargo.toml file, you’ll notice that some other dependencies contained different invocations in their declarations. While the documentation provides a full accounting of how these different properties work, most of them are actually rather self-explanatory. For example, the declaration below is a dependency on the brilliant serde serialization crate. In addition to the version constraint, we also have a property features = ["derive"]. In rust, crates may have default and optional “features”, these describe functionality that the crate can be built without or that it can provide. Here, we are declaring that we wish to enable the “derive” feature of the serde crate (which lets us use the derive macro to quickly build out serialization and deserialization for the structs and types in our program).

serde = { version = "1.0.136", features = ["derive"] }

Declaring dependency version constraints

Cargo allows several ways to declare constraints on dependency versions. The default behavior “X.Y.Z” is equivalent to the syntax “~X.Y.Z” which restricts the dependency from being satisfied by another version that makes breaking changes. If you want or need to pin your dependency to a specific version, you can use the syntax “=X.Y.Z”, which will require pulling down exactly this version of the corresponding crate. There are many other types of constraints you can place on the dependencies (e.g. “>X.Y.Z”, etc.). These various constraints and how they work is documented nicely in the book.

Resolving dependencies (and the Cargo.lock file)

When you ask Cargo to build your program, it will resolve the relevant dependencies, downloading the corresponding crates from crates.io (or other sources like GitHub if you have specified those) and building them to be linked with your program. In the process of doing so, it’s performing dependency resolution. That is, it will find a corresponding set of versions that, given the current state of crates.io (i.e. the current set of available versions of all of the crates on which your program depends), satisfies all of the constraints on versions you requested. Generally, subject to these constraints, it pulls down the newest possible versions. This behavior is great, because this means that if a corresponding crate updates their latest available version with something that is compatible (in terms of semantic versioning and your specified constrains), then you can get this updated version just by re-building your program.

Of course, there are situations where, for the purposes of reproducible builds, you may wish to be a bit more strict on how dependencies are resolved. Cargo’s way of allowing this is what is called the Cargo.lock file. The contents of the Cargo.lock file look somewhat different than those of the Cargo.toml file (and they are generated by Cargo itself, so you’re not responsible for making this), and the book has a section on these. For example, an entry from the Cargo.lock file for alevin-fry looks like this:

[[package]]
name = "anyhow"
version = "1.0.65"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "98161a4e3e2184da77bb14f02184cdd111e83bbbcc9979dfee3c44b9a85f5602"

This specifies the specific resolution for a dependency that was obtained during the solve when the program was built. When Cargo attempts to build your program, before it attempts to check the available upstream crate versions and resolve your program’s dependencies, it first checks for the existence of a Cargo.lock file. If this file is present, then it will simply use the versions declared therein (that is, it will re-use this “solve” of your set of dependencies). One thing that’s really nice about this is that it’s possible to essentially “freeze” a build using the Cargo.lock file, such that, if some upstream dependency fails to properly use semantic versioning and makes a breaking change with a “patch” bump, builds that use the successful Cargo.lock file won’t be affected.

The standard recommended practices around Cargo.lock files is actually something that I only learned relatively recently. Initially, I’d assumed that these generated files were essentially not for user consumption, and so I added the Cargo.lock files to my .gitignore list for my repositories and kept them out of version control (they are small, so this was for the purposes of keeping a clean file history rather than for worrying about repository size). However, I since learned that recommended practice is basically the following: If you are building a user-facing program or tool, then you should include the Cargo.lock file in your version control and in your set of distributed source files; On the other hand, if you are building a library for other tools to pull in and depend upon, then you should generally not include the Cargo.lock file in your version control and distributed source files. Nonetheless, the Cargo.lock file is a neat solution to the problem of reproducible builds and solves. Even when a specific dependency is “yanked” from crates.io (essentially, the authors of a crate can “unlist” a specific version of their crate), if you are in possession of the Cargo.lock file that solved using this yanked crate, your build will still be able to pull it down and compile against it. In other words, even if certain versions of a crate are no longer publicly listed, the Cargo.lock file lets you re-create a build with the precise versions used before, making it easy to reproduce the set of dependencies of a previous build exactly.

Cargo-edit

Cargo has a plethora of different commands it exposes, many are built in and some come a “plugins” that expand the capabilities of Cargo. One particularly cool plugin that I wanted to mention is cargo-edit, and specifically, the cargo upgrade command. When your project has several dependencies, tracking and upgrading those dependencies can be a pain. The cargo-edit plugin provides commands to help manage the contents of your Cargo.toml file, including the add command to add an entry for a new crate (given its name and the set of features you want), and to remove (rm) dependencies. It also includes a brilliant upgrade command that scans your list of dependencies, determines which can be safely upgraded, modifies your Cargo.toml accordingly, and also reports which packages can’t be upgraded given your current versions and constraints. For example, running cargo upgrade on alevin-fry (as of commit a77c96e162758e8cf5f4e509263216158bb580c9) gives the following output:

    Updating 'https://github.com/rust-lang/crates.io-index' index
    Checking alevin-fry's dependencies
name            old req compatible latest  new req note
====            ======= ========== ======  ======= ====
ahash           0.8.1   0.8.2      0.8.2   0.8.2
crossbeam-queue 0.3.6   0.3.8      0.3.8   0.3.8
flate2          1.0.24  1.0.25     1.0.25  1.0.25
serde           1.0.147 1.0.148    1.0.148 1.0.148
serde_json      1.0.87  1.0.89     1.0.89  1.0.89
snap            1.0.5   1.1.0      1.1.0   1.1.0
chrono          0.4.22  0.4.23     0.4.23  0.4.23
mimalloc        0.1.31  0.1.32     0.1.32  0.1.32
clap            =3.2.16 3.2.16     4.0.27  =3.2.16 pinned
   Upgrading recursive dependencies
note: Re-run with `--pinned` to upgrade pinned version requirements
note: Re-run with `--verbose` to show all dependencies
  unchanged: anyhow, arrayvec, bincode, bio-types, bstr, crossbeam-channel, csv, indicatif, itertools, libradicl, needletail, num-format, num_cpus, petgraph, rand, rust-htslib, sce, scroll, slog, slog-async, slog-term, smallvec, sprs, statrs, thiserror, typed-builder

So we see what the current upgradable dependency is, the latest compatible version, the latest version (ignoring compatibility), and version to which our dependency has been upgraded. Finally, as you can see in the case of the clap dependency, if you have specific constraints that preclude upgrading a crate, it will also include relevant notes. While the upgrade command will not perform version breaking upgrades by default, you can pass the -i, --incompatible option to allow upgrading to an incompatible version and the -p, --package argument to target a specific package. The command also has a --dry-run flag to show you what upgrades would be made without actually performing them.

Overall, cargo-edit makes adding, removing, and upgrading your dependencies easy, by taking the monotonous grunt work out of parts of the process that really should be automated.

crates.io — The source for official dependencies

I have mentioned crates.io above many times. It is the official registry for rust language crates (dependencies), and currently home to >98,000 different crates! You can browse the crates by category or search for them by name. Moreover, when you start building your own libraries and tools, you can easily host them on crates.io for free. All you have to do is register and use the cargo publish command to upload your locally developed crate to the crates.io registry. After that, you (and others) can add dependencies on your crate simply by adding the appropriate declaration to your Cargo.toml file as we have discussed above. In my opinion, one of the brilliant things about the rust ecosystem, is how the ease of both using and publishing crates encourages the development of small, modular, and reusable components in rust software. There are crates for a host of different purposes, and it’s trivial to make your own. When you make a crate to serve a specific purpose, it is then easy to reuse it across many projects by simply declaring it as a dependency. In my opinion, this works much better than the alternatives in languages like C/C++, where it is common to either vendor your dependencies and copy (potentially different versions) into the source tree across many projects that use them. While certain package management solutions for C++ exist, like conan and vcpkg, these are all 3rd party solutions and they lack the scope and breadth of cratres.io, and also the tight and elegant integration with the rest of the development ecosystem that is enjoyed by crates.io and cargo. In my (admittedly biased) opinion, the dependency management solutions provided by rust are phenomenal, and probably the best among any language in which I’ve worked — and this includes non-compiled languages such as Python and R.

Some fun bioinformatics crates

I’ll close this post by mentioning that a search for bioinformatics on crates.io turn up 156 results (and related terms turn up more e.g. genomics turns up 112). I encourage you to go exploring yourself! However, it is worth mentioning some common crates in the bioinformatics space that are pretty awesome:

• The bio crate is a bioinformatics library for Rust that provides implementations of several critical data structures (e.g. the FM-index) algorithms (e.g. alignment) and parsers (e.g. GTF). It’s a great place to start if you’re looking for a crate that tackles many common problems

• The seq_io crate is a particularly fast FASTA/Q parser. There are several such crates, so it’s worth exploring the different options here.

• The rust-htslib crate provides Rust bindings for the venerable htslib C library for reading and writing SAM/BAM/CRAM files.

• Related to the above, the noodles crate provides readers and writers for “BAM 1.6, BCF 2.2, BED, BGZF, CRAM 3.0, CSI, FASTA, FASTQ, GFF3, GTF 2.2, SAM 1.6, tabix, and VCF 4.3.” written entirely in Rust (so no binding to an external C library). It’s definitely a crate to keep an eye on in terms of native Rust support for these common file formats.

• The debruijn crate from 10x genomics provides a de Bruijn graph implementation in Rust. In fact, 10x is quite prolific in terms of creating Rust crates in the bioinformatics space, many of which you can find here — including a Rust implementation of the BBhash minimal perfect hashing algorithm.

• If you’re doing sequence alignment in Rust, definitely check out the block-aligner crate by Daniel Liu, for a high-performance, SIMD-accelerated, block-adaptive sequence alignment algorithm.

This is in no way a comprehensive list, but I would absolutely appreciate feedback if there are crates that you use regularly that you’d like me to list here! There are already a ton of great tools in Rust in the bioinformatics space, but the list above is mostly for library-level / reusable components.

So, an obvious question a reader may ask is “why?”. Why am I so zealous in my advocacy for Rust and its use in Bioinformatics? What benefits do I think it provides over the alternatives? What even are the alternatives? Are there places in bioinformatics where I don’t think Rust is the right choice?

I intended this post to be an initial foray into addressing these questions, but quickly realized that, at least with the current queue of things that need my attention, a sprawling, long-form blog post was not the way to go. Therefore, I am hopeful that this will become a series of posts, each one small and bite-sized, that together lay out some of my thoughts on the use of Rust in bioinformatics, each touching on a smaller piece of the whole picture. Yet, my prior blogging discipline is lacking, so I will make promises at this point about the length or frequency of this series. So, without further ado, let’s get started.

In today’s post, I simply wish to define the problem space, that is usually implicit in my comments, when I advocate for the use of Rust in bioinformatics.

Defining the problem space

When I advocate for the use of Rust as a language for developing bioinformatics methods and tools, I do so from my own particular perspective. Bioinformatics is a giant field, with many sub-disciplines, problem areas, and methodological approaches. Perhaps, Rust is applicable everywhere here, but that is not the argument I mean to make. Rather, I would like to advocate for the use of Rust when developing tools and methods for data-intensive, high-throughput analysis.

The types of applications I have in mind are sequencing indexing, read mapping and alignment, genome and transcriptome assembly, bulk and single-cell RNA-seq and metagenomic quantification, etc. These applications are characterized by a need to process a large volume of input data — often what many consider as raw input data. These applications have several characteristics that tie them together. They often require reading and sometimes writing large quantities of data. They often require low-level and / or binary parsing and interpretation of records. Given the problem sizes we encounter in practice, these are usually applications where memory usage is a concern and so they often rely on efficient (in space and time) implementation specific data structures. Further, in many such applications, memory needs are often semi-regular and predictable. Also, these applications often have components or subroutines that can be made embarrassingly parallel. There are other things such problems have in common, but these will suffice for now.

This, of course, leaves out huge areas of bioinformatics — problems where we have processed or pre-processed data, and we want to perform exploratory data analysis, or particular types of statistical testing, or do certain types of dynamical modeling. It is not that I don’t think Rust could be a compelling choice for these types of applications (though I have my doubts about exploratory data analysis), it is just that these are not the application areas where I typically work, and so they are not the areas where I have the experience or confidence to advocate for Rust (yet).

So what other languages occupy this space?

To argue for what makes Rust compelling, I first have to lay out what I think are the other common (and, perhaps, not-so-common) languages used for these types of problems.

C and C++

Perhaps the most common languages are C and C++. I mention these together, as is common, but it is critical to understand that they are very different languages. The C language is relatively small and in many senses minimalistic. It provides a small set of tools for modeling problems and for interacting with the system. It has a standard library, but a comparatively small one by modern standards, and things like collections must be written or provided as third-party libraries. I won’t say too much else about C here specifically, except that almost all the issues I raise about safety with respect to C++, and often times they are even worse. Since the C type system isn’t as rich as the one in C++, C often makes specific procedures generic by simply casting around data into a payload (e.g. a void*) rather than generating type-safe code for specific invocations at compile-time (e.g. C++’s templates and monomorphization).

As opposed to C, the C++ language is a giant monster. First, you must ask, to which C++ is one referring? C++98, C++03, C++11, C++14, C++17, C++20? If we consider the modern variants of C++ (e.g. C++11 and later), then these languages have added many useful features (but also a huge amount of complexity) to what they offer. While there is a substantial amount shared between Rust and C++ that I hope to cover in future posts, one of the biggest areas they differ is in the way that they handle “safety” — that is, how the user interacts with memory and mutable data, and how those interactions affect program behavior and state. Put simply, Rust aims to be a safe language (with the option to perform tightly-scoped unsafe operations via the use of the unsafe keyword), while C++ is absolutely not a safe language. Now, to be sure, many in the C++ community have recognized the importance of safety — how unsafe code can lead to security vulnerabilities, incorrect results, and unexpected program crashes or other behavior. However, the language itself has incredibly limited support in terms of tools to helping the programmer to write safe code. Though there are efforts at laying out best practice like the C++ core guidelines, there is a huge qualitative difference between best practice guidelines that a programmer should know and follow, versus safety guarantees provided by the language and compiler itself. There are many other differences between these languages, but I’ll stake out the approach to safety to be perhaps the most salient.

The AoT, compiled, GC languages

While they are perhaps less-widely used for tools like the ones I’ve laid out above, there are a host of languages that are still designed to provide the computational performance necessary to accomplish such tasks. Here, I’ll group together the most popular ahead-of-time (AoT) compiled and garbage collected (GC) languages. This includes languages like Java (and other JVM languages like Scala and Kotlin), as well as different languages that nonetheless adopt the GC approach to memory safety, like Go. There are also languages that mix in a GC with other memory management strategies and provide the user with “opt-in” garbage collection (e.g. D and Nim). However, since people using these languages tend to produce programs that eventually make use of GC somewhat, I’m grouping them in here, though technically there are ways to avoid the GC there.

These languages go a very different route that C/C++, and they do, to a large extent, provide important types of memory safety. However, they do this at the runtime cost of having a garbage collector, a runtime component of the language that tracks allocated memory and is responsible for ensuring that memory is kept alive while it can still be accessed and freed (eventually) when it is no longer in use. The progress in the theory and practice of building scalable garbage collectors has been astounding, and many modern GCs are marvels of engineering. Nonetheless, the very presence of a GC imposes a runtime overhead in the presence of heap allocations, and it has been observed that idiomatic reliance on the GC for memory management can typically impose a memory overhead of up to 2 times over languages where memory is managed manually (and responsibly). One reason for this (though not the only one) is that such languages tend to make many more heap allocations than languages like C/C++/Rust, and store fewer things on the stack. So, the place where these differences show up most frequently when comparing GC’d languages to something like C/C++/Rust is that (a) the GC’d languages typically use a constant factor more memory and (b) long-running or memory intensive processes typically incur GC “pauses” when the garbage collector kicks in to reclaim large amounts of memory. Of course, there are several strategies that can be (and often are) used to mitigate these issues (e.g. like retaining and using small object pools to avoid the garbage collector becoming involved in common and reusable allocations), however such strategies often require extra work (sometimes substantial), and are typically not idiomatic in the language.

Therefore, AoT compiled GC’d languages can be quite fast (though typically they take some speed hit compared C/C++/Rust), and they are, in many important ways safe, but this often comes at the expense of GC overhead in terms of both runtime and memory usage.

Others

The list above is in no way meant to be comprehensive. Further, as we have been going through a renaissance of types in the development of new programming languages, there are also many other (newish) languages that could reasonably by said to occupy a similar space. For example, a language like Zig aims to be a modern systems-level language, and brings with it some very compelling features. Since I don’t know too much about these other emerging languages (including Zig), I won’t attempt to contrast them too deeply with Rust. However, I’ll note that while Zig adds features on top of C that are a boon for safety, it explicitly does not make the same guarantees or claims on safety as does Rust. There are also other langauges like Pony of whose existence I am aware, but about which I know even less. Thus, I won’t be trying to argue in future posts that Rust is the only new laguage that is well-suited to developing high-throuhgput bioinformatics tools and methods, but rather that it is a great choice for this task, and that in the space of such newish languages, it is certainly one of the most widely-adopted and mature.