This week I’ve seen a lot of posts characterizing RSpec and factories as inherently slow and inferior to Minitest and fixtures.
The premise that spec-based development with factories is inherently slower than minitest and fixtures didn’t pass my smell test, so I dug in to find out why people are seeing such dramatic speed-ups in these rewrites, and figure out if something could be fixed with RSpec or FactoryBot so we could keep our nice syntax.
Factories are slow
Ok so turns out, factories are slow. Before I dug into this, I didn’t understand exactly why, but I found out that it’s way too easy to accidentally create boatloads of records when setting up factories.
The reason? Setting up a single user factory might seem fast at first glance, but it might have to create an associated account record, which might have associated billing records, which might have … it turns out these graphs can get quite large.
Fixtures are fast because the data is loaded directly into the database for a suite, then each test starts a transaction and rolls it back. So much faster!
Fixture syntax is tedious
The reason I don’t like fixtures isn’t because I like factories, it’s because I like the syntactic sugar of factories.
I did what any self-respecting Rubyist would do and created a gem with the syntactic sugar of factories and the speed of fixtures called FixtureBot.
Here’s what a FixtureBot looks like for a blog application:
FixtureBot.define do
user.email { "#{name.downcase}@blog.test" }
user :brad do
name "Brad"
email "brad@blog.test"
end
user :alice do
name "Alice"
# "alice@blog.test" is auotmatically generated
end
user :charlie do
name "Charlie"
email nil # Don't set an email for Charlie!
end
tag :ruby do
name "Ruby"
end
tag :rails do
name "Rails"
end
tag :testing do
name "Testing"
end
post :hello_world do
title "Hello World"
body "Welcome to the blog!"
author :brad
tags :ruby, :rails
end
post :tdd_guide do
title "Getting Started with TDD"
body "Test-driven development is a practice where you write tests before code."
author :alice
tags :ruby, :testing
end
comment :great_post do
body "Great post, thanks for sharing!"
post :hello_world
author :alice
end
comment :helpful do
body "This was really helpful."
post :tdd_guide
author :charlie
end
comment :follow_up do
body "Could you write a follow-up on mocking?"
post :tdd_guide
author :brad
end
end
When a test suite is run, FixtureBot dumps the resulting graph into the Rails fixtures path and that’s it. Rails fixtures takes care of the rest.
Stable IDs
One important thing about fixtures is they remain stable. The ID of a fixture should never change. FixtureBot is designed with that in mind. The IDs of records are derived from the name of the record and the table, so [:users, :admin] gets an integer that becomes the database ID that will never change unless :users or :admin names change.
How it works
Install the gem:
gem "fixturebot"
Then generate your fixtures:
$ rails fixturebot:generate
That’s it. Fixturebot reads your factory definitions and writes out YAML fixture files that Rails can load with its built-in fixture mechanism.
What about RSpec?
Is RSpec inherently slow? Depends. On one core, RSpec and minitest practically run at the same speed. Minitest pulls ahead when workloads are run across multiple cores, which practically speaking is how test suites should be run on modern machines that have 12+ cores.
The parallel_rspec gem
The good news for RSpec is there are gems that can be installed that run specs in parallel, like the parallel_rspec gem.
Want to hear the bad news? While it speeds up the overall RSpec suite by a lot, it doesn’t do as good of a job breaking apart the suite and scheduling test runs as minitest.
minitest expectations
Minitest ships with a spec-like syntax, but it’s not as expressive as RSpec’s syntax and I find the _() at the beginning of everything meh.
Will RSpec ever support parallelism out of the box?
I asked why RSpec doesn’t run in parallel and the answer is that it wasn’t built for it.
I’d love to see somebody dive deep into the problem with an LLM, but there’s also sus.
The sus gem
The sus gem describes itself as “It’s similar to RSpec but with less baggage and more parallelism.”
Like RSpec, it has a specification-style syntax, but it was designed with parallelism in mind.
A modern spec stack
Why did I bother with FixtureBot and writing this article when we live in a time where an LLM could convert everything from RSpec and factories to Minitest and fixtures? Because fundamentally I believe code is written for humans to read, not for machines to execute.
As LLMs write more code, I’m going to spend more time reading it, and I want something pleasing to read that’s fast. I figured it would be worth spending a night understanding why factories are slow and see if I could come up with a better solution. FixtureBot is half of it, sus is the other half.
Now I need to roll FixtureBot and sus into my OpenGraph+ repo and see what happens. By the way, you should add OpenGraph+ to your Rails app so your site looks decent when you share it on social media or in chat apps.