This allows us to get back to single binary tests where appropriate, which will help in-depth profiling efforts. I've built this atop my smoke_test changes as they inspired me to get this done.
