Diagnosis
Introduction
Mathematicians have a saying; "if you can't prove it, you don't know it." Or at least, I said this a lot while I was fumbling my way through a maths degree. The point I'm making is that it's very easy in this game to let your hunches cloud your judgement, or to turn something slightly inefficient you noticed once become the sole focus of your attention in chasing performance. I've learned (as we all do) that a hunch is often a good starting point, but you have to measure to be sure.
Big handfuls
A good starting place is at the really high level. If your program isn't running as fast as you'd like, the reason can be broadly split into one of three categories:
- The CPU is saturated and simply can't process your instructions any faster.
- You're using so much memory that you're using the disk as additional memory (very slow).
- The CPU is sitting around waiting for something else to happen first.
These are really broad and not very well defined, but you get the point. You're using all the CPU, or you're using so much memory that you're having to use slow alternatives (e.g. hard drive), or you're waiting around for something else to happen (e.g. file reading or writing).
Each of these problems has a different solution; do not guess what the problem. Measure. Record. Be sure what the problem is, and then fix it. Spending your time massively optimising something that isn't actually the bottleneck is a waste of your time.
Good first tools for this are top and htop. Both pretty standard on Linux distributions (and I've seen htop on FreeBSD). It will show how busy your cores are, and who's eating all the memory. Let's take a look at invoking top.