In video games development, there is a need for a specific type of a math library to deal with tasks like 3D transformations, collision detection, etc. Primarily, such a library contains types for 3D and 4D vectors and matrices, quaternions, and the associated operations.

Most game development companies wouldn't even trust something as simple as a standard string, so it is only natural that the industry hasn't settled on a standard math interface: every team designs their own "math library".

In fact, many games contain multiple math libraries which do the same thing but use slightly different interfaces. That's because of the emergence of middleware: things like physics simulation libraries or sound libraries tend to use their own math types and functions.

Reinventing the wheel?

In many cases it can be argued that you need custom code because available libraries don't quite fit your needs.

But let's face it: a matrix is a matrix and a vector is a vector. You don't even have to understand the theory behind all this to whip up "your own" implementation.

You don't even have to copy the code from a book: this type of code has been written so many times by so many programmers that a simple search would get you everything you need: copy, paste, change a variable name or two to match your naming conventions, and voila, you have the best math library on earth. :)

So why does everyone keep doing it?

In trying to justify this behavior, you'll hear people talking about SIMD optimizations, hand-crafted assembly code, and of course the always relevant "Dude, you can't do this on the Playstation!"

But the net effect is that very soon the math library needs to be tweaked, and then tweaked some more, and in a few iterations it starts getting quite messy.

And then it becomes so messy that it gets rewritten: "this time, we'll get it right and it'll be perfect!"

And so on and so forth, GOTO 10, for(;;), while(true).

Insanity?

I believe it was Einstein who said that "Insanity is doing the same thing over and over again and expecting different results."

The problem is that in a math library design there are two competing forces: the desire to make the library easy to use (because it's needed in virtually all engine systems) and the desire for maximum optimizations (assuming we're talking about realtime video games.)

But as stated, the problem has no solution. It doesn't matter how many times you rewrite your math library, as long as you want it to be both easy to use and optimal, it's going to be a compromise.

From this point of view, game developers aren't insane. It's just that each team settles down on a different compromise for a while. When this compromise doesn't work any more -- because it's not easy enough to use or because it's not optimal enough -- the math library is tweaked if possible, and rewritten otherwise.

An interesting analogy

The quest for the perfect video game math library is like the quest for the perfect class hierarchy.

Let's say you want to design a class hierarchy for your game objects. Let's say you want to have a class called drawable, for objects that can be drawn, and another class called tickable for objects that need to be ticked on each frame to update their state.

If you want to waste a few hours at work, get a few programmers in a room and start a discussion whether tickable should derive from drawable or the other way around. :) Either way, there are practical use cases that won't fit the class hierarchy.

The solution is to realize that cramming everything in a single hierarchy makes no sense at all, despite that entire languages -- Java and Smalltalk come to mind -- have been designed on that flawed principle (btw this statement is also pretty good for starting a time-wasting discussion at work.)

But to get back to the topic of this post: why can't a math library be

Both easy to use and optimal?

The answer is that the level of abstraction that makes a math library easy to use pretty much guarantees that it can't achieve maximum performance in many practical scenarios.

This is actually easy to demonstrate: suppose you want to transform a large array of 4D vectors as fast as possible. Many people think that you can't get more basic than

If you're thinking that I'll be talking about alignment requirements and cache misses, that's not my point (btw, adding alignment requirements to float4 compromises ease of use because that makes it harder to port, tricky to use in containers, etc.)

The issue is that the domain of a "math library" is so large that it's impossible to know if float4 as defined above is optimal or not. What if the fastest way to transform an array of 4D vectors on a given platform is to pass the input not as an array of float4, but as 4 separate float arrays? Clearly, in that case no amount of SIMD, alignent requirements, or hand-crafted assembly can help.

The only solution is to design a special interface for transforming many 4D vectors that doesn't take a float4 array. That's in fact easy to do.

And if this was the only performance-critical part of our game, we don't have to worry about the math library being optimal: it just has to be easy to use.

The problem is that there are many performance-critical parts of a video game. So what happens is that you give your team a nice and easy to use math library, and soon it's used in many performance-critical interfaces. So many in fact that you figure that if you speed up matrix_multiply() by a factor of 8, you'll get 20% boost in framerate.

And then you SIMD and __align__(16) your way to... well, a compromise.

Conclusions

A) The only way to allow maximum performance for math operations is to not use the math library in performance-critical interfaces (note that at first such an interface can and should be implemented using the math library.)

B) The only way to ensure that the math library remains easy to use is to never assume that it will give you maximum performance.

By the way, less controversially, A) and B) are also applicable when talking about the memory manager. But that's for another post. :)

Formatting hint: when posting comments, surround code blocks in [@ and @].