Anatomy of a buffer overflow - learning from Apple's latest security update

Apple has released its latest Security Update for OS X.

Dubbed simply 2013-003, the update fixes a trifecta of memory corruption bugs in QuickTime caused by buffer overflows.

Technically, one of the bugs is listed as a buffer underflow, which is just a buffer overflow the other way round.

An overflow writes past the end of your own memory buffer, trampling on the next block of memory, which may well be in use for something else; an underflow writes in front of your memory, with a similarly risky outcome.

These bug fixes should all be considered critical, because they could be exploited for remote code execution.

In other words, a deliberately-tweaked movie file could trick your Mac into running program code hidden in the movie itself, even though such files are supposed to consist entirely of data.

Getting the update

As usual, Apple Menu followed by Software Update... will sort you out if you want a single-click update. (Annoyingly, but not entirely unusually, a reboot is required to install this update, so don't do it smack in in the middle of something interesting or important.)

If you want to do the update by hand or to grab the installer now in case you ever want to reinstall OS X, here is a table of co-ordinates:

The flaws described

All three of the flaws get a very cursory explanation from Apple, limited to a boilerplate comment for each one, in this format:

Buffer overflow in Apple QuickTime...

...allows remote attackers to execute arbitrary code or cause a denial of service (application crash) via N

The various values of N are:

• CVE-2013-1019: N = "a crafted movie file with Sorenson encoding."
• CVE-2013-1018: N = "a crafted movie file with H.264 encoding."
• CVE-2013-1022: N = "crafted mvhd atoms in a movie file."

So much for the update.

Why movie files?

Why, you may be wondering, do files such as images and movies, as well as documents of various sorts, appear frequently in this sort of vulnerability report?

What is it that makes files of this type especially suited to "crafting"?

→ Crafting a file is geek shorthand for manipulating a file so that, although it is out of specification and ought to be rejected by the application it is targeting, it will nevertheless load and deliberately trigger dangerous misbehaviour in that application.

Part of the problem with multimedia files is that they are often pacakaged as a giant blob of binary data that resembles machine code for a virtual machine.

The files consist of opcodes, also known as instructions, and variably-sized operands, or data to go with the instructions.

So, an application that parses and processes a file like this has to take it apart byte-by-byte and manipulate it in memory, often relying on the correctness of individual data items inside the blob itself to work out what to do next.

A QuickTime file, for example, consists of a sequence of instructions calledatoms that each require special processing, and which can be nested and packaged together into MOV files of considerable complexity:

Each atom starts with a fixed four-byte value that tells you how big it is.

Here you can see the main Movie atom in a sample MOV file, denoted by the header moov, and stated to be 47,074 (0xB7E2 in hexadecimal) bytes long:

The MOOV starts with an embedded Movie Header of 108 (0x6c) bytes, labelled with mvhd:

The MVHD is followed by a Track, labelled with trak:

And the TRAK, already nested inside the MOOV, itself contains a nested Track Header, denoted tkhd:

It's already looking pretty complex, and that's before you've even tried to turn the MVHD or the TVHD atoms into any meaningful data inside your application.

The MVHD vulnerability

You'll note that the MVHD atom is one of the object types listed above as exploitable.

We don't know exactly what form the bug took, and Apple will probably never publicly say, but we can use the sample file above to remind ourselves of the sort of problem that processing this type of data can cause.

So let's speculate - it doesn't matter if we're right or wrong, as our goal here is just to think about the sort of mistake that is easy to make when coding.

Firstly, notice that the size marker of each atom includes the 4-byte size of the size marker itself.

So you might write code like this to process an atom:

That's not going to work, because size includes the size of itself, so there are only size-4 bytes of data:

This code works, but it's fragile.

For example, what happens if someone deliberately creates an atom with asize of 1?

You should notice that and reject it, since the mimimum size of an atom is the 4-byte size field.

If you don't look for and block this error, then a "crafted" size of 1 makes your code do this:

What happens here depends on many things, notably whether the calculationsize-4 is done using signed or unsigned integers.

If you subtract 3 from a value of zero treated as a 32-bit unsigned number, you can't get a negative result, because unsigned integers wrap around at zero.

It's like winding a clock back one hour from midnight: you don't get minus-one o'clock, you get 11pm on a 12-hour clock or 23:00 on a 24-hour clock.

So an unsigned 32-bit value of zero with 3 subtracted turns into 232-3, or 4,294,967,293, so your code suddenly does this:

In this example, an integer underflow leads to a buffer overflow.

Another possible problem is that the mvhd atom has a fixed-length structure that is always supposed to be 108 (0x6C) bytes long:

So you might slip up, and write the above code inflexibly, coding straight from the specification:

Faced with a well-formed file, your code will work just fine, because size will be 108 (as it is in the sample file above) and everything will line up.

If someone "crafts" a file with a size of -4096, however, you'll be doing this:

When you come to manipulate the buffer for the next atom, you will be operating on memory in front of your own memory block, and things will almost certainly go wrong.

That's just two examples of potential coding bugs in one multimedia file type - and so far, we haven't even tried to do anything with the data in the file.

Check, check, and check again

In real life, we'll have loads more to do, such as extracting and using the movie header and track data, as well as actually decompressing and displaying the video images.

So be careful out there if you're a programmer, and remember that you can never have too much error-checking:

And don't wait for cybercrooks to come up with movie files containing negative MVHD sizes.

Stay ahead of your attackers with a robust and well-thought out set of test files.