Quick Intro
User Guide
System Design
File Formats
Object Layout
Debugging Guide
Timeline View
Frame Server

Why new file formats?

When you use the Cuisine system for more than just light file conversions, the core thing you're buying is the file formats. Cuisine also contains editors, analyzers, renderers, etc for its formats, but as those come and go, the formats should live on.

I'm very sad about introducing any new file format. I researched OMF, MXF, AAF, and other EDL and media description formats, but (among other complications) none of them are human-readable, which is high on my requirements list. Also, I needed to store all sorts of rich, quirky data (such as annotations and historical versions). Although some of the formats accepted blocks of "application-specific data", I'd have to layout all my complex data within that section, which is equivalent to another file format anyway.

We all have great ideas about how a file format should be built, but this isn't the time for new data format ideas. I set out to choose my structure from all the widely-implemented, human-readable, popular-enough-to-last-awhile formats that could hold a tree structure, which means it's XML.

The other part of my file format plan is that there should be very few limits and requirements on what can go into my data files. It sounds responsible to say that a clip must have a duration that's greater than zero, or that every video must have a width and height, but really: it doesn't matter to the file. The file doesn't care if it's incredibly degenerate, and we can't guarantee that the next tool used on the file will need any particular piece of data. So any guides or schemas for the data files will be purely to help direct data to the right places and format it in the standard way, never to require any data.

The clip description format


On disk are many files that contain: sequences of DV frames, AVI format clips of DV frames, audio files, single images, image sequences stored as directories of single images, etc. I refer to these as media files, each of which contains one clip, with a variable number of components (audio, video, left-eye stereo, etc).

The goals of the clip description format are to normalize all the intrinsic info about the data (duration, rate, dimensions) and to provide a container for all data that users and tools might ever produce regarding the data.

I'm sick of programs forgetting that a video image is upper-field-first interlaced, or that one clip has square pixels and another doesn't. Or if I know that a clip has a lead-in that's ugly, but might be needed in a dissolve cut or time blur. Or if I watched a clip and found 4 frames with dust on them. Or that the files with an pt prefix have been painted on, but not yet color corrected. Every project I've ever done has consumed at least one page of notepaper where I had to jot down some frames or ranges or filenames, and there have always been plenty of things that I had to keep in my memory throughout a project that should really be encoded in the files.

Separate files

I didn't really mean "in the media files", though. Some of the industry metadata systems like to keep the "essence" (the actual data) in the same file as the metadata, but for flexibility reasons, I want separate files. These files probably have the same name as their essence files plus ".desc" at the end (after any existing extensions). Advantages of keeping the metadata in its own XML files:

  • In most cases, the metadata is a lot more precious than the essence data, since the essence data can be recaptured (or re-converted) from its sources. Different "preciousness" means different backup strategies, which are hard to implement when the tiny, important data is stuck in the same file as the giant, reproducible data. Also, drive speeds and read patterns favor optimizing the storage of fixed-size frame data one way and the storage of small, variable-length metadata another way.
  • You might opt to not keep certain essence data online at all, at which point the metadata becomes the archive index, telling you all the details about the essence data that's offline. The asset manager could even specify the steps needed to bring the data online right in the metadata file if it wanted.
  • It's even easier for general XML tools and general unix text file tools to work with the metadata. "Show me all files that have a creator tag of ronald, since he works at the wrong gamma some of the time so we need to check all his output" or "Give me a quick count of how many frames we're storing per project, by finding all the duration tags and adding their times together".
  • They're easy to edit in a text editor (or an XML editor, when those become common), so if you need to do a weird tweak that no tool will do for you, you've got a way.

The big disadvantage of separate files is that it's harder to keep the desc files with their video files. Fortunately, file systems are very visible to the user, mistakes should be pretty easy to correct, and it will be possible to add some "go find my lost data file" features to the tools (which edit suites have already been doing for a long time with their EDL files that reference clip files).

The exception I have in mind for "one desc per media file" is the case of a clip stored as many image files. A standalone image should have a desc file like normal, but a directory of sequentially-numbered images should probably be able to get by with one desc file (for the same reason an AVI file with a bunch of frames-- and audio-- has only a single desc file).

I imagine eventually having a real database that can quickly do certain queries over all the desc files, but it'll always be a secondary source to the easy-to-handle text files. So if you've got tens of thousands of desc files and you're querying them all day long, talk to me about ways of speeding that up.

Data layout

How do you find a piece of data such as the duration of a clip? And more importantly, where should you add a new piece? I deliberately didn't start some committees and figure out all 300 bits of data that everyone will need to store to be compliant. Instead, I (and other authors)

The EDL format

The show files contain a different style of information than most EDL formats, and like my clip format, they can contain a lot more information that any of the fixed-field specs.

Future work

Show files may become part of clip files. In favor:

  • shows have a lot in common with clips; you should be able to insert them in other shows, play them directly in the clip player, convert them to other formats


  • clip .desc files have the property that there's one per movie file (or set of frames). A show may be exported to a new movie file, in which case a .desc will be written. But the show itself isn't a clip, so it doesn't deserve a .desc file.