Progress on the NEF / RAW file reader is slow, since i only work on it occasionally when I feel like coding in my spare time. However, I made some progress tonight, that I would like to update you about.
First, NEF files are really TIFF files. So I need to parse them as a TIFF file to get some useful information out of them. TIFF files consists of so-called IFD tags, which in parts contains metadata about the image as well as the actual image data. Currently I have developed a basic IFD tag parser, that parses all of the IFD tags in a NEF file.
Furthermore I have found, by looking at the IFD tags, that there are at least 3 embedded images in each NEF files. These are a small, low-quality 160x120 thumbnail represented as RGB data, a nearly full size JPG in low quality, as well as the actual NEF image data. While implementing the IFD parser, I had a lot of help from this TIFF FAQ, as well as from the official Adobe TIFF specification version 6.
Of course, the NEF image data is the interesting part. I have studied the file format, and made these conclusions; mostly based on what I can tell from the dcraw source code:
- The NEF file consists of Width * Height samples.
- The data is represented as a Color Filter Array, that is each sample represents the intensity of a single color (since this is what the camera shoots, as many other digital cameras);
- therefore, I must interpolate 2 of the colors for each sample, to get a full-color, full-scale image.
- Each sample is 12 bits.
But this is not all; it is not so simple as to just read 12 bits per sample. The data is compressed. As far as I can tell currently; the following is the compression scheme.
- A µ-law type compression curve is used to limit the number of possible sample values to 634 (and consequently companding the 12 bit values to log2(634) = 9.4 bits of actual accuray; while preserving the 12 bits dynamic range).
- The curve values are embedded in the file.
- Samples are indices to values in the curve.
Samples are encoded (compressed) like this:
- A simple predictor is used, so each value read is actually the difference between a previous sample value and the current. This keeps values stored in the file low and keeps differences between each value needed to be stored, low.
- Each value is stored as a run-length encoded length (in bits) of the value, and immediately thereafter, the actual value.
- Run length encoding is accomplished by using a classic Huffman table.
So, now I need to implement the reading of the actual sample values and interpolate the values to come up with a full color image. After that, I probably need to implement the reading or approximation of white balance of the image, so that the program will be able to produce actually usable images.