Posts tagged Digital photography

NEF File reader is on hold ...

I have not had very much time to blog nor work on my NEF file reader project for a while. It has been turbulent and busy times at work, lately.

This post is just to let you know, that the project is curently on hold. Perhaps I will resume it later, when I have more free time at hand. If anyone is interested in a peek at the source code as it looks now (incomplete and raw), please let me know; and I can mail it to you. There is not too much functionality, however. The largest obstacle in getting further with the project, is the need to decode the Huffman-encoded sample bit lengths embedded in the file - I have not resolved this yet.

NEF file reader update

Progress on the NEF / RAW file reader is slow, since i only work on it occasionally when I feel like coding in my spare time. However, I made some progress tonight, that I would like to update you about.

First, NEF files are really TIFF files. So I need to parse them as a TIFF file to get some useful information out of them. TIFF files consists of so-called IFD tags, which in parts contains metadata about the image as well as the actual image data. Currently I have developed a basic IFD tag parser, that parses all of the IFD tags in a NEF file.

Furthermore I have found, by looking at the IFD tags, that there are at least 3 embedded images in each NEF files. These are a small, low-quality 160x120 thumbnail represented as RGB data, a nearly full size JPG in low quality, as well as the actual NEF image data. While implementing the IFD parser, I had a lot of help from this TIFF FAQ, as well as from the official Adobe TIFF specification version 6.

Of course, the NEF image data is the interesting part. I have studied the file format, and made these conclusions; mostly based on what I can tell from the dcraw source code:

  • The NEF file consists of Width * Height samples.
  • The data is represented as a Color Filter Array, that is each sample represents the intensity of a single color (since this is what the camera shoots, as many other digital cameras);
  • therefore, I must interpolate 2 of the colors for each sample, to get a full-color, full-scale image.
  • Each sample is 12 bits.

But this is not all; it is not so simple as to just read 12 bits per sample. The data is compressed. As far as I can tell currently; the following is the compression scheme.

  • A ยต-law type compression curve is used to limit the number of possible sample values to 634 (and consequently companding the 12 bit values to log2(634) = 9.4 bits of actual accuray; while preserving the 12 bits dynamic range).
  • The curve values are embedded in the file.
  • Samples are indices to values in the curve.

Samples are encoded (compressed) like this:

  • A simple predictor is used, so each value read is actually the difference between a previous sample value and the current. This keeps values stored in the file low and keeps differences between each value needed to be stored, low.
  • Each value is  stored as a run-length encoded length (in bits) of the value, and immediately thereafter, the actual value.
  • Run length encoding is accomplished by using a classic Huffman table.

So, now I need to implement the reading of the actual sample values and interpolate the values to come up with a full color image. After that, I probably need to implement the reading or approximation of white balance of the image, so that the program will be able to produce actually usable images.

Hobby project - Nikon NEF file reader

So, my new hobby is digital photography. I have bought a slightly used Nikon D50 camera, and has already taken a lot of pictures.

If you're into digital photography, you will know that most cameras can produce both JPG and RAW output, where the JPG images are compressed by the camera firmware before being written to the flash card. The RAW files are uncompressed data as it was sampled by the camera CMOS (typically).  Remembe, JPG is a lossy compression format, so detail will be lost if you just save the JPG's. Therefore, if you are serious about photography, you will want to shoot in RAW.

However, as RAW files are essentially untouched image data, a RAW file needs to be processed by some image processing software before it can be used. This can be a cumbersome process, loading up the camera manufacturers software and maybe Photoshop or another image processing utility, processing each image, perhaps adjusting colors and saving the file into a more common format.

Therefore I have decided to try and create a .NET program that does the conversion for me. For "version 1", it will probably only support the NEF file format that my Nikon produces (Nikon's proprietary RAW format), but perhaps I will add support for others along the way. This program will not be a replacement for professional photo processing software, but is meant to be an easy to use tool, that can be used when one simply needs to convert a NEF file to a JPEG or bitmap representation quickly.

The NEF file format is not documented anywhere that I have been able to find on the web. But, looking at the files, they seem to be based on the TIFF image format, and they carry the TIFF file header.

Also, I have found that Dave Coffin has created an excellent utility for doing just what I want, dcraw. I could just as well just use that, but I think that it will be an interesting challenge to try to implement it myself. But I am quite sure that digging in to the dcraw source code will provide me with most of the details that I need to decode the format.