Are you ready for the Dark?

Earth Hour is tomorrow. What started as an Australian initiative in 2007 has now gone global, and is gathering a lot of support as a symbolic action. In short, it's about turning off all your lights for 60 minutes at 8:30 PM in your local timezone. Check it out to see how much energy consumption such collective efforts actually shave off. It might surprise you.

Here in Norway, the action is endorsed by the government and organised by WWF. Pentax is one of the sponsors, and lo and behold! There's even a sponsored photo competition with prices by Pentax. Nice touch. :-)

Personally, I'd like to encourage all Pentaxians to make the most out of this occasion. Try shooting the night sky. Try out long exposures. Be part of it!

Mirela the blog crawler

I recently got a nice little comment from "Sarah", on my post about laptop screen quality. The exact text of the comment was:
"I recently came across your blog and have been reading along. I thought I would leave my first comment. I don't know what to say except that I have enjoyed reading. Nice blog. I will keep visiting this blog very often"
The comment concluded with a link to a blog testing laptops.
Today I noticed that Ned Bunnell had got a comment with exactly the same wording from one "Ann" on one of his posts. "Ann", however, linked to a blog for large pets.
Since both "Ann" and "Sarah" are linked to the same blogger profile "Mirela", I got curious. A quick Google search on the comment phrase brought up more than 100 exact matches. All of which were in blog contexts. Now how likely is that, given the lenght of the phrase? I sampled some of them, and found them all to be signed by random female names and pointing to various blogs.
So clearly Mirela must be some sort of robot. She is probably set up to drive traffic to the blogs she advertise. For the sake of the authors of those blogs, I really hope they're not paying for that kind of service.
I have removed her comment from my blog, but left my naïve answer in. Little did I know then...:-)
So as of now, I have turned on comment moderation. As they say, I'm sorry about the inconvenience. But I don't want to play host for parasite promotions.

Digital archiving - about hierarchical keywords

Hierarchical keywords are almost magic.

If used correctly, it applies many levels of organisation to an image in One Go. I'll get back to that particular "if", but first a couple of illustrations of hierarchies:

My examples are from Photoshop Elements 6. The first four categories in fig. 1 are predefined by Adobe. You can rename them and otherwise do exactly what you want with them, but I think those are nice categories for general purposes.



However, as a nature photographer I would like more categories. So I added one called "Animals" myself. Below you can see how this can be expanded to a hierarchial structure, branching out in many places (the "imported Keyword Tags" can be ignored for now). Here's how I have defined my Animals category, with subcategories and all:


Now, say I have a pic of a House Sparrow. When adding "house sparrow" as a keyword to the image, I would also like to add the whole hierarchy that words belongs to. Thus, by adding one keyword, I want the program to understand that I add both Birds, passerines and house sparrow. Also, if I later find out that I want to split the passerines into, say, "chirpers", "squeakers" and "thrushes", then my keywords for the affected images should be updated automatically.


Further, I would like the program to write those keywords into the files in such a way that other archive programs will understand it. In case I want to switch systems again, another five years down the line.

Lastly, there's a complicating factor called language. English is not my primary language, and many keywords will have synonyms in Norwegian. I want the software to keep track of that as well, so I don't have to double every hierarchy of keywords that I build.

Digital archiving - about metadata

The more I entange myself in thoughts about keywording, the more I realise it's hard to keep all this metadata stuff apart. Different standards, different types of metadata, duplication of data between standards... - And that's just for starters.
So, in an attempt to regain a sort of bird's perspective on what this is all about, I wanted to write this down. If it's useful to you as a reader too, I'm glad.

First, what's metadata anyway.
In my mind, it's as simple as "information about an image". With this definition, everything in a image data file that is not the image itself is metadata. I have included three graphics to illustrate how I imagine metadata is stored in various file formats at the bottom of this post. You may want to have a peek at those while you read, but I thought it best to put them all at the bottom.

Most image file formats have placeholders for metadata. Little compartments where all kinds of information can be stored. There is actually a scary part about that; it is perfectly possible to embed malicious program code inside these compartments. Sony's PSP game console, for example, has been attacked by such code embedded inside TIFF files. But I digress. It just goes to show that practically anything can be embedded inside image files. The upside is that there is room for plenty of metadata. If only everyone could agree on how and what to put in there.

That's where the standards comes in, of course. As far as I know there are three different standards to pay heed to; EXIF, IPTC and XMP. So I'll try to make heads and tails of these.

Every bit of info contained in these standards, by the way, can be referred to as a "tag".

EXIF specifies all kinds of technical info, like shutter speed and aperture for example. EXIF can be supplied by the camera automatically at the time of shooting. The tag set is the standard.

IPTC is a way of supplying various sorts of additional info that the photographer can provide. Such as location, subject, copyright, title of image, keywords, free-text description etc., etc. As for EXIF, the tag set is the standard. The list of possible tags is longer than any photographer cares to know about, just have a look at the below screendump from ThumbsPlus v7.

Figure 1. The IPTC registration panel in ThumbsPlus
Click image to enlarge.

I find IPTC quite confusing, because there is no intuitive way of knowing exactly how each of the fields are supposed to be used. Take the tags "object name", "title", "headline" and "caption", for example. For all practical purposes, one or two of these will do. But which ones? The same kind of overlap exists between "caption", "keywords" and "supplemental categories". When I'm through this study of archiving software I might have a clue, but how much time should a lazy photographer be required to spend, just to embark on the tedious job of keywording images? Not this much, surely?
It makes me think that I will be partial to any software that takes the thinking out of this process as much as possible.

XMP has a great advantage over the other two, in that it doesn't have to be embedded into the image file itself. XMP can also be stored in a sidecar-file. I have slapped my forehead several times already (Homer Simpson style) for not realising until now the huge bonus this brings: It opens for keywording raw files. For file formats that support embedding, however, XMP will just occupy another one of those compartments I mentioned earlier.
XMP will gladly duplicate anything you can find in EXIF and IPTC, but it can also add a lot more. The X in XMP stands for eXtensible, and in principle it's so open that any software developer can add their own (and proprietary) tags. So it's not really the tag set that constitutes the standard, but the method used to add tags and information into the compartment. That's both its beauty and its curse, it seems. Most of the tags used in XMP are described by Adobe and adhered to by all others. What's funny, though, is that Adobe doesn't quite adhere to it themselves. That is, LightRoom has expanded the tag set somewhat for its own purposes, and the extra tags are not correctly implemented by Adobe Bridge (the file browser in Photoshop). :-)
In general, it seems that the software industry has been quite reluctant to adopt XMP, trying instead to keep IPTC as the main way of including metadata. There are some very interesting differences between my shortlisted programs in how they deal with IPTC and XMP which I will address when considering the individual products.


My next post, however, will elaborate on the virtues of hierarchical keywords, which was part of my wishlist in the second post in this series. I will explain in a bit more detail what I am thinking about, and how it can be fitted into XMP and IPTC.

Figure 2. A raw file will contain EXIF-data supplied by the camera, plus whatever proprietary information the camera vendor puts in. Nikon, for example, is infamous for encrypting the vendor specific info in this part of their raw (*.NEF) files. IMO, it can be argued that the embedded thumbnail is actually a sort of metadata, since it is just a representation of the real thing. I've never seen it referred to that way, though, just my personal opinion.
XMP-data may be added to the file as a sidecar, while IPTC can not be added.

Figure 3. TIFF and DNG files will embed just about anything. There may be proprietary camera information in here too, I think, but I believe that is supposed to go into the XMP compartment. If I'm wrong, it just means that there could be another blue box on top. :-)
The DNG is based on the TIFF, by the way, and both formats are controlled by Adobe...

Figure 4. A JPEG file will embed just about anything too, except for a thumbnail as far as I know.

Digital Archiving - part 3 - the shortlist

I have looked at a lot of programs. From the lot, I have selected a handful I will study more closely. There will be future posts considering my findings for each of these:

- ThumbsPlus
- Adobe Photoshop Elements
- Adobe LightRoom
- PhotoMechanic
- IDimager

The links takes you to the product homepages.

There are many other packages out there. I may have missed a few good ones too, like Microsoft's Expression Media (MEM). It used to be iView before Microsoft bought the company. From looking at the specs, reading whitepapers and reading the support forum, I must say the program looks very promising. However, it has a showstopper for me. MEM relies on a component in Windows called WIC (Windows Imaging Component) for rendering raw files. Microsoft has made calls to the camera industry to provide codecs that enable WIC to render their files, and most manufacturers have made an effort. But only for 32-bit versions of Windows, at the time of writing. So since I run 64-bit versions on my computers, I have not been able to test MEM.
If you use 32-bit Windows and plan to continue doing so for a while, I think MEM will be worth considering.

As I write this, I have only studied two of the shortlisted programs. So this series of blogposts may be drawn out in time. However, I need to make a decision for my own sake before summer. So that's my final "event horizon". I just hope I will have time and occasion to take some nice photos too, before then. Otherwise all this testing is bound to drive me nuts... :-)

Workflow vs. archiving

This post is a step to the side in my process of looking at image archiving software. But I think it is a quite important one.
The question I want to address is how workflow can impact archiving.
My own workflow is rather fuzzy, but there are a few regular milestones.

1. I take pictures. (wow, eh?)
2. Load onto computer
3. Cull the captures (conservatively)
4. Rename files
5. Post-process

I have been through all randomisations of points 3-5, and believe me, it gets uncomfortable when this order is distorted...
I would like to argue that archiving is implicit in points 2-4, but that the process is not complete. In my opinion, an archive is not complete until backup and keywording is done.
Backup should be done regularly, of course, but maybe an extra backup should take place just after an upload from the camera?
Keywording is also best done when memory of the shoot is fresh. I would like it done on the master copy of the file (ie. the raw file), so it can be propagated to post-process created versions automatically.

Keeping post-processing at the end, I end up with this workflow:

1. I take pictures.
2. Load onto computer
3. Cull the captures
4. Rename files
5. Put keywords on 'em
6. Unscheduled extra-backup
7. Post-process

For many photographers it is probably not acceptable to postpone the post-processing to after the culling, keywording and backup is done. Fashion, news and sports, for example, often have tight deadlines and need to fix up some captures fast to get them off to the customer or image desk.

The workflow for them may be more like:

1. Take pictures.
2. Load onto computer
3. Rename files
4. Post-process and send off to customer
5. Cull the captures
6. Put keywords on 'em
7. Unscheduled extra-backup

This is something to keep in mind when considering photo archiving software. In my scenario, I emphasize archiving functionality over image throughput, while other photographers may do the opposite.
I find that this difference is reflected in the software. Some are dedicated archiving tools, some are archiving tools with raw file conversion functionality, and some are raw file converters with archiving functionality.

Digital archiving - part 2

Part 1 in this little series of posts is here.
Digital Asset Management has emerged as a separate field of expertise. One quite authoritative source for describing the issues seems to be The DAM book by Peter Krogh. I have not read the book myself, but from looking at the index on the website I think it's likely that I will. Soon.
Meanwhile, I guess I must risk emphasising the trivial and ignoring the salient, by making my own list of things I find essential for a good system.

1. Magic words
I know the last time I made a real effort of putting keywords on my images was in August 2007; eighteen months ago. I hate it, and even posted about it here. So I need a system that makes this job as simple and rational as possible.
In particular, I would be very partial to a system that allows setting whole sets of keywords at once. If I, for example, add the keyword "Eagle" to 200 shots, it should automatically add some associated keywords like "bird" and "predator"; provided that I have made "Eagle" a predefined sub-category of both.

2. Version handling and bundling
I shoot RAW, publish JPG on the web, and print from TIFF. Then maybe I want JPGs in different resolutions for other purposes. All of which are versions of the same image. I want my archive to group and deal with all versions of one image together, and to display only ONE thumbnail for the whole heap. Likewise, I would like it to bundle related images together from eg. rapidly recorded sequences as in sports, or tiles of a panorama. It should also have a clever way of letting new versions or whole bundles inherit metadata (eg. keywords) from one chosen master-file (eg. the assembled pano, the best of a burst series, the raw file when developed, etc.)
3. Backup, restore, and migration
The importance of having a system that will survive such major changes in hardware or software cannot be underestimated, in my opinion.
It should be easy to create a backup, and it should be easy to restore from that backup.
It should also be realtively simple to migrate to another archiving system without losing the metadata.

4. Support for offline archives
"Offline archive" means different things in different contexts. I am thinking of two different meanings:
4a. Disconnected storage media
The software should retain a thumbnail and the metadata for images that are stored on disconnected media such as DVDs, USB-drives, NAS-units, and so on.
4b. "Branch" archive on a laptop
The software should be able to upload parts of the archive to a laptop. For example a prepared slide show. It should also be possible to import from the branch archive back into the main archive in a simple way. If modifications of existing files are discovered, the version handling (see pt. 1) should deal with it.

5. I could go on
to add a lot of other things too. However I feel that maintaining metadata, handling versions, and versatile backup are the most important functions of an archive. Other things, such as intuitiveness of the interface and overall convenience will emerge in overall evaluation of a product anyway.

In the next post I will short-list some candidates that seem to have a lot of potential.

Digital archiving - part 1

Back in the Film Era it was so easy to manage the digital images. There was no point in scanning anything but the real keepers, and so my archive didn't grow by more than say, 100-150 images per year. After all, the gelatine strips was the master archive anyway.
Not so since going digital. Now the PC harddrive is the master archive.
Archiving large numbers of images has become an issue for more than just dedicated geeks. It has become mainstream, to the degree that it has got it's own Three Letter Abbreviation; DAM. Digital Asset Management. A google search on that term will point you to both consultancies and software companies. Systems and services have evolved to cover every little sub-niche of the market, ranging from multi-national stock agencies, to freelance photographers, and to Mom with her compact camera.
I'm in the process of changing my archive software right now, and have spent a good deal of time with the search engines. Initially I thought it would be easy to find the best offerings now, with DAM becoming mainstream.
Boy, was I wrong. :-)
After finding a handful of programs, I have spent the better part of a month playing around with trial versions, reading manuals and skimming through online support forums. It isn't always easy to find out what functionality the variuos programs offer. Surprisingly few have bothered to test this kind of software, and many of the offerings are not tested at all.
So after gathering all this information for my own benefit, I thought maybe sharing it could save someone else a bit of time. Over a couple of posts, I will share with you my thoughts and my findings here. The next part will be about what I want from the software, and will be posted shortly.