I've just been playing with another experimental image indexing tool, that's using image analysis to suggest keywords for images, called ALIPR: Automatic Linguistic Indexing of Pictures. You feed it an image file, or a URL to a web accessible image, and it suggests keywords that might apply to that image. You're then prompted to correct the suggestions, and add alteratives, so the program "learns" through user feedback and prompting.

For fun, i fed it the 'most tagged' steve image, Winslow Homer's The Gulf Stream, 1899 using the URL for the largest image from the MMA web site.

Here's what it suggested: ocean, ice, landscape, water, wind, waterfall, mountain, snow, frost, wild_life, plant, bonsai, wave, people, sport.

Only 1/3 of the 15 suggested terms are accurate. The five that are, can also be found in the 117 unique (340 total) tags suggested for this work in the steve prototype.

Matches in the tag set are: ocean (assigned 24 times), water (8 times), wind (1 time), wave (14 times, but in the plural as "waves"). The top 4 tags boat (52 times), storm (41 times), sharks (26 times) and sea (26 times) weren't suggested by the ALIPR tool.

i'll confess, i didn't submit the image... so i'll be able to test the 'learning' later by asking again.

i want to come back to this question, and definitely study this further.

/jt

[The steve prototype data analysis is reported as Social Classification and Folksonomy in Art Museums: early data from the steve.museum tagger prototype. a paper for the ASIST SIG-CR workshop on Social Classification, November 4, 2006. [pdf preprint] [pdf of presentation slides]].