FAQ
Gaming Click!??!?
Yikes! This week I wanted to take a moment and look at some rather amusing things (or scary things, depending on your perspective) that happened during the evaluation process. We stayed mum on this during the eval period to avoid bringing more attention to it, but it’s worth looking at now while revisiting chapter three of The Wisdom of Crowds.
In chapter three, Surowiecki discusses information cascades, the effect of one person making a decision and others following suit. He writes, “The fundamental problem with an information cascade is that after a certain point it becomes rational for people to stop paying attention to their own knowledge—their private information—and to start looking instead at the actions of others and imitate them.” A bit later in the chapter he concludes with, “Collective decisions are most likely to be good ones when they’re made by people with diverse opinions reaching independent conclusions, relying primarily on their private information.” Avoiding a cascade was the primary reason for suppressing comments and direct links in the evaluation tool, but, of course, we couldn’t think of every situation that might present itself. It’s important to keep in mind that there were very few instances of this, but the fact I didn’t see this coming makes me wonder about my optimistic state of mind…
The evaluation was being conducted on the web and we were specifically asking for people to help us spread the word. Some of the posts, like this one, were awesome - here’s an artist who’s spreading the word using our widget which minimizes influence by randomizing images. But, then we noticed posts like this and this popping up (even more here) and, well, I felt a little daft for not instituting a specific rule about it. But, hey, a big part of this project is on the ‘net and who wants to post to their blog without a little self-promo? So, I get it. The reality is these posts are fascinating to read and provide a great opportunity to see the process from the point of view of some of the artists - yay! And it’s really awesome to come across a post like this, where the artist says, “i can’t/won’t tell which image i selected, as i believe it’s against the rules.”—well, it still makes me smile.
All this self-promotion begs the question…did it do any good? We don’t think so. Evaluation was not a quick and easy process and you can see how this type of implementation starts to help when issues like this crop up. We carefully looked at the data to see how much of a difference this kind of self-promotion may have made. Luckily for us, we had enough participation (data) to counteract any skewing and the results were fairly balanced (thank you, again). Of course, this could have easily changed had any one instances above (or similar) generated a lot of traffic with very determined users.
Have fun exploring some of these posts, it will be even more interesting to reference them when you see the results on June 27. Next week we’ve invited a special guest blogger to post about the subject matter in the submitted works…more soon.
RSS 
Eastern Parkway/Brooklyn Museum
June 4th, 2008 at 11:41 am
this really is an interesting experiment and so much work has gone into this.i am however,confused as to whether all 389 photographs will be in the exhibition or just a selection(according to rank) i’m sure all 389 artists would like to know whether or not they will be exhibited.i understand you have a ton of work and data to go through but,seeing that the exhibition is opening on the 27th of this month-i’m sure the artists would love to know if they are actually in it! again,this was a fantastic opportunity for many artists-thank you shelley-and everyone at the museum for making this possible!!!!!!
June 4th, 2008 at 12:49 pm
Thanks for the shout out on my plog - now I can finally go out with a link to my submission and do a bit of shameless self-promotion
//p
June 4th, 2008 at 3:21 pm
Hi Robert, we actually won’t be releasing anything specific until June 27 or possibly right before it. I realize it’s tough to wait it out, but this is for good reason - the exhibition is happening at a fairly quick pace and we are still in the planning phases with the gallery design - we need to maintain as much flexibility as possible. That said, we hope to blog about the design in detail as we get closer to opening and give you updates here as we can. I can tell you not all will be in the (small) gallery and it will be a selection by rank, but which ones and exactly how many will have to wait.
July 4th, 2008 at 1:32 pm
This is interesting only for the way it is examining or investigating what is happening to the notion of photography in the age of the internet. ALL photography sites are under the spell of the “lemming look” with little new or interesting work rising to the top, as it were.
What’s more interesting to me, is the artist’s statements. Almost every one seems canned and plagiarized from some other source available on the internet. More evidence of the lemming attitude at work and sad implications on what is down the road for photography.
Finally, one of the consultants on this project is the publisher of jpg magazine, which is perhaps the ultimate resource for the dumbing down of photography (only exceeded by Flickr). Oh, and isn’t it interesting that jpg mag is a sorta subsidiary of Flickr???
Talk about incestuous!
July 4th, 2008 at 2:45 pm
Mary,
I’m inclined to agree with you to a degree about Flickr & JPG mag but they are what they are - primarily social networking sites for photographers to share their images out into the “internet” world not unlike vanity galleries - just cheaper. If you look closely however you will see some inspired and compelling images. I’m willing to bet that if the technology had been around back in the day we would have seen photographers like Atget, Abbott and countless other greats using these sites to self promote their work.
July 7th, 2008 at 3:15 pm
I’m probably the reverse of the self-promotion that has been described here. I submitted an image and told no one about it. It was an OK image and it seemed to suit the theme of “changing faces.” Somehow it just made the cut and is part of the show. Reading the articles in the NYTimes and the Washington Post I think they unjustifiably disparage the work of the photographers who submitted images. I don’t know that the work of the great photographers mentioned included images that fit the theme, at least in my view. But I have no interest in slighting other photographers in order to counter critics. Ironically I have what I think is an interesting image of a group of children running on the sidewalk next to the Cyclone. Each child is frozen in an almost surreal position. But like some Diane Arbus images, it is anything but flattering to the subjects. So it might have appealed to the newspaper critics but it didn’t appeal to me as something to submit to the museum for view by the general public with a likely secondary interest in photography. Anyway, I’m flattering myself in comparing a somewhat strange image of mine to the art of Diane Arbus. Just trying to make a point.
I was surprised to see so many images from Coney Island. I had found out about “Click!” through a posting at a Fickr group. I had seen a posting about a prior Brooklyn Museum request for submissions of images on Coney Island, which I had missed, and so with the “changing faces of Brooklyn” request for submissions I didn’t expect to see Coney Island as a regular subject, though obviously it’s a part of Brooklyn.
Regarding the evaluation process I’ll describe my own experience. I didn’t evaluate any images. Not that I didn’t want to but I found myself to be unsuited to the process. Ideally I would have preferred to have been able to view icons or small sized versions of the images and select the ones that appealed to me for closer, full sized, evaluation. I don’t remember the exact constraints of the process at this point, but that overview which I would have preferred was not available. Reading the commentary I can now understand the wisdom of that, in that it would have allowed some people to flood the evaluations with good or bad judgments. I think that still could have been possible if someone or group was very much inclined to do so. But at least it would have required some patience and effort. So the aspect of random image choice makes sense.
As I’ve mentioned I don’t remember the specifics but I think it was necessary to make an evaluation on each image viewed. The images I did see didn’t appeal to me but I’m reluctant to pass a negative judgment, preferring to abstain and instead praise images that I find interesting. I’m not sure if I had to log off and log back in or exit the process and re-enter, but avoiding negative evaluations wasn’t an easy process. And so I gave up after looking at about five images that I thought were flawed, getting frustrated with the process. Maybe I’m wrong about the process or I missed something but the impression I got was that looking at more than a hundred images would have taken a good part of a day if not longer. I didn’t want to do that, especially if I had to pass negative evaluations on most of the images I did see.
Another problem is that unless every image is viewed beforehand, without evaluation, it’s not possible to get an idea of the range of quality that is available. If only twenty or thirty images are viewed, the best of that group might not be even close to the actual “best.” Any superlatives used may not be worthy, at least in comparison to images not seen. Maybe that was part of my reluctance to pass judgment on the images I did see. Maybe the ones I didn’t care for were among the best. Maybe they were among the “worst.” I had no way to know other than in comparison to may own ability as a photographer and that was obviously a biased view.