With the opening of Click! rapidly approaching, I have been asked to describe my approach in designing and mounting this particular exhibition. It is important to note that although this exhibition is comprised entirely of photographs, it is not foremost a photography show, but rather an art installation addressing the conceptual nature of a crowd-curated exhibition. For this reason, the show will not be hung in a traditional manner, but rather laid out in a way that illustrates the diverse, anonymous, web-based crowd selection process.
The exhibition is being held in an intimate gallery space to allow the viewer to be immersed in the images of the “changing faces of Brooklyn.” Because of the gallery’s space constraints it was determined that of the 389 images submitted, the top-ranked 20% of images would be printed for display. Of this 20% (or roughly 78 photographs) a distinction was required to reflect the crowds selection process; for this reason some photographs were printed as large as 20” x 30” and some as small as 5” x 7”, depending upon the crowd’s rankings.
Rough layout of one of the gallery walls. Full layout: click_full.jpg.
When these images of varying size are displayed in a random arrangement it serves to illustrate the crowds’ selection process not as linear, but rather a diverse response with certain ideas or, in this case, photographs rising to the front of a collective conscious in much the way a tag cloud uses text to visually illustrate how within many voices certain responses carry varying degrees of impact.
Cloud tag from Flickr.com
The exhibition promises to be interesting on many levels, and I hope all that participated both in submitting photographs and those involved in the selection process can make it to see the results.
Author RSS 
RSS 
Eastern Parkway/Brooklyn Museum
Question -
Will the exhibition include some information on the selection process, and the gobs of informaiton gathered? Will there be references to the book or will there be an online component that will go into more depth about the information gathered?
Amy Dreher
Hi Amy,
We are planning on two computer kiosks that will display the web component for Click, including the blog posts. Lance has designed a really awesome seating area, where you’ll be able to browse the website using laptops and we’ll have a few copies of the book around too, so people can reference.
I’ll be writing about the online component as we get a little closer to opening.
Shelley
Thanks Shelley and Lance – sounds like a very interesting installation and I can’t wait to see it!!
I love the relationship between the physical scale of the images and the crowd selections. I hope we will see some installation photos on the blog!
Hey Despi! Even better – we’ve got one of Jook Leung’s 360 virtual tours coming as early next Friday. We wanted to do that for all the evaluators we know can’t make it
’cause we know how much time everybody spent on this!
Lance, I really like this idea. One of those things that seems just right and completely obvious, once someone else thinks of it first!
I think it would be nice to know in advance, if and who’s images were selected, either via personal notification or general public announcement, which ever is found the most appropriate. I’ve experienced this in the past, inviting family and friends to see my contribution in a crowd type exhibit similar to Click, (HereisNewYork.org) only to learn, with embarrassment, and disappointment that my piece wasn’t included amongst horde of images displayed at the announced exhibition. Therefore, the question is: Who is in the 20%? And who is not? In the why one call appropriately gauge the amount of excitement, big (30×20), Small (5×7), or appreciative that should transfer to their family and friends. I think, nobody, seeks to have their expectations dashed, due to a disappointment, equivalent to the prize found in a Crakerjack box.
Hi Mark,
I’ve mentioned this before on many posts in the blog. We won’t be giving this information until the show opens. I’m sorry, but we are working with an incredibly short time-frame for a museum and this is the situation we are all in. There is a slight chance we will release information early and, if we do, we will be posting about that here. As we’ve mentioned many times over, please watch the blog for updates. That’s the best we can do.
I find it a little strange that people’s photographs will be printed without their input. Was this mentioned when everyone submitted their photographs (i could have easily missed it)? I was under the impression that if one’s photo was chosen, the museum would ask for a print. By printing it themselves, it seems the museum is endorsing the view that the printing of photographs is not part of the artistic process of photography, a view that is becoming increasingly more prevelant in the digital age.
Hi Noah,
Thanks for the comment. Lance can probably speak better to this, but I’m going to take a shot at it. We did think about this aspect, but it’s important to go back to what Lance mentions here – “It is important to note that although this exhibition is comprised entirely of photographs, it is not foremost a photography show, but rather an art installation addressing the conceptual nature of a crowd-curated exhibition.” – the exhibition design reflects that. It was important to us that the look and feel of the show come across with equal quality in printing – paper and inks all matching, etc. If we had asked for prints directly from photographers, variance in paper or type of print would have easily made this show more about what quality each photographer could have given us, which would send the wrong message totally.
You bring up an important point here, so let’s remember that people evaluated images on screen via the web, not via prints. What looked good on screen may change when we get in the gallery – not something that can be helped in this situation regardless of artists printing their own work (variance in printing would have actually made more of a discrepancy). In this case we are looking to simply present results in a physical space and we are doing so with prints of the same quality, so visitors can get a sense of the process without too many undetermined elements such as paper/ink variables.
That said, we are lucky we have a fabulous staff in our Digital Lab and they have been working non-stop to make sure this show looks amazing (thank you Christine and Sarah). They recently printed the images that were in our Goodbye Coney Island? exhibition and I couldn’t be more grateful they printed for this show.
Mark,
Lots of people I know who submitted are planning on on inviting their friends and family to First Saturday on July 5th, a full week after the opening – they way they can find out if they are in the show before they invite folks…..
Amy
This morning’s mention in the Brooklyn News section of the NYDN would seem to indicate that those who were chosen have already been notified:
- “Still, photographers whose works were chosen were pleased.”
- “Donna Aceto, who submitted a photograph of girls riding the Astroland roller coaster, added, ‘I’m very excited to be included in this exhibit. I love the Brooklyn Museum.’”
So it is the case that the photographers whose works were chosen, or some subset thereof, have already been notified? Or is this a matter of presumption on the part of Aceto, and sloppy reporting on the part of the News?
Hi Richard,
We have not notified anyone as of yet. We will be doing so via the blog, when we do it.
When do you think you will be announcing, since the show is days away?
Hi. Aceto and two other photographers work was featured in a listing of the exhibition, which was published yesterday on Flavorpill:
http://flavorpill.com/newyork/events/2008/6/27/click-a-crowd-curated-e xhibition?context=art
It would seem then that the NYDN took some liberties in their reporting, but are not exactly fabricating news either, as the images on Flavorpill were presumably submitted by the museum.
As with every exhibition, press needed images. We were able to hold off doing that until after the evaluation process (we needed to ensure we minimized influence), but after evaluation ended images were released and, even then, that was a bit late for a typical exhibition. Seriously folks, hang tight.
Shelley:
Thank you for your reply. I really do appreciate all of your efforts and am very happy to be a part of this fascinating and intelligent process!
I work for a gallery in Chelsea and do understand the circumstances surrounding an upcoming exhibition.
My comments were not directed at your or meant to hasten further anxiety!!
Hi Christopher, no problemo – this was a really difficult part of the process for us since it was so outside the norm of what we do here all the time. We didn’t want to say anything specific until we could tell everyone at the same time – the best way to do it was to wait until the site was ready, but it took a bit longer than we expected and…press peeps needed images…and this kind issue started to crop up and couldn’t be helped
My anxiety is getting the best of me, I’m afraid, but hopefully that will go away now that the site is live and everyone can see what we’ve been looking at for the past several weeks. Hope you enjoy the site – everything is live right now! Go to the results button of this page: http://www.brooklynmuseum.org/exhibitions/click/
I have a question regarding this process of selection using Click – I believe it is a valid concern.
I have read through the posted statistics and I noticed that 389 photos were submitted and “on average, each evaluator looked at 135 works”.
I think that this would skew the results. Photos that were unfortunately posted beyond the midpoint probably lost out because of that fact alone which taints the results. Was that taken into consideration?
In my opinion, I believe that in the future the initial entries should be reviewed by a panel of people with mixed levels of expertise. Probably about 100 photos should be selected for the general reviewing and voting – and the photos should somehow rotate so that the photo-stream does not open up to the same specific photo every time.
Thank you;
James Hanlon
Regarding printing, I was concerned from the beginning that the submission requirements evidenced a lack of knowledge of photographic printing. It is true that permitting each photographer to supply his or her own prints would result in disparate results. However, the lack of uniform standards, such as choice of color space, will also result in dispartate results. Since the pictures will not be displayed on a monitor, but rather as a print, it is a disservice to the individual photographers to not consider the printing as part of the artistic process.
Hi James,
This is why all entries were randomized and the randomization was based on least seen, meaning our algorithm would keep pushing images up that had been seen the least to ensure all entries were seen an equal amount of times. This random order was not set per person (meaning, the order was not pre-determined for each person), it was set per image and was solely based on what images had been seen at any given time.
Hi Gary,
Perhaps you should see the show first? You may still not be happy, but I’ve outlined the reasons for our decision above and we have to keep going back to the fact that the show is more conceptual in nature.
first, i would like to congratulate all who got in.i do however find it strange that the nydn ran an article on moday 6/23 and three of the photographers mentioned(and obviously)interviewed-happen to be in the top 20%-since we were told the results weren’t posted until wednesday,so apparently they were told earlier.also, it would be very interesting to see which 78 the “experts” would have chose. although the point was to see what a “diverse crowd” would choose-esentially the museum currated this show-by selecting which % would be hung in the actual exhibit-so it would be interesting to see if the results would have been the same.i also agree w/ gary slomans post-since we are in the digital age-the artistic process was taken out of this-this was more about the power of the internet-than the power of an image.
Hi Nicole,
You can see some of what you are looking for by going to the website results and going to Search in the top menu. If you use the drop down menu to select the top 20%, then select the “experts” checkbox and run the query. The results will be displayed. Then, you can compare that by going back into the search box and this time select “none” instead of experts. You can start to see how differing groups looked at the same images.
Shelly -
I’m sorry but I just don’t buy your explanation.
In the “Facts” section it is reported that “on average, each evaluator looked at 135 works”. That means that more than half of the contributions were not evaluated.
In my opinion, the results were tainted. The playing field was not equal across the board for every contributor.
There were “3,344 evaluators (who) cast 410,089 evaluations”. Out of the 3,344 evaluators only “575 people evaluated all 389 of the submitted works, completing the evaluation.”
The bottom line is that the bulk of the entries never had a chance and the efforts of those photographers were wasted and exploited for the sake of statistics and numbers.
I would like to know what the results would have been if you had only looked at the evaluations of the 575 people who evaluated all 389 entries.
Hi James,
Thanks for writing. One of the things we really wanted to do with this show is be as transparent as possible, so much of the data is there for you to analyze and interpret as you wish. You bring up an interesting point here and it is one that our consultants talked to us about at length, actually. We were cautioned to think about this process not as individuals seeing everything, but rather the “crowd” was curating and the “crowd” would self-correct over time. As long as we could get enough people to participate, there would be enough sets of eyes seeing photos at differing times (owing to the randomization) to allow for the crowd’s self-correction. (This notion of self-correction is something that James Surowiecki brings up in The Wisdom of Crowds quite a bit.) The consultants explained this very idea was the difference between an individual curating and a crowd curating and we designed the entire process around this crowd-self-correction idea. For instance, we didn’t offer a way for evaluators to return to works to change their response and we didn’t invalidate data if an entire evaulation wasn’t completed. But I’m no expert
James Surowiecki is going to be writing about the data for the blog a bit later in the run. I don’t know if he’ll cover either of these issues, but I’m pretty interested in what he has to say.
The “crowd-self-correction idea” is flawed. Do you have supporting research for this “theory” or is it just a “cool idea” that museum curators are buying into?
Some of the “Facts/Statistics” offered are inaccurate and/or of little value.
For example: “Each of the 389 images was seen approximately 1,054 times”. That is false. That is a useless statistic which was obtained by merely dividing the number of reviewers by the number of entries – and therefore serves as misleading invalid pseud-data.
I am very disappointed that the Brooklyn Museum has bought into the whole “Crowd-self-correction idea” as if it was a tested and verified concept. Its is not.
I have not read Surwiecki’s book, but if popular culture or national election results represents the wisdom of crowds, I am not impressed with the concept. However, even if one were to accept that somehow the masses will have the right answer, the methodology used, of viewing each photo seriatum in a vacuum, except for remembering the prior images, seems fundamentally flawed to me. No curator would ever make selections in such a fashion, but would view them all and gradually winnow them down. Why it is better for the crowd to use this peculiar (annoying and frustrating) method, is beyond me. And why is it important not to go back – again, this is totally artificial and would never be required in the real world.
Hi folks I’m sorry I opened or re-opened this can of worms.
They released the names, and images, thats all I wanted.
We are all a part of a momentous endeavor, for which I AM EXCEEDINGLY GRATEFUL!
I hope to evaluate the exhibit in person, very soon, at which point I will access just how excited I think I should be.
If anything, I can add to my resume, the fact, that my work was in the Brooklyn Museum!
That makes it all worthwhile for me. Life is too short for me to get twisted up over all these statistical and technical details.
You might not agree or accept my zen like acceptance of all this drama, but I think we all need to relax and and enjoy the free ride. For that is what it was, for me anyway.
Professional artists/photographers work for years for the opportunity that I have received. MY WORK IS IN THE BROOKLYN MUSEUM OF ART.
The process of selection was painstaking and even if it was slightly flawed, there is NO reason to denigrate the Museum, Shelley or anyone involved. Name me another museum anywhere in the world where this would even be considered.
And always remember, this museum has 1.5 million holdings, one of the greatest collections of Egyptian Art in the world…repeat in the world, a Luce Study Center for American Art – there are only 4 in the entire country.
My resume that centers on business analysis and technical writing will be updated tomorrow.
Thankyou Brooklyn Museum for the honor of giving me 5 x 7 inches of space on your fabulous walls.
I applaud the open concept of this exhibition, and I think that the process has been explained clearly since the submission phase, and if I submitted my work, was understanding the experimental nature of this show. I participated and didn’t make it to the top 78 images but that doesn’t mean I should criticize the process and question the crowd theory after the fact. It is clear to me that this has been an experimental exhibition and we all should look at it with those eyes. It sounds like a bunch of sour grapes to me.
My impression at the start of this project was that all viewers were to view all entries. That didn’t happen. That skews the results and questions the logic of this endeavor.
It would be of interest to see the results obtained when looking at the choices made by the group of reviewers that viewed all of the entries.
That ain’t “sour grapes” – that’s a justifiable and logical request.
Hi James,
With this exhibition, we designed the evaluation interface so that individuals were looking at a work of photography individually – we did not expect people to finish and discussed this aspect at length with our consultants. We were following strictures in the book The Wisdom of Crowds and we purposely designed the interface to reflect these theories. The stats are published for your own interpretation and you may disagree, but the exhibition stands as it is.
I do want to thank you for posting – the blog is a great place for a discussion like this.