During Click! A Crowd-Curated Exhibition, 389 images were submitted during an open call held March 1–March 31, 2008. Artists were asked to electronically submit a work of photography that responded to the exhibition’s theme, “The Changing Faces of Brooklyn,” along with a 100-word artist statement. The open call was a blind process; artists could not see what other artists were submitting. The general public was asked to evaluate submissions online using an evaluation tool from April 1–May 23, 2008. As part of the evaluation, each evaluator self-selected his/her knowledge level (from “none” to “expert”) and designated his/her geographic location. Evaluators were asked to asses the photographs that were submitted, using a sliding scale from most to least effective, taking into consideration aesthetics the photographic techniques used, and the work’s relevancy to the exhibition’s theme. The online evaluation tool was designed to promote fairness. Works were presented at random, and our algorithm ensured that all photographs were seen an equal amount of times. To minimize influence, works were displayed without the artist attribution; evaluators were unable to skip past images or to forward links to individual works. Evaluator comments weren’t visible during the evaluation process

These charts display knowledge level and location breakdowns of the 3,344 participating evaluators and the 410,089 evaluations cast. Here are a few highlights:

• 3,344 evaluators cast 410,089 evaluations.

• Each of the 389 images was seen approximately 1,054 times.

• On average, an evaluator viewed an image for 22 seconds before casting an evaluation.

• 3,098 comments were given during the evaluation period.

• On average, each evaluator looked at 135 works.

• If the evaluator had submitted a photograph, the average number of works he/she evaluated was 289.

• 575 people evaluated all 389 of the submitted works, completing the evaluation. (163 of these people were participating artists.)

The top 20 percent of the 389 images curated by the crowd were installed in the physical gallery during the exhibition.

During Click! A Crowd-Curated Exhibition, 389 images were submitted during an open call held March 1–March 31, 2008. Artists were asked to electronically submit a work of photography that responded to the exhibition’s theme, “The Changing Faces of Brooklyn,” along with a 100-word artist statement. The open call was a blind process; artists could not see what other artists were submitting. The general public was asked to evaluate submissions online using an evaluation tool from April 1–May 23, 2008. As part of the evaluation, each evaluator self-selected his/her knowledge level (from “none” to “expert”) and designated his/her geographic location. Evaluators were asked to asses the photographs that were submitted, using a sliding scale from most to least effective, taking into consideration aesthetics the photographic techniques used, and the work’s relevancy to the exhibition’s theme. The online evaluation tool was designed to promote fairness. Works were presented at random, and our algorithm ensured that all photographs were seen an equal amount of times. To minimize influence, works were displayed without the artist attribution; evaluators were unable to skip past images or to forward links to individual works. Evaluator comments weren’t visible during the evaluation process

These charts display knowledge level and location breakdowns of the 3,344 participating evaluators and the 410,089 evaluations cast. Here are a few highlights:

• 3,344 evaluators cast 410,089 evaluations.

• Each of the 389 images was seen approximately 1,054 times.

• On average, an evaluator viewed an image for 22 seconds before casting an evaluation.

• 3,098 comments were given during the evaluation period.

• On average, each evaluator looked at 135 works.

• If the evaluator had submitted a photograph, the average number of works he/she evaluated was 289.

• 575 people evaluated all 389 of the submitted works, completing the evaluation. (163 of these people were participating artists.)

The top 20 percent of the 389 images curated by the crowd were installed in the physical gallery during the exhibition.