One of the questions people always ask me is how web differs from what happens in the building and that’s a difficult thing to get metrics on.  With Split Second, we are in a unique position to answer that question because we’ve been running the same online activity on kiosks in the gallery.  In this final Split Second blog post, I’m going to compare these two sets of data.

Visitors were invited to take the online activity using kiosks in the gallery, so the data could be compared. 2600 visitors sat down at the kiosks to take the activity for a spin.

You may remember from an earlier post, even though part of the project took place online, we were surprised to see a mostly local audience taking part.  Overall, that local audience spent an average of 15 minutes completing the online activity (as opposed to the general average of 7 minutes).  In the gallery, our visitors spent an average of 4 minutes 18 seconds completing the activity at the kiosks.  Even though they spent less time doing the activity, the average ratings per person were quite similar:  online – 39.1 vs. gallery – 36.7. Also, the in-gallery vs. online completion rates were very similar, which suggests a highly focused visitor consuming content at the kiosks very quickly.  Here are a few charts to show off some of the online vs. in-gallery differences.

PausePlayPlayPrev|Next

Gender (Online): Online women showed up to take the activity in almost twice the force as men.

Gender (Gallery): In the gallery, we had more of an even male to female ratio of participants. Also, in the gallery women tended to be younger than male participants.

Age (Online): Online participants were a bit older than gallery participants.

Age (Gallery): In the gallery, the participants tended to be younger. This may suggest younger visitors to the show overall or, perhaps, younger visitors were more attracted to the in-gallery technology.

Experience (Online): Online participants tended to self-identify in the "some," "more than a little," and "above average" categories.

Experience (Gallery): In the gallery, participants self-identified with a lower experience level. As Beau mentions below, older people tended to self-identify with higher experience levels. Given most participants at the kiosks tended to be young, this flip in the metrics seems to be on target.

Completion (Online): Completion rates were similar with online being slightly higher.

Completion (Gallery): Even though completion rates were similar, in the gallery there was a slight uptick in participants aborting (10.3% in-gallery vs. 6.3% online) the activity at stage two, which focused on engagement. The abort rate in stage three, was almost equal suggesting that both sets of participants were equally engaged around the adding information part of the experiment.

When it came to some of the data that Beau’s been delving into, he ran a comparison of in-gallery versus online data and found his original findings still held:

• No correlation between experience and time spent.
• Slight negative correlation between rating and birth-year. i.e. older people give slightly higher ratings.
• Women rank things slightly higher than men.
• Slight positive correlation between rating and experience, but women consistently rate themselves as being more experienced, so it’s hard to tell whether the aforementioned correlation is caused by experience or gender or what.
• Older people tend to self-identify as slightly more experienced.
• Complexity and information findings still hold.
• Engagement and rating variance, the finding also still holds, though there is an interesting change. In the gallery, rating variance tended to be much higher than online. For the control task, online variance was 520.6, while in-gallery variance was 668.5. For the free task, online variance was 459.1, while in-gallery variance was 510.1. So we’re still seeing massive reductions in variance, but the variance in the gallery was higher to begin with.
• Adding information, the finding still holds, though in the gallery the increase in ratings was not quite as big. (The muting of this effect might be related to the age/mean rating issue discussed above.)

Intoxicated Lady at a Window, late 18th century. Opaque watercolor and gold on paper, sheet: 13 3/4 x 11 3/8 in. (34.9 x 28.9 cm). Brooklyn Museum, Gift of Dr. and Mrs. Robert Walzer, 79.285.

Beau also took a look at the rankings data and found, for the most part, the same works win and lose.  As he notes, “There are some minor upsets, and a few things which might be worth a story. In particular, Intoxicated Lady at a Window seemed to always do quite a bit worse in the gallery than online.”  While we are not totally sure why this painting didn’t do so well in the gallery, it’s interesting to note that this was the image that the New York Times used when the project was first announced.  It’s very possible that we had an information cascade happen online with participants rating this work higher because they might have been more familiar with it. This is one case where the in-gallery metrics might actually be more accurate and it shows just how delicate subconscious effects may be.

As Joan mentioned in one of her posts, Split Second closes at the end of the year.  If you have not managed to see it in the gallery, we hope you can come take a visit because the show will be gone in the blink of an eye.

I’ve had a lot of time to mull over the results of the Split Second, so here are a few of my thoughts—roughly one week before the Split Second exhibition closes. Please bear in mind that I don’t bring any expertise on Sociology or Psychology or Statistics to the picture.  What I do bring is many years of experience working with Indian art and with people who are looking at Indian art for the first time.

The original intent of the Split Second experiment was to measure people’s reactions to works of art as they encountered 1) objects that varied in degrees of complexity and 2) viewing situations that varied by time of exposure or degree of engagement. In theory the experiment could have used almost any type of art, and participants would have behaved in the same ways whether they were looking at Japanese prints, or Goya etchings, or Plains Indian ledger drawings.  After looking at the outcome I have to say that I’m not totally sure that we would have gotten the same patterns of response for different genres or traditions of art.  I think the fact that we used Indian paintings affected the outcome and here’s why:

First let me say that I wasn’t one bit surprised that people liked the objects better after they were given some information about them. My first experience of Indian art could best be described as “love at first sight” but I know that was unusual. The vast majority of people can’t get comfortable with an image of a guy with an elephant head and extra arms  —no matter how gorgeous—until they know why he has that head and what those extra arms mean.  When we ask visitors what they liked or disliked about installations of Indian art they almost always assess the quality and quantity of the information we offered first and then talk about the beauty or selection of the art as a very secondary concern. I am pretty sure that this is not the case when the same people are asked their opinion of displays of Western art, particularly paintings.

I have to wonder whether there would have been a marked difference in reaction to the same object in informed versus uninformed viewing experiences if we had used American still life paintings or French landscapes.  I think that the unfamiliarity of Indian painting—which I cited as a good quality for the project in my last blog—led to more dramatic results in the informed/uninformed section of the experiment.

The other place where I think our use of Indian paintings affected the data was in the complexity issue. I was initially really surprised that complex images rated as highly as they did in the split second viewing.  Advertisers know that you can grab people’s attention in an instant using big, bold graphics and a simple message.  I would have thought that the more brightly colored images with less going on would have rated higher because people could take them in quickly.  But the opposite was true.  Straight-forward, easily legible images like this one didn’t do very well at all (in fact it was among the least popular)…

Nayika Awaits Her Lover (Rajasthan, Bikaner), 1692. Anonymous Gift, 81.192.3.

…while very complex images with more than one focal point fared very well despite the fact that there was no way people could take in all the info in 4 seconds. Here’s an example of one that did really well:

Krishna and Radha Under a Tree in a Storm (Punjab Hills, Kangra), c.1800. Ella C. Woodward Memorial Fund, 70.145.1. (Punjab Hills, Kangra), c.1800. Ella C. Woodward Memorial Fund, 70.145.1.

I think the preference for complexity comes from the fact that participants knew they were rating art, and people have different criteria for judging art than they do for other means of communication.  Even in an age when conceptual art and minimalism are part of the canon, I think a lot of people retain an old-fashioned preference for art that looks like it took some effort to create.  And I would argue that this is particularly true among those who know even a little bit about Indian art: people expect Indian art to display virtuoso craftsmanship and lots of elaborate detailing. So participants—consciously or not— gravitated toward objects that looked the way they thought Indian art should look.  Again, I have to wonder if complexity would have been as popular if participants were judging British portraiture or Greek sculpture.

People have asked me if the results of the Split Second experiment will change anything about the way I present works of art in the galleries and I have to say that the answer is probably no.  Mostly that’s because I’m not trying to sell anything in the galleries.  I’m not in the business of giving people what they like.  I’m in the business of informing people and of introducing them to things that they haven’t seen before.  Obviously we want the art to look as beautiful as possible, and if visitors leave the galleries feeling that they like the art, that’s great, but that’s not the only response I’m hoping for.

One of the most universally rejected paintings in the Split Second experiment is also one of the most significant from a historical and even political vantage point:

A Maid’s Words to Radha, from a manuscript of the Rasikapriya. Central India, 1634. Gift of the Ernest Erickson Foundation, Inc., 86.227.51.

This painting comes from a manuscript that is important to art historians because it can be dated to a precise year (most early Indian paintings cannot) so it serves as a landmark for dating all other paintings of its type. It’s also in a style that one very influential Indian art historian promoted and popularized as “quintessentially Indian,” a designation that was particularly important in the first half of the twentieth century as India was struggling to gain independence and to re-establish its own traditional culture after centuries of change brought by foreign conquerors.  I’m hoping that these facts enhance your interest in the painting, but I’m guessing that they don’t make you like the painting any more than you did before.  Because the truth is that it’s kind of crudely painted and you either appreciate its rough simplicity or you don’t.  But the fact that you didn’t like it doesn’t mean that I’m going to stop showing it in the gallery.

The one place where we want to give people art that they can instantly like (or at least find engaging) is in choosing the images we use for our advertising.  Maybe the results of Split Second can give us some insight into the kinds of Indian paintings we choose for promotional materials in the future. Those images can get people into the galleries and then I’ll take it from there.

I am listed as a contributor to the Split Second project, but I really wasn’t the brains behind it; I’m just the person who okayed the use of Indian paintings and then wrote the accompanying labels.  Think of me as the grocer who provided the ingredients for the meal that Shelley and Beau cooked up.

I’ve been silent so far because the analysis of the results is really a matter for someone with a more statistical bent.  But since the project assessed the perception of works of art there might as well be a little discussion of the art we used.  I’m going to give you a little background info here and then later I’ll talk about my responses to the data we gathered.

First of all, a plug: the exhibition closes December 31, so I encourage you to get to the Museum before that.  The paintings are really wonderful and won’t be on view again for a while because they’re light sensitive. We’ve got some serious masterpieces on view.  This one in particular is a show-stopper, made by a team of the best artists in India for an emperor who spared no expense:

Led by Songhur Balkhi and Lulu the Spy, the Ayyars Slit the Throats of Prison Guards and Free Sa'id Farrukh-Nizhad, page from a Hamza Nama manuscript. Imperial Mughal school, 1562-77. Museum Collection Fund, 24.46.

If you come to see the paintings in person I think you’ll be surprised.  They’re definitely not as flat as they seem on a computer screen and they’re all different sizes—something you just don’t comprehend when you look at reproductions, even if the dimensions are listed.  This painting, for instance, is the size of a subway poster (for a train not a station) while most of the others are more the size of a page in a coffee table book or even smaller. In many cases, you can see the exquisitely painted details far better in person.  So hurry over!

Let me tell you a little about why we chose Indian paintings in the first place.  First of all there are the nuts-and-bolts reasons: we have a lot of high-quality Indian paintings in the Brooklyn Museum collection and all of them had been photographed in color thanks to a big digital capture project we did a couple of years ago.  It also seemed like a nice complement to, and subtle promo for, the big Vishnu exhibition, which was going to be on view for much of the same period as the Split Second installation (Vishnu closed in October).

Then there are the more intellectual reasons: Indian paintings are basically flat, and they are unfamiliar territory for much of our audience.  Flat is good because photographic reproductions of flat objects are more straight-forward and uniform than photographs of three-dimensional objects.  We were worried that variable factors like background color and dramatic lighting would influence participant reactions to photos of teapots or scarabs.  There are variables in the photography of Indian painting—whether one uses raking light to pick up the glint of metallic paint, whether one includes all or some or none of the border that appears around most Indian paintings—but they’re not as significant as those for photography of 3D objects.

Unfamiliar is good because we wanted people to come to the material with fresh eyes and few preconceptions.  We didn’t want them to recognize masterpieces or famous artists and rate them more highly because they felt like they should.  We had people describe their level of expertise or familiarity with Indian art before doing the experiment and most were complete newcomers.

There is one way in which Indian paintings were inappropriate material for a split-second experiment: these paintings definitely weren’t designed to be glimpsed quickly.  “In your face” impact isn’t a quality many of them were supposed to have. With the exception of the oversized painting illustrated here, they were all gathered or bound into manuscripts; their aristocratic owners held them in their hands or on a table. In intimate groups or solo, the viewers went slowly through the pages, looking at the paintings as a form of entertainment. Book illustrations require a different style and approach to image-making than wall-hung paintings that might be seen from across the room. The many tiny details that you can find in Indian manuscript paintings are a result of their relatively small size, but they are due even more to the practice of looking at manuscripts closely and at length: the artist wanted the viewer to have plenty to look at, to make new discoveries every time he or she opened the book.  So these illustrations were rarely judged on their ability to make a split-second impression—until now!

A big part of experiencing art is talking about it. Sometimes (or, uh, frequently) artworks are successful because they provoke disagreement, and along with that disagreement, some good conversation. Because the participants in the Split Second online experiment weren’t communicating with one another, we didn’t get an opportunity to measure conversations about the artworks directly. However, we did want to get a sense of which works might be contentious, and to make an effort to figure out why.

To measure contentiousness, we looked at the variance of the ratings for each work. If most participants gave a work roughly the same rating, then it’s safe to say that work is not contentious. However, if participants disagree, if there’s a large amount of variance in the ratings, then that work might be contentious. (I say “might” for a good reason: while high variance of ratings may indicate disagreement, it could also simply indicate confusion. I’ll come back to this later.)

In Split Second Stats #4: Engagement we found that certain tasks in the experiment had a strong effect on the variance of ratings. This is important because it indicates that the context of presentation and the way participants engage with a work can change the variance. Here, however, we’ll take a look at how variance and contentiousness were related to specific properties of the works themselves. All of the analyses below apply to the unlimited time experimental tasks only.

As in many of the analyses described in previous blog posts, complexity played a big role here. We found that as paintings got more complex, they became less contentious. That is, we found a negative correlation between complexity and variance (cor = -.35, p = 0.03). This is not too surprising: we found previously that when time was unlimited, people tend to rate complex paintings very well, a finding which already implies inter-participant agreement. A more puzzling finding concerned color: The higher the overall saturation of the colors in a work, the higher the variance (cor = .42, p < 0.01). One possible, but entirely speculative, explanation for this effect is that one large group of our participants reacted very positively to highly saturated color palettes, which another large group reacted very negatively. Similarly, we found that the larger the frame of the painting, the more variance in ratings. This again might suggest (speculatively!) a division of the participant population into two groups: those that found large frames interesting, and those that found them to get in the way of the work.

Some of the strongest effects concerning variance were not clearly related to quantifiable properties of the works themselves. One very strong, reliable finding was that as the average amount of time participants spend looking at a work increased, the variance of the ratings of that work decreased (cor = -.47, p = 0.002). That is, the more time was spent looking at a work, the more our participants tended to agree about how to rate it. Though this finding seems to push against the gist of the thin slicing theory, it also seems like an encouraging experimental result: in order to get people to agree about art, you just need to get them to hold still and look at it for a long time. However, it’s a little bit more complicated than that. People decide for themselves whether or not they want to spend a long time looking at an artwork. This finding lets us know that when our participants spent that time, they tended to agree, but it doesn’t tell us why they decided to spend their time in the first place. There is also a cause-and-effect problem: it could be that the decreasing variance and the increasing time are themselves caused by a third factor we didn’t measure. (Though complexity looks like it may account for some of this effect, it certainly doesn’t account for all of it.)

Utka Nayika was the most contentious painting.

Finally, we found that some of the works in the experiment were simply contentious on their own terms. The most contentious object, Utka Nayika (pictured above), is unfinished. Though we have no quantifiable measure that points toward it being an unfinished work, it seems like a safe bet that this peculiarity accounts for the high variance in participants’ ratings. As I mentioned before, it’s important to differentiate between contentiousness and confusion. We can identify this work as being truly contentious, and not simply confusing, by looking at a histogram showing how it was rated.

In the case of a work which was simply confusing, we would expect a uniform distribution of ratings, where any one rating was as likely to occur as any other. Instead, what we see here are distinct peaks and valleys. There are small peaks around 25 and 100, and larger peaks around 50 and 75. This indicates participants’ opinions about the work split them into at least three groups: those who did not like it (the peak at 25), those who were decidedly indifferent (the peak at 50), and those who liked it a lot (the peaks at 75 and 100). A similar situation can be seen in the rankings histogram for the second most contentious object, The Bismillah, a work which is distinguished by its calligraphic, non-representational nature:

The Bismillah was the second most contentious painting.

In both of these cases, symbolic factors not accounted for by our experimental model had an extremely strong effect on the results, strongly suggesting a direction for further research. As interesting as it is to see the symbolic world bursting out of our tightly constrained experimental framework, it’s not surprising: we are, after all, looking at art.