Many of you may have seen Edward Rothstein’s assessment of mobile technology in museums, but if you haven’t it is certainly worth a read and a bit of discussion. The article looks at our mobile application along with the Museum of Modern Art and the Museum of Natural History and Rothstein pretty much dislikes the state of the union across the board.
I had mixed feelings about the article—I mostly agree that these apps all leave much to be desired, but I disagree that we shouldn’t be trying. Experimentation without perfection is a good thing. You may remember, I have my own issues with the use of technology in museums, had a less than stellar experience using the AMNH Explorer app and we’ve had to rework our own mobile app once already. Now is a good time to look at what the author is saying and discuss the current state of our mobile application.
One of the things Rothstein brings up is the lack of geolocation in our app. He wants the device to automatically locate where he’s standing and magically deliver content—don’t we all? We have GPS and AMNH’s Explorer app to thank for setting the bar so high, but in terms of what we can do here in Brooklyn, it’s just not possible yet. While we do have a museum-wide wireless system, it was put in during 2004 and we don’t have the meshing technology required to triangulate signal (something that would require replacing the existing wireless network in its entirety), so we rely on people’s use of accession numbers to look up information about objects. This is not perfect by any means, but it’s the simplest, clearest and most sustainable way we’ve come up with to deal with the nearly 6000 objects on view. We tried other methods in version 1 of our app to no avail and we’ve considered switching to QR codes or short numeric codes, but that’s not realistic for this many objects. Given every object has a unique number published on the object label and we need to develop a system that works with every object on view, accession number lookup is the way to do it…at least for now.
Rothstein makes an assumption about low usage of our app and this is true in some ways, but not true in others. First and foremost, we don’t have a large audience for our app. In the galleries on any given day (especially Target First Saturday), you’ll see very few visitors pulling out smartphones. Eventually, that will change and it’s important to have a system in place as we start to see this turn around, but for now we are consistently seeing clamshell phones on cheaper monthly plans. Beyond this, our app has suffered from poor visibility throughout the building. I will admit that I’m really jealous of the amount of visibility the AMNH app—big signage everywhere, staff have Explorer t-shirts and ads are seemingly all over the place—as simply as I can put this: I want.
Just recently, we managed to get directory signage better positioned and our designers are helping us by including a picture of the iphone. We saw a slight rise in usage when the signage went in, so that’s helping a bit.
So, let’s take a look at what’s really happening when people use this. The statistics are indicating that they are doing so for pre-visit information (directions, hours, exhibitions, calendar) and that’s something that closely mirrors our general website traffic patterns. It’s not that visitors are trying to use BklynMuse (our collection search) and failing or trying to play Gallery Tag! (our gallery game) and giving up—they are not getting that far. This could indicate two things: 1) that visitors want to use the application pre-visit, but they don’t want it to be part of their in-gallery experience and/or 2) our app’s home screen is not clear enough to explain all the choices available. For our next round of changes, we are going to concentrate on the latter and see if that changes the metrics.
What in the world do I get behind doors labeled BklynMuse and Gallery Tag? It’s just not clear.
Where Rothstein’s assumption falls really short is what happens when people use BklynMuse. What we are seeing in the statistics indicates that when people are using it, they are using it in an interactive way. When you compare visitor’s use of the “Like This” feature in-gallery to the collection online, what you see is that on the whole, people in the gallery are using this feature to recommend objects to other visitors. So, in theory, this kind of recommendation layer where we directly ask people to help guide others is working—we just need to do a better job getting people to the feature.
Low usage overall? Yes, but “Like this” feature is being utilized in the gallery more than on our website.
Rothstein goes on at length to talk about why none of these apps measure up to the experience he wants in the gallery and there’s a point to that. Each and every visitor walking in our doors is likely to expect something different from an app and every visitor is going to respond differently to what we provide. My point is that it is our responsibility, collectively, to try new approaches and provide as many entry points into content and the museum as possible. In terms of Brooklyn’s people-focused mission, we believe a people-focused application is the way to go. The curated content is already on the walls in the form of object installation, labels and didactics, in-gallery multimedia and gallery design. The power of the device means we can provide something else, something more unique. We believe leveraging the power of our visitor’s voices in combination with our own is a worthy goal. Are we there yet? No. Should we try, discuss, learn from our visitors and continue to iterate? Yes, yes, yes.
I’d love to discuss more via the comments. There’s a lot to cover on this subject, that’s for sure.
Author RSS 
RSS 
Eastern Parkway/Brooklyn Museum
Shelley – Thanks for sharing this. What you are doing is so important. For years now the Brooklyn Museum has been dreaming of what could be, and creatively pushing the limits of technology in ways that inspire other arts organizations around the world.
It seems like there is a certain kind of naivety in expecting that a phone app will measure up without any really understanding of your approach from the ground up. That is – building organically, listening and responding to feedback, and rebuilding iteratively.
QR codes on object’s labels please! (that lead to the object’s permalink)
You make excellent points here, Shelley. Rothstein’s quest for _his_ digital experience will become more and more common as the public is increasingly conditioned to customize the presentation of information. So, how do we invest in infrastructure that will allow museums to grow with the changing landscape of real/virtual visit mash-ups? Wish there was a magic answer for that bad boy.
Having actually worked on several mobile experiments with 2 major museums in NYC, I can honestly say … cut the institutions some slack. There are a variety of difficulties that these institutions have to overcome … not only on the technical end, but funding and politics.
Museum politics is as bad as any fortune 500 company and their appetites for spending on forays into non-core museum programs is tiny. So I applaud any institution that can pull off things like experimenting in mobile platforms (even if it doesn’t live up to everyone’s expectations).
I won’t say which museum, but I remember one experiment we did in location/geo-based way-finding mobile guide, only to discover that the building was built like a bomb shelter and shielded 95% of electronic signals from room to room.
So apps will succeed and fail, but without positive support these programs will disappear from the board of trustee’s funding schedule.
I totally agree with you about trying. About people that don’t use mobile app in the galleries, this is my experience: people really rarely come alone in museum. Often they’re with friends, or groups, or with family…isn’t possible that the “phone experience” it’s a too solitary way to explore museum? I saw so many times couples of visitors who was listening, very closely, on the same audio-tour device. But you can’t do this with a smartphone. How to fix? Really, for now I don’t know, but I think that yours “like this” button is the better try so far.
Thanks so much for responding to this article in such a thoughtful and positive way, Shelley. I have to agree with other comments that allude to the investment of blood, sweat, and tears on the part of museum staff who conceive of and build these applications. I may sound dramatic, but anyone who’s working in the trenches on these projects knows that blood, sweat, and tears it is!
Politics aside, basic data issues, rights and reproduction, staff time and expertise, etc. all make producing these apps a much lengthier and more difficult process than meets the visitor’s eye.
I was particularly disheartened to read Rothstein’s comments about the lack of information available in apps, as if he assumed he’d be able to find every object with a complete catalog record. Comments like that assume that museums are withholding information from the public, when the situation is exactly the opposite. We publish incomplete records instead of no records at all. Wouldn’t we all love to have fully fleshed out records at all, much less on our websites and mobile apps?? Likewise, the argument about social tagging lacks any discussion of authority control (if Rothstein is aware of that half of the process) and the positive ways that social tagging (if implemented well) can act as a bridge to exactly the kind of information he was hoping to find.
One of my favorite features of the Brooklyn Museum’s online collections is the indicator of record completeness. This level of transparency is admirable and I only wish everyone could celebrate burgeoning museum technologies as dynamic works-in-constant-progress, rather than disappointments or somehow failed attempts.
Thanks so much to the Brooklyn Museum for continuing to lead the vanguard of innovation for interactive museum technology. Those of us who are newer to the game are greatly indebted to your vision and good example.
And amen to derekdj’s comment about the need for good PR!
I agree with all of the above comments. Brooklyn is a leader in the field and you should be commended for your willingness to experiment with new technologies. I also admire your bravery to share your successes and failures with the museum community. We have to find creative solutions to big problems and may only succeed if we try, take stock of what works and what doesn’t, and then try again.
I applaud you guys for being in the space, and also being willing to be transparent about the challenges in a forum like this. There will always be critics, but kudos to you for continuing to evolve and to set the standard high for other institutions.
I agree with T. Lowe. QR Codes on the labels would be a great “a la carte” method of accessing more information about a piece. Also like the other commenters, I applaud your efforts.
The Museum of London’s iphone app, “Street Museum” uses historical photographs from the Museum’s collection combined with a GPS system to locate where you are in London and show images from the past of the location. http://www.museumoflondon.org.uk/MuseumOfLondon/Resources/app/you-are- here-app/index.html
I agree with Carlotta that using a smartphone in a museum is a too solitary way to explore a museum. I have seen touristguides using a small microphone for a guided group and group-participants using small in-ear headphones listening to the guide. Would a combination of a smart phone with a second or even a third (bluetooth connected ?) in-ear-hedphone be a solution for this problem ?
Has anyone experimented with this?
Hi everyone, many thanks for your comments and discussion thus far.
As I mentioned in the blog, use of QR codes on 6000 object labels is not sustainable in any way shape or form. Just creating that many codes and keeping them up to date as objects move on and off view is a total nightmare and I certainly don’t want a museum full of distracting codes and 6000 is too many! QR only works well when you are doing things selectively and that’s not the case with our app, where we want every object to be available. When in use selectively, then you have #1 a visibility issue and #2 a user expereince issue…the codes don’t work well in low-light and they don’t have broad user acceptance yet…it’s like throwing more tech in the mix for tech sake and that’s not good.
I do agree with Carlotta that we do need to better think about the way visitors are coming here and, if in pairs, need to develop accordingly. The phone is a solitary experience and even if multiplayer interfaces were developed, I’m not sure it’s the kind of social experience visitors want.
In addition to my thoughts on the article that I blogged at http://wiki.museummobile.info/archives/13727 I just wanted to add a quick note to the discussion of the “solitary” mobile museum experience. I really think this is a content and not a technology issue: I’ve taken great old-fashioned audio tours whose content was so designed that it had everyone playing and collaborating during their museum visit, or even uses the same broadcast content to get everyone doing something different. Equally I’ve visited plenty of social media sites with all the technical bells & whistles where no conversations or interaction were happening at all. I think Robin White Owen is asking the right questions about using mobile to facilitate conversations in the museum: http://www.mediacombo.net/blog/2010/10/what-does-it-take-to-get-a-conv ersation-going/ This is not just a question of technology, and throwing QR codes or any other flavor-of-the-month at the problem isn’t going to solve it. It’s about content and user experience design first and foremost; then we pick the technology that serves up the experience, and ideally becomes invisible within that experience.
Ms. Bernstein,
The article and your subsequent post are both useful to the museum professionals and visitors. Although I agree that Rothstein’s criticism was directed towards his own interests, his perspective raises some interesting points about the use of smartphone applications in museums today. As the patrons themselves are the ones using these apps at museums, Rothstein’s opinion does indeed show that the development of these apps will continue to be patron driven as you confirm in your post. I commend your commitment that further developments and improvements must prioritize creating the best visitor experience. Central to Rothstein’s critique was the demand for greater content. As patrons turn to these applications they are obviously showing an initiative to learn more outside of the information provided on the wall label. These mobile applications are the perfect place to house additional and exclusive content. Any of the supplementary content should aim to create a personal relationship between the object and the viewer. Perhaps future improvements can provide audio commentary about the work from the artists themselves (when possible) or short videos illustrating the work of art’s production or restoration process? Or maybe even references to similar works of art and book titles for further information?
I agree with Rothstein’s evaluation of the Gallery Tag game, where its current format seems to dumb down curatorial practice. However it is beneficial as a teaching tool for a younger audience. Perhaps the game could be modified to a timed gallery scavenger hunt for older audiences? Instead of a keyword, provide a cropped pixilated image of work of art that visitors have to correctly guess under a time limit. Additionally, why not provide timed gallery quizzes where art history buffs can compete against other visitors? In addition to bragging rights for winning these timed games, why not provide exclusive rewards, like discounts on membership or at the gift shop as an incentive for a top score? I believe that the overarching aim of these technologies should generate content you can only access within the museum. With the proliferation of resources available on the web, patrons can often default to viewing information from the comfort from their own home. Exciting mobile content has the chance to provide deeper educational information or fun museum games will compel visitors to personally and physically experience the objects of art at the museum. I hope these suggestions will in the future create a richer interaction in a patron’s visit.
Last week, I visited the Brooklyn Museum, eager to use the much written about multimedia app, but there was a small issue – I don’t have a smart-phone. I approached the woman at the info booth, and inquired about how to rent one for the afternoon. She promptly replied that I could get an Ipod audioguide for a $5 fee, but no smart-phone apps were available. Four days later, I’m still frustrated over this; so here I am, reading these articles without any first hand experience in the newest wave of apps.
That being said, I think that the sheer concept of these apps hold great potential, and that some of the critical articles “missed the mark”. Rothstein’s article pointed out the flaws in the app, without acknowledging the positive aspects or the fact that the developers are trying to work out these issues, knowing full well that they do exist. Was he under the impression the developers thought the apps were perfect? As you mention, “it’s just not possible yet. While we do have a museum-wide wireless system, it was put in during 2004 and we don’t have the meshing technology required to triangulate signal.” Perhaps Rothstein doesn’t remember the controversy of a few months ago, as his company, the NYTimes decided that they will implement a pay wall in the future – assuming they can figure out how to do that successfully.
In a Museums and the Web (2009) article by Koven Smith, he makes reference to the demographic issues. While we, blog readers and writers, feel comfortable with technology, we must remember to ask ourselves who is going to use the technology, and at what museums? There are some museums in NYC in which the majority of the audience hovers around the age of 60, and these people can not figure out how to use a audioguide. Should we give them an iphone with apps? Surely not. And for this reason, AMNH and the Brooklyn Museum have apps aimed at their young audience, rather than the Frick Collection. And, as Smith points out, the technology is ahead of the designers, creating a lack of cohesion, at present time. Additionally, these devises can not be the most important part of the visitor experience, as the Doyles (Museum and the Web 2010) point out, “we have to remember that museums have no distinct competitive advantage as Web site content producers. The ultimate value of the museum lies in its physical presence, its collections, exhibits, visitors, staff, and community.”
The crux of this issue is not developing a better app, its how the museum’s structure will change in the next few years. The visitors services departments will have to communicate more with the education and media departments as visitors transition to more complicated devises, while still learning artistic content (perhaps now the visitors are focusing on technological learnablity rather than art). Visitor research will have to be redone, and the museum as a whole will change, just as it did many years ago when education took prominence over voyeuristic modes of display.
PS- Please, please, please make some iphones available to rent!
Hi Stephanie,
One of the things that bothered me most about the review was that Rothstein got our in-gallery tagging mixed up with our online tagging (this is easy to see in what he points to) and that is unfortunate. Both of these applications are very different and there’s no question the benefit of our tagging system online which is both very successful and made our database increasingly better indexed. Gallery Tag! operates differently and needs to be played in-gallery (something you are advocating!). I notice you are commenting from USC, so I’m not sure if you’ve had the chance to try it in person? I will mention, during the mobile meetup we had good audience reaction to it and that’s what we wanted to see – average visitors coming to the museum and enjoying it. Of course, there’s always more that can be done, but it’s important to try one thing at a time and see how visitors are reacting to it.
Hi Blaire,
I’m sorry that we cannot provide units for checkout at this time. I mentioned this in an earlier blog post – right now we are in a highly experimental phase and we do require visitors to bring their own hardware. As much as we’d like to provide rental units, it comes down to a factor of cost for us. In the end, we decided to do the project anyway (all in-house, no budget) and then gauge demand for rental. So far, we’ve not seen many requests. I will say, this was also true of my visit to AMNH. They did have the budget and are providing units at the desk for rental, but the day I was there we took out unit 001 and that was the only unit checked out during our three hour visit. I asked the folks at the desk how much demand they had and they said not much. Should we ideally be providing units? Yes, no question – it makes for a better visitor experience overall, but I don’t think we should avoid a project if we can’t afford it – especially a project in early stages.
So what’s wrong with using accession numbers anyway. They’re already there, and each item has a unique number. Seems to me that there’s very little advantage to adding yet another code that would require more maintenance. On the subject of infrastructure and design, this is nothing new. I’ve been in the IT business for longer than some of the target audience for this app has been alive. Hardware has ALWAYS been more advanced than the software running on it. So cut the museum some slack. It is virtually impossible to know what the museum goer will want in a smart phone app. It’s a market place like anything else. You bring something out. You find out what works and what doesn’t, and you go from there.
Hi Shelley,
I’d like to echo the thanks for your time and effort. I’m a technology-friendly artist new to the city and haven’t visited the museum yet, but have followed the blog posts and love what you are doing.
Rothstein is a frustrating audience but a typical one. An average-leveled technology user knows enough to expect advanced features, but not enough to understand or fully utilize what is already there.
The article is still useful. It opens with a great point: people love taking photos in museums (I, personally, love taking photos of people taking photos in museums). I see from the comments that QR isn’t viable… is image-matching? (Sorry if I missed this in other comments…) Codewise it’s not too hard, and you seem to have a well photographed archive. Visitor-generated image galleries could make the at-home web experience friendlier (and inspirational), too.
R’s comments on tagging are less useful. Broad tags find deeper connections. And the fact that you’ve distilled the interactivity to the two most loved and useful inventions of blogging and social networks (tagging and liking, respectively), shows you know what you’re doing.
So thanks, congrats, keep it up, and I look forward to visiting and trying all this fun stuff out soon.
Laura