Fresh Eyes Provide Insight on ASK

Our entire ASK program has been built upon regular user testing and evaluation, which we’ve always completed ourselves…until now. Since we’ve been trying for over a year to increase the use rate with limited success, we felt like it was time to ask for help. In the fall of last year, we hired ERm, a marketing and research firm here in New York, to bring fresh eyes to the problem. The evaluation had several objectives, including:

• to gauge overall awareness and recall of the ASK app among visitors to the Brooklyn Museum;
• to understand expectations and perceptions of the app based on existing materials and concept;
• to determine response to ASK among users, including best- and least-like features; and
• to pinpoint barriers to downloading ASK among non-users.

During the months of October and November, ERm conducted online surveys, which were sent to email addresses visitors shared with us at the admissions desk and in the museum shop. We also completed four focus groups, two comprised of pre-recruited individuals who downloaded the app and used it during their visit, and two comprised of regular visitors intercepted and recruited in the galleries. Let me tell you, those focus groups were super enlightening! Since app use is anonymous, we’ve rarely had the opportunity to speak with users about their experience (once we were past beta testing). We’ve mainly relied on app store reviews to give us insight.

ERm provided us with a really comprehensive report that we’re still sinking our teeth into.

We are still processing all the information, but ERm has provided some actionable items related to overall awareness and user experience that we can begin to work on right away. According to the study, awareness is quite low. Only 56% of survey respondents had never heard of the app at all. Interestingly, by comparison, the vast majority of respondents (87%) had not downloaded an app from any other museums either. For those that had heard of ASK, their main source of information were staff members (52%) and museum signs (32%) like the labels pictured below. This is good news since we are in the midst of building an ASK Ambassador team that will provide that much-needed personal invitation to ask.

We have continued to tweak messaging in an attempt to build awareness and clarity around the app. The top image is a label in the American art gallery using the ASK brand approach from April 2016. The bottom image is a label in the Marilyn Minter special exhibition using a new approach featuring a question combined with directive to download. Both are up in the galleries now. Labels will be included as part of the fresh look we’ll take at all our messaging around ASK.

Despite our fairly extensive testing around messaging, our current version still leaves visitors unclear of how ASK might enhance their experience. They assume it’s a tour app with a map (an assumption we’ve been fighting all along). Additionally, they are unsure of the usefulness of the app in this age where you can Google anything. A few focus group participants felt that the interpretation already provided was enough information and weren’t compelled to investigate further. (Great news to those of us who have ever written a wall label!) A few respondents felt that being on their phone would detract from their museum-going experience, however 76% of survey-takers said they used their phone at some point during their visit. All of these factors contribute to an overall apprehension to download an app for a few hours’ use. Overcoming these factors through different messaging in particular, is an important next step.

ERm also reported three key areas where users like the app most:

• When it enhances the experience: Users enjoyed ASK most when the information provided supplemented the other forms of interpretation. Specific information that offered fresh details (i.e. the kind of information Googling can’t find) was well-received. Visitors need to be assured that the app will enhance their experience.  A key component to the experience is also length of information and timing of the response. These two components are more difficult to ensure because they are so arbitrary—response length in particular. What may be too much information to some is not enough for others. Ensuring we provide a response before the person has walked away (the ideal timing) is also a unique challenge.
• Because it is practical to use: Users praised the intuitive functionality of ASK and felt that the in-depth information provided by the team helped make the most of their visit. However, that same intuitive function also meant that for some, the app was too one-dimensional. This is likely compounded by the expectation that the app offers a map and tour information. (That being said, even when visitors have thought this in the past, they still didn’t download it.) Additionally, some users felt there was too much pressure to generate questions or maintain conversation. Visitors want to choose whether they will be a passive or active user.
• It offers a human element: Users love the personal and responsive nature of the exchange, which was both conversational and enlightening. The information went beyond a Google search. However, there was quite a bit of confusion as to whether the responses via the app were generated by a human or a computer. This was compounded by the uneven responses a few users received (e.g. some answers felt copied-and-pasted, though the team doesn’t do this as a rule). A few people expressed the pressure they felt to socialize while interacting with a human, while others thought interacting via text removed that same pressure. We need to convey that ASK offers users the compelling opportunity to have an intelligent dialogue with an expert who cares.

We will explore these elements further in the coming months as we move toward ASK 2.0. The good news is, nothing we heard from the evaluation is a surprise. Much of the results cover things which we have suspected for some time. It’s nice to have some solid evidence, as opposed to just anecdotes, behind our assumptions. Stay tuned for more on this as we roll out changes, test solutions, and put these ideas into practice.