Pilot 3: Texting

Last week we wrapped up our final planned pilot project to help determine the direction for ASK 2.0.  Another somewhat obvious solution to the challenge of people not wanting to download an app, why not text us instead? We set up a Twilio account and spent two weeks essentially pretending we didn’t have an app. The ASK Ambassadors pitched the texting service, and with the exception of international visitors without data plans, didn’t talk about the app at all. We had dedicated palm cards featuring the phone number and a few “helpful hints” for ways to use the service.

Our texting palm card included the phone number as well as “helpful hints” on what kinds of things to text. Unlike the app (which is geo-fenced), functionally you can text us anytime, but we won’t answer outside of Museum hours. The system will autofire the same “out of office” notification” app users also receive.

Our developers did some backend magic so that the Twilio messages would push to the dashboard, which allows us to keep that single source for incoming messages. Unlike the app, the SMS messages do not utilize location aware, which meant the team was flying somewhat blind in the dashboard. Normally, when a visitor sends us a message, the nearest beacon responds and the dashboard populates with the artworks on view in that gallery. Each artwork has the associated metadata from our collection online as well as “snippets” (question and answer pairs) from previous conversations that have been tagged to the work. SMS messages provided few of these tools to the team, who had to manually search the collection online (or good, old Google) if they didn’t know the work already. However, this isn’t the first time we’ve dealt with this challenge, so going into this pilot we felt pretty confident we could handle it. As expected, response time suffered a bit, but overall the team did really well and lack of location data didn’t hinder them much at all. One happy discovery was that the image recognition the developers put into place last year occasionally worked with MMS message, so if a user sent us an image (many do), there was a chance the dashboard would find it and pull the metadata as well.

I have to say, out of all the pilots, I thought this would be the most successful, and I was right. Visitors really responded well to the idea of texting, and we ended up having to reorder palm cards twice. Despite this enthusiasm, use rate was not what I would like to have seen; we averaged just above 2%. Interestingly, not all of that traffic was from texting. We still had some iOS and Android users, only 19 of whom were repeat users, so some folks were finding and using the app despite our promotion of texting (and likely not all international visitors). Out of curiosity, I wanted to compare this most successful pilot to our most successful two non-pilot weeks. Turns out the average use rate of those two weeks is higher than any of the pilots. So alas, while this pilot had the best use rate of the three, it did not hint at a “solution” to our use rate plateau and was not better than our best two (non-pilot) weeks combined.

By way of quick recap: pilot 1 (provided devices) proved charging for devices doesn’t work, while free devices didn’t give us more app traffic; pilot 2 (ASK on Demand) showed that while people liked the idea of an in-person chat, few took us up on it; and pilot 3 (texting) did not show better numbers than really successful non-pilot weeks.

What’s next? I’m not 100% sure, but we’re kicking around a few ideas. We just had a big team meeting with the ASK team and Ambassadors to discuss these metrics and share observations, which I’m still chewing over. Some things we are exploring include providing devices for scheduled groups and keeping the texting service as an alternative for those that just don’t want to download.

It’s going to take a little while to figure out, but I promise more to come after vacation (yay!) and time to think.