Cross-posting the Collection to Wikimedia Commons and the Internet Archive

I’ve said it before and I’ll say it again:  it’s simply not enough to publish assets on our own website—we cannot expect people to come to and we need to be reaching out to communities on the web to engage interest in our collections.   With that, I’m happy to announce that we are now cross-posting our collections to Wikimedia Commons and the Internet Archive.  This is something we’d been wanting to do for a long time, but in order to get here we needed to get through the recent rights project and the records release project.  Now that those two key elements are in place, cross-posting made sense as the next logical step.


I should probably take an aside at this point and remind everyone that we’ve not always had great success working with the Wikipedia community.  If you remember Wikipedia Loves Art and managed to get through all the blog posts including the four-part lessons learned, then you know just how complicated and painful that project was.  However, the wiki community is one of the most vibrant on the web and, as a community-minded organization, we needed to regroup and figure out a better way of working with these folks—turning our backs and giving up was not an option.  Let’s look at how this is different.

One of the biggest issues with Wikipedia Loves Art was how much work it created for all involved—countless hours from volunteer photographers, hundreds of staff hours to clean up and caption submissions and even more hours from the wiki community to upload all the assets.  This was a project that simply didn’t scale.  So, in this round, our aim was to keep this a collaborative process with much simpler information management. By cross-posting our assets using a programmable bot, we can get the information to the wiki community in a much more efficient way.  Since the wiki is a living, breathing thing that constantly changes, one of the most important parts of this project is creating a second bot that will monitor the changes the wiki community makes to these records and show us how the records are used.  Once we can get a grasp on these collective changes, we can think about ways to integrate that information back into our collection online—it’s this second bot that creates a two-way exchange and allows us to collaborate more effectively with the wiki community.  In addition, this second bot will also write metadata changes to the wiki, so our data does not get stale.  This is a process that must be carefully choreographed so we don’t overwrite community changes, but we think this delicate dance is one that we can learn an enormous amount from.  All that said, the first bot has been created and is happily uploading assets as I write this. The second bot will follow shortly after we’ve gotten everything posted.

By contrast, posting to the Internet Archive was a much simpler process primarily because it’s a one-way dump—they’ve got a clearly documented API and they have a very open structure to work with.  It’s a bit of a blank slate—you can create your own fields, which means you can apply rights information as needed.  We are posting all of our “no known copyright” images there, as well as all images that we’ve licensed with CC-BY-NC.    Wikipedia will be getting fewer assets because they don’t accept Creative Commons licensing that restricts to non-commercial use and retaining commercial rights is still something that the Museum is interested in maintaining.

Posting to these two communities complements what we’ve been doing at The Commons on Flickr.  Seb has an excellent blog detailing why The Commons on Flickr is fundamentally different in nature and these are all things that we agree with.  Here’s to hoping that bot number two helps to bridge some of that gap.

Our bots are uploading now and will be making progress throughout the next several weeks to finish up the initial upload.  To see the progress, check out: Internet Archive | Wikimedia Commons.

Many thanks are owed to Paul Beaudoin for his great work (bot magic, really!) on this project – thanks, Paul.  For all of their help and coordination, thanks are also owed to Maarten Dammers, Richard Knipel, and Liam Wyatt on the Wikipedia front; Alexis Rossi, George Oates and Yolanda King at the Internet Archive.  Cheers!