MW2010 User experience Web metrics

Tracking what gets ‘used’

Theres been a fair bit of excitement around the traps today about the revealing of Amazon’s tracking of highlighting on their Kindle devices.

In fact this sort of interaction tracking has been going on on the web for quite a while – but the Kindle example is one of the first where this data is being used to encourage serendipitous discovery and interest.

I started doing some work around this on the Powerhouse collection site in July last year and it forms the basis of the paper I presented at Museums and the Web this year (as well as briefly mentioning it at Webstock in February).

We’ve been trying to figure out alternative ways of measuring the success or otherwise of making large amounts of our content available on the web. Traditional web metrics just don’t cut it – millions of views of your content isn’t really helpful in improving the content you make available. And whilst qualitative research is invaluable it is generally expensive and just doesn’t scale.

So in July last year we started using a tool called Tynt Tracer.

What Tynt does is intercepts cut & paste using Javascript. It records what is copied, and, inserts into the buffer the license information and a unique hyperlink. We chose to use Tynt because it was the least intrusive and most anonymous of the options available to do the same task (there are quite a number of similar solutions out there). Tynt was also the option that made the least mention of ‘enforcement’ – which seems to be the selling point of the other options.

We aren’t interested in ‘enforcement’ or preventing visitors from cutting and pasting content – but we are primarily interested in learning about what parts of our content is the most useful to cut & pasters, and where it ends up so we can improve it and its structure.

Here’s what Tynt says about their service.

Tynt Insight anonymously detects when content is copied from your site, and can help determine what they are doing with it. At Tynt we believe content copying can be beneficial to the site owner. We find that most people copy content innocently because they are your fans. They copy content to either preserve it for themselves or to share it. Half of copied content is still shared by email because it is still the easiest and most familiar way to share content.

My paper explores how we applied this in a fair bit of detail as well as some of the findings of roughly six months’ worth of data. Suffice to say, it isn’t perfect and the paper ended up revealing that there is far less educational use of our collection in schools than we hoped for (education users being the ones we’d expect would most likely cut & paste!) – but that’s another blogpost.

Nearly 3 million words had been cut and pasted during the sample period. That’s possibly a better measure of the success, or ‘usefulness’, of our collection metadata than object views.

During a six-month period, 20,749 copies were made: 5% of these copies were images – predominantly thumbnails and, curiously, the Museum’s corporate logo; 36% (7,601) were copies of 7 words or less in length. Tynt calls these ‘search copies’ and implies that their likely use was for use in search. These search copies do not have licence and linkback text appended to them. The remaining 58% (12,608) were copies of greater than 7 words and thus had license and linkback details added to them. These 12,608 copies contained nearly 3 million copied words (2,906,330 words).

We’ve been looking at the resultant heatmaps that highlight the content that gets most cut and pasted. These offer the opportunity for us to learn and think about how we present and refine content for certain types of users.