A new framework for altmetrics

At total-impact, we love data. So we get a lot of it, and we show a lot of it, like this:

There’s plenty of data here. But we’re missing another thing we love: stories supported by data. The Wall Of Numbers approach tells much, but reveals little.

One way to fix this is to Use Math to condense all of this information into just one, easy-to-understand number. Although this approach has been popular, we think it’s a huge mistake. We are not in the business of assigning relative values to different metrics; the whole point of altmetrics is that depending on the story you’re interested in, they’re all valuable.

So we (and from what they tell us, our users) just want to make those stories more obvious—to connect the metrics with the story they tell. To do that,  we suggest categorizing metrics along two axis: engagement type and audience. This gives us a handy little table:

Now we can make way more sense of the metrics we’re seeing. “I’m being discussed by the public” means a lot more than “I seem to have many blogs, some twitter, and ton of Facebook likes.” We can still show all the data (yay!) in each cell—but we can also present context that gives it meaning.

Of course, that context is always going to involve an element of subjectivity. I’m sure some people will disagree about elements of this table. We categorized tweets as public, but some tweets are certainly from scholars. Sometimes scholars download html, and sometimes the public downloads PDFs.

Those are good points, and there are plenty more. We’re excited to hear them, and we’re excited to modify this based on user feedback. But we’re also excited about the power of this framework to help people understand and engage with metrics. We think it’ll be essential as we grow altmetrics from a source of numbers into a source of data-supported stories that inform real decisions.

2 thoughts on “A new framework for altmetrics

  1. Hey. my dashboard doesn’t look anything like that. How come?

    I think your breakdown has a few problems (scholars don’t view html? Or use twitter?). You’re better off breaking it into cited (citations, blogs), saved (mendeley, bookmarking sites), viewed (downloads, html views), and social (tweets, facebook, maybe press releases).

    Incidentally, what I’d like to be able to do is to order everything in data order and visualise (over time, by category.

    • Hi Tom!

      Your profile doesn’t look like this because this is an old view into the metrics. We’ve since changed the layout to tell stories about the data up front, resulting in our “Highly viewed by scholars” and other badges that appear on profiles’ front pages, and including the numerical data in hover pop-ups as well as the drill down view.

      Your thoughts on how to organize the metrics are good ones. Our friends at Plum Analytics actually take this strategy, aggregating all cites, views, etc (separated by service) and differentiating them from social media, like you suggest. The rubric that Jason & Heather propose will likely evolve over time, as new altmetrics research can shed light on our guess that scholars do download PDFs more than the public views HTML, etc.

      As for your final suggestion, I like it! We’ve previously had someone suggest other sorting options be added to profiles (http://feedback.impactstory.org/forums/166950-general/suggestions/5811444-add-sorting-options-to-profile); if you’re so inclined, add your thought to this ticket (or create a new one) and vote it up!

Leave a Reply