Unexpectedly, having a computer science undergrad occasionally bites me in the rear.
One of my projects this semester is to do some bibliographic analysis. I love being able to understand the connections between things, and bibliometrics offers me a way to explore how different authors relate to each other, how works relate to each other, how things change over time in the academic publishing world. It’s a great time.
There are a lot of data sources available for doing this kind of analysis. My favourite so far (in my limited experience) is OpenAlex. I like their licensing, I like their interface, I like their ethos. But like many of these online databases, the web interface and the API don’t always offer the level of expressiveness I would prefer. I’m a student, and the goal of this assignment isn’t to do a super detailed complex analysis of a difficult bibliometric problem. But I have a question that I want to answer.
I hypothesize that there’s a positive correlation between author impact factors (e.g. h-index) and the number of different authors they collaborate with (via co-authorship). The more people you work with, the broader your understanding of a topic and its applications, the more areas of science your work touches, and thus the higher impact you have as an author.
Bibliometrics databases generally have authors and their publications as part of their data, along with citation information for those publications. Unfortunately, the number of distinct collaborators every author in the dataset has isn’t something these tools are able to easily query. Same for non-distinct (ie counting collaborations between two people based on the number of publications they share). A reasonable student wanting to get this assignment finished would simply pick a different research topic.
I am, unfortunately, not a reasonable student. And that brings us back to OpenAlex, which offers a free download of their entire 1.7 terabyte bibliometric dataset. And I happen to have 1.7 terabytes of disk space free on my computer, and the software skills to process that data myself to answer my question.
I don’t recommend this path for other students. I am not going to receive any support from the professor if I run into trouble. The statistics for correlating the variables is an entirely additional effort after the queries are run. But it also isn’t the kind of question I get to answer every day, it will provide me with a much more thorough understanding of the data involved, and it will stand out among the projects undertaken by my peers.
The dataset is going to take a full day to download, leaving me plenty of time to regret my choices (and blog about them). I’ll share my results (or failure) in the future.