A big trend in libraries everywhere is data. Usage data, collection data, user survey data, the list goes on and I’m sure you all can name several more types of data off the top of your heads. Using data to measure the human experience is something that those of us interested in the digital humanities grapple with and theorize about to no end. But what about using data to literally measure human activity, their wants and needs? This can be tricky to say the list, but I would like to share some thoughts on this topic today and I hope to spark some discussion.

First, I think that it is concerning that libraries, especially academic libraries are increasingly forced to demonstrate value in order to ensure their continued existence (Soria, Fransen, and Nackerun, 2013, p. 148). Second, I think that if academic libraries must demonstrate the value they generate, then student success is the best kind of metric to measure it. In this post, I will discuss the benefits and the concerns of collecting data in libraries in order to demonstrate value, and the promise and pitfalls that come with using student success as a metric.

I want to stress that I do not think that libraries should not be accountable to their stakeholders and not bother with assessment designed specifically to improve services to their community. Assessment for improving services is possible, and student success is a great metric, “yet we understand that our assessment work is often deeply problematic in its alignment with a neoliberal culture of accountability and consumerism in higher education” (Magnus, Belanger, and Faber, 2018). The question that I struggle with is if this adherence to a neoliberal approach to service and the dominance of profit motive, etc., may not be in the best interest of libraries, of universities, or of students. Another issue I find with using “big data” for these purposes is inherently problematic due to the trouble of consequentialism and scientism that it can engender (Johnson, 2014, p.5-8). To paraphrase Johnson, the former suggests that consequences from implementing policy that makes potentially harmful assumptions about correlations while the latter erases human concerns in favor of raw, data-driven efficiency. From Johnson’s four key differences of data mining from other forms of research: “Data mining aims strictly at identifying previously unseen data relationships rather than ascribing causality to variables in those relationships” (p. 4). While it is great that data miners acknowledge that data mining is not ideal for finding causation, but Johnson does bring up some examples of problematic policies enacted based on the insights of data analyses where no causation existed, but were assumed anyway, causing policies that ultimately go against the needs of people. For example, No Child Left Behind which was borne of a fetishization of the scientific process to the point of erasing local contexts and various cultural differences (2014). While big data and data mining pose major ethical concerns and inefficiencies, I believe data mining can serve a useful purpose if properly applied and its limitations for directing policy acknowledged.

Cox and Jantti (2012) present an intriguing way of using large datasets about library usage with demographic and GPA average. It is not terribly clear what they did with this demographic data, and they relied more on GPA and correlated GPA with the use of online library resource use. They do well to note that correlation does not prove correlation, and that there are many other factors involved in academic success beyond checking out books or accessing a database (p. 4). Deploying demographic data could help us better understand other factors like whether students are first-generation students or not, and how that impacts their student experience (though, this does pose other ethical concerns such as privacy, assuming things about other cultures and so on, but that’s beyond the scope of this short essay). In a similar study focused on high school students and standardized tests, we again see a correlation between library use and student success. At least, success at standardized tests. The interesting aspect in this study is its inclusion of survey data of various educator groups and administrators to see if they agreed that library use was a contributing factor in academic success, and there was a strong consensus there. (Lance, Schwarz, Rodney, 2014, p. 50) So, though using standardized testing as a metric of student success is a bit problematic (as with using GPA as the sole metric, as above), the added benefit here is having that other source of data in order to further corroborate,  humanize, and contextualize the more quantitative findings.

To wrap up this brief consideration of data-driven assessment and student success as a metric, I want to reiterate and tie a few things together. I believe that a consumer-model and corporate-style approach to education has massive issues. Further, an approach to understanding student success through the collection and mining of big data carries potentially dangerous consequences. The big problem with both this consumer satisfaction approach to assessment and the big data approach is that they both erase local concerns and enforce hegemonic, neoliberal values. This approach places profit and faceless data-driven “efficient” solutions above solutions that would actually improve learning outcomes, even though these are notoriously difficult to measure quantitatively (Magnus, Belanger, and Faber, 2018). After all, we are trying to measure student success, not simply how satisfied they are with the customer service of their educators. Using large amounts of hard data can be a good way to establish baselines and raise interesting questions, but how best to improve student success, I think, is not really measured well by correlating GPA, standardized scores, or even retention with library usage (Greater Western Library Alliance, 2017 p. 10). While the correlations are useful, putting data in the driver seat of policy decisions can have disastrous effects, paving over individual concerns. Case in point: Arizona State University, after examining student success metrics, they began, “compelling students making insufficient academic progress to change their major [which] is very much coercive, explicitly denying students to opportunity to exercise their own agency” (Johnson, 2014, p. 6).

References

Cox, B. & Jantti, M. (2012). Discovering the impact of library use and student performance. Educause Review, 18, 1-9.

Greater Western Library Alliance (GWLA). (2017). The impact of information literacy instruction on student success: A multi-institutional investigation and analysis. Retrieved from https://www.arl.org/storage/documents/publications/The_Impact_of_Information_Literac y_Instruction_on_Student_Success_October_2017.pdf

Johnson, J. A. (2014). The ethics of big data in higher education. International Review of Information Ethics, 7(1), 3-10.

Lance, K. C., Schwarz, B., & Rodney, M. J. (2014). How libraries transform schools by contributing to student success: evidence linking South Carolina school libraries and PASS & HSAP results. South Carolina Association of School Librarians.

Lance, K. C., Schwarz, B., & Rodney, M. J. (2014). How libraries transform schools by contributing to student success: evidence linking South Carolina school libraries and PASS & HSAP results. South Carolina Association of School Librarians.

Soria, K. M., Fransen, J., & Nackerud, S. (2013). Library use and undergraduate student outcomes: New evidence for students’ retention and academic success. portal: Libraries and the Academy, 13(2), 147-164.

Image credit: ISTOCKPHOTO.COM/DRAFTER123