Discussions on the art and craft of research

Tag: evaluating resources

Research and the Information Process

What is the “information creation process,” and what does it have to do with scholarly research? Short answer: a lot.

Longer answer (if you’re still with me!): In a previous post, I wrote about the first of the threshold concepts developed by the Association of College & Research Libraries’ (ACRL) in its “Framework for Information Literacy for Higher Education.” Threshold concepts are key points that an information-literate student or researcher needs to be able to grasp and utilize. The ACRL’s first threshold concept, “authority is constructed and contextual,” seemed tailor made for a political moment rife with discussions and anxieties about fake news, post truth and alternative facts, and indeed we librarians engaged in several robust discussions with faculty about how to approach this topic in the curriculum.

At first glance, the second threshold concept, “Information creation as a process,” might not resonate quite as strongly. But perhaps it should. Because an understanding of the process of how information is created is necessary for discerning how various books, articles, etc., might be useful or not useful — or for figuring out how authoritative they might be.

First, what does that mean, “information creation as a process”? Information objects — resources that are in various containers, including books, newspaper articles, scholarly journal articles, magazine articles, websites, blog posts, tweets, scholarly proceedings and yes, Facebook posts of dubious origin — all are informed by, and in turn inform, other information objects. In other words, the information contained in these various objects is used to create other objects; in turn, as these objects are read and shared, they may themselves serve to generate tweets, news articles, scholarly articles and books. The process is not merely circular, but infinitely weblike in the way that new information shapes, and is shaped by, already existing sources.

What’s more, the kinds of information objects that are available on any given topic depend on several factors: how long the topic has been in discussion, the extent of the discourse surrounding that topic, and the information objects that have already been created on that topic.

Let’s take, for example a story that’s very much been in the news: Donald Trump’s controversial executive order on immigration from Muslim-majority countries. Although the news of the order hit newspaper websites very quickly following the announcement, even before those stories appeared there were innumerable tweets, Facebook posts and other social media commentary offering even quicker takes.

How useful are these quick takes on the issue? Certainly, they serve as a record of the issue’s explosiveness — its vast potential for altering domestic and global social affairs, politics and business. If one’s project is to document the proliferation of information objects about the issue, then gathering the tweets and quick news pieces would be essential. Similarly, a researcher who sought to document the speculation about the effects of the travel bans would need to look at these early sources.

But the usefulness of any particular information source depends very much on the nature of the project at hand — and that source’s place in the information cycle. What if, instead of looking at the speculation about the executive order, one’s project was about examining its effects? For that, one needs something more analytical — one that views the immigration orders at a distance. One might first look at any newspaper articles that appeared since Trump’s announcement — but again, depending on when the research was being conducted (six months after an event? a year? five years?), more detailed, rigorously conducted scholarly sources might be available.

Let’s look at how this plays out in various searches. Searching Google for “Donald Trump executive order immigration” in April — roughly three months after the initial announcement — yielded nearly 3 million results. Closer to home, entering the same keywords into the library’s CrossSearch tool (which searches books as well as numerous article databases) predictably yields fewer results. (Still quite a few at more than 2,000, but not quite the 3 million that Google unearths.)

Screen Shot 2017-04-19 at 11.35.12 AM

Of these results, most are news articles:

Screen Shot 2017-04-19 at 11.35.41 AM

Only a handful appeared in academic journals, and most of these are quick takes of only a page or two, certainly not the detailed, rigorous, analytical pieces one might expect to find in a scholarly publication. To find such an article, it would be necessary to wait until more time had passed — until scholars had been able to perform studies that involved gathering and interpreting survey data, and that carefully surveyed the available literature on the subject.

Screen Shot 2017-04-19 at 11.36.11 AM

Note, too, that the search doesn’t yield any books at all. That’s because books take even more time than scholarly journal articles to put together. Books need to be proposed, written, edited, rewritten and then published before they can arrive on the scene. The advantage of books, whether they appear in print or electronic form, is that they typically represent some of the most considered, most rigorous thinking on a particular topic. But they take time, and usually depend on the fact that previous news articles, scholarly articles and other materials have already appeared on the topic.

And so a hypothetical researcher seeking to examine the effects of immigration bans would certainly want to look at detailed scholarly journals and books, if they were available. But to determine the kind of questions that are even possible, it’s necessary to understand what materials might be in the information-creation process, and thus how helpful or authoritative they might be in answering the question one poses.

So when you ask a research question, it’s a good idea to think about where a topic might be in the overall information-creation process. The answers will help not only in guiding you to the best possible sources to answer your question, but also in figuring out what might be available in the first place.

— Fred Folmer

Authority: Who Needs It?

(Note: This is the first in a series of planned blog posts exploring the key concepts of the Association of College and Research Libraries’ [ACRL] Framework for Information Literacy.)

In 2016 ACRL — the national organization for academic librarians — finalized its Framework for Information Literacy, a document that had been a number of years in the making. Simply put, the Framework seeks to define and describe six key points, called threshold concepts, that students seeking to become information literate need to understand.

I thought it might be useful in our blog to explore, point by point, the six key concepts of the framework; part of why I thought so is that the first point, “authority is constructed and contextual,” seems to be such an urgent topic to discuss, given the social and political moment in which we find ourselves.

As readers of this blog may recall, last fall I discussed one of the hot buttons generated by the divisive 2016 presidential election: the issue of media literacy or, if you will, fake news. Put succinctly, the fake news crisis can be thought of as a crisis of authority. In what news, media or other information sources can we trust? If people don’t take the time to discern which sources are trustworthy (in other words, authoritative), what can or will they be led to believe, and what does that do to the state of education, research and democracy itself?

The framework — advancing a view of authority that is constructed and contextual — states: “Authority is constructed in that various communities may recognize different types of authority.” It goes on to recognize the need for acknowledging biases “that privilege some sources of authority over others, especially in terms of others’ worldviews, gender, sexual orientation or cultural orientation.” The traditionally dominant voices of authority, in other words, need to be tempered or even overruled by the inclusion of voices that have been long shut out of the conversation. This, to me, is necessary, and frankly unassailable. Authority must be queried and questioned; researchers and students need to pay attention to the social processes by which authority is constructed, and be ready to challenge those processes.

But at the same time, to what extent does the assigning of context to authority mean that facts themselves can be overruled by asserting an alternative authority that claims to have, as has been famously done recently, “alternative facts”? Are facts — which, after all, are important products of authority — themselves constructed and contextual?

One may be tempted to say no — that a fact is a fact, and we’re not entitled to our own facts in the way we are entitled to opinions. While this isn’t exactly wrong, it’s also true that some facts, such as scientific findings, can be revised as new information comes to light. It’s also true that what counts as a fact can vary from discipline to discipline. So do these qualifications leave us in a world without facts — and, some might say, without moorings? Are all researchers thus in a “post-truth” quandary, between a proverbial rock and hard place?

The way through, I think, is to recognize that there simply isn’t anything like a one-size-fits-all edict that provides all the answers to the issue. Instead, researchers need to adopt a set of practices that help to evaluate materials and claims, and to think of this as a process, rather than as a quick judgment or a foregone conclusion. I offer a few suggestions here; some of these are partially based on some of the recommendations that ACRL provides in its “Dispositions” section of the framework.

First, query everything, constantly asking questions about the information’s provenance, its reason for being, its date of creation and its own sources of information. Remember that facts can sometimes be disproved, and that the questioning of established truth is part of a healthy research community (and, by extension, democracy).

Second, place different sources into relation to one another. This is what journalists do when they determine what should go into a story: Is there a second, third or fourth source that corroborates what this first source is arguing? To what does the preponderance of evidence lead? If multiple sources are pointing in the direction of a fact, it’s more likely to be true; but even then, since such things can themselves be the product of groupthink or a particular way of constructing authority, they often need to be qualified or tempered in their description.

The overarching recommendation of the ACRL framework is, as it states, to “develop and maintain an open mind” when thinking about authority: to recognize that authority is constructed socially, and can be made and unmade; that it may be constructed or interpreted differently in various contexts; and that established authority may be related to power.

These points won’t necessarily solve the conundrum about how to know when something is truly, incontrovertibly, absolutely a fact — particularly in an age when information sources claiming to be authoritative are thrown at us in ever greater volume, and at ever greater speed. But they may serve as good points to keep in mind when one is trying to decide how to think about a given claim as authoritative or not.

— Fred Folmer



On Fake News and Research Skills

In light of the emergence of fake news as one of the key stories following the 2016 presidential election, it’s worth (re-)considering the importance of evaluating information to any research process—whether that process involves writing a paper or gathering information about a candidate for office.

Although developing evaluation skills has always been integral to any research process, it’s arguably even more urgently needed now. That’s because libraries are no longer the sole gatekeepers of information, and it’s now possible to simply do a quick search on the web, find something that appears to relate to the topic at hand, and either forward  to someone else, or incorporate it into a paper or other piece of research.

As has been widely reported, a great deal of the fake news now circulates on social media networks. In this New York Times op-ed written by Zeynep Tufekci, a professor of library and information science at the University of North Carolina, the author takes Facebook to task for becoming a platform for misinformation campaigns (the pope endorses Donald Trump! An FBI agent who leaked Hillary Clinton’s emails found dead!).

Part of the problem, Tufekci argues, is Facebook’s algorithmic system, which promotes updates based on whether users find them “comforting.” But research isn’t supposed to be comforting; neither, correspondingly, is the moral and ethical work of citizenship. And helping students learn the moral and ethical work of citizenship is—or should be—in large part why we teach research skills on a college campus.

There have been signs that Facebook is taking steps to limit the fake news stories that are shared on its servers, but researchers—that is, those doing a paper or those simply gathering information to make an informed choice on an election—need to ask themselves a set of questions about every source they’re using, no matter how much the source may support one’s thesis or existing worldview, and no matter how much that source has been useful in the past.

First, who is responsible for the piece? A name isn’t enough; one needs to ask about the author’s credentials or authority to have written something on a particular topic. If it’s a news story, does it come from a reputable service—one that checks its facts, verifies its sources and provides multiple perspectives? Some of the fake Facebook posts came from the “Denver Guardian,” which sounds great until one realizes that no such news source exists. (Go ahead, Google it.)

Second, when was the piece written? In this election season, I saw articles forwarded and shared on social media that had been created months and even years earlier, making it seem as though they had just appeared. But facts and situations can change quickly, and in many research or fact-finding situations, it’s important to have current information, or at least to be aware of when an article appeared so that its date of creation can factor into one’s judgment about it.

Why was the piece written? To report the news, or to advance knowledge in a particular field? To get someone elected to an office? To spread fear, or to propagandize an issue? To make money? This question is often entangled with who wrote the story, but it’s equally important. (To think about the ways in which who wrote a piece can be bound up with why he or she wrote it, I suggest checking out this self-exculpatory New York Times op-ed written by someone who works for WikiLeaks.)

How and where did the author(s) get their information? In scholarly writing, this is precisely why citations must be provided—so that authors cannot simply assert something without some kind of backup. We need to be able to believe what authors are saying; it’s equally important to be able to verify their sources.

I’ve been trying to share the above questions with the first-year seminars with whom I’ve worked this past semester. We’ve looked at sources we found on the web and tried to think about evaluating them based on the above questions, rather than applying such abstract, blanket maxims such as “sites that come from a .edu or .org address are okay.” That’s not necessarily true; it’s always necessary to look closer at each article or book.

One of the first-year seminars I worked with was entitled “Performing Citizenship.” It was striking to me that the course focus and our work with evaluating sources were in particular alignment—and, similarly, that the task of critically evaluating research information and that of truly becoming an informed, participating citizen are one and the same. Whenever we undertake or assign research—and learn or teach the requisite skills to perform this research—we would do well to keep the responsibilities and imperatives of citizenship in full view.

— Fred Folmer

© 2017 ResearchScapes

Theme by Anders NorenUp ↑