Perspectives on the evidence, value and impact of LIS research: conceptual challenges presented by Andrew Dillon
Andrew Dillon began his keynote presentation Perspectives on the evidence, value and impact of LIS research: conceptual challenges, by outlining the major background shifts in the ecology of information. He emphasised the need to separate two types of research: (1) looking at the technology of organising and presenting and (2) studying the ways that humans deal with information. In doing so it is more likely that we shape technology so it serves people better.
He noted the historically acknowledged importance of keeping and managing information by referring to Newton. Newton stood on the shoulders of giants, but he was only able to do so because those giants left a carefully curated record from which he could work. This inevitably shaped his thinking and what he was able to achieve. However, only now are we able to judge the impact and value of those records: at the time it would have been impossible to predict their potential impact and value accurately.
In analysing the shifts that are taking place in the information space, Dillon observed that there is no discussion of IT design to reflect the changing demographics of the population. He took us through some of the statistics about Internet usage and discussed how the population is learning to access, use and interpret information delivered via the Internet. In particular, he noted that 99.99% of all new data is created digitally, and that of the 40 million US Internet users who claim that this is their primary source of scientific information, 80% check the veracity of information accessed. In order to do this, they must have learnt a new set of information-related skills.
Having used IT as an example, Dillon went on to emphasise that the problem is not owned by the computer science discipline, or by any other discipline. He made the point that disciplines do not live forever. However, questions and issues have a much greater longevity. Professions shift – inelegantly – according to cultural and social forces. As the sub-disciplines of information shift, problems with communication between these sub-disciplines arise and there is boundary confusion. He identified the boundaries between LIS, Social Informatics, Information Science/Studies/Technology, Instructional/Educational Technology and Information Architecture/Policy/Management, noting that the boundaries themselves present questions about credentials and jurisdiction.
Dillon then took us through a quick 15,000 year history of information to remind the audience that information is part of the human condition: we have been scratching utterances on walls for thousands of years. He also elaborated on his earlier point about Newton, using the example of Gutenberg. Explaining Gutenberg’s impact took centuries, yet when it comes to demonstrating value and impact, we hope (or are expected) to explain the contribution of the Internet now. He noted that civilisation shifts as information explodes, and we are at a moment of profound change in the ecology of information. It will take time to fully understand and appreciate this in terms of long term impacts.
Dillon then moved on to highlight a success story for LIS research: information retrieval. LIS research informed computer scientists that information retrieval is not just a technical issue, but also a human issue. It is connected with how humans search for and use information, and this has helped to inform the design of digital information retrieval systems.
However, Dillon pointed out that our obsession with information retrieval places too little emphasis on longitudinal outcomes. We need to move beyond the instant and study the longer tale of information use. He explained that the process of adjustment to new technology is extremely important. Unfortunately usability studies tend only to look at the instant response, not at how the information is then used and interpreted, or how the human interacting with the new technology adapts to it over time.
Dillon also noted that how people share information is again under-appreciated and is very difficult to study. He illustrated this by citing research that examined folksonomies. In short, this demonstrated that community tagging of items of information can create a coherent system.
Dillon’s current concerns include the emergence of a new literacy that emphasises search over comprehension, and leads to a loss of “deep” reading skills. We have been very successful in building link-based systems, so it is inevitable that people will be reading in this way. In itself this is not a bad thing. However, an implication of this is that we still need to design systems that make it possible for people to be able learn, and this must take into account human behaviour. If we only study the technology, we do so at expense of humans. There needs to be an understanding of how people use information, which is considered seriously when information systems are designed.
Dillon highlighted studies that show people want information to be comprehensible, believable and timely. It is interesting that when asked about what they require, and the value in the information that they need to use, people never mention the technology that delivers that information.
Dillion also discussed a study of young, tech-savvy people to examine their biases related to the credibility of information source. The study showed that the participants felt information was more credible from Britannica than Wikipedia (in tests that dressed up text to appear to come from one source or the other), despite the fact that the difference in error margin between the sources is not great. The study shows how we inherit judgements about information, and that information is judged as part of long-term learning.
In conclusion, Dillon said that LIS research should be looking at how to build information systems that reflect the way people think and use information. Demonstrating impact and value is about identifying the human rules, remembering that technology is simply an enabler. He remarked that data is stored; information is experienced.
Dillon then turned to the problem of using hard data to demonstrate the value of LIS. For example, he expressed surprise that many people in this field can’t produce hard data that show that reading matters. He cited a study that achieves this. It showed that the single strongest factor in predicting a child’s success at college is whether or not the child grows up in a house with books. He challenged us to think of other studies we could point to as evidence of our value.
Dillon acknowledged that a lot of what we do is qualitative work, so it is hard to meta-analyse to judge impact, but we are not alone in that. He concluded by describing ours as a discovery discipline, which is art and science, politics and economics, research and teaching. It is a social contract with our future.
To find out more, and to see Dillon’s statistics and references, please scroll through the slides from his keynote paper.