En-garde: Source Evaluations in the Digital Age
DOI:
https://doi.org/10.14738/assrj.99.13066Keywords:
epistemic cognition, source evaluation, digital literacy, mixed methods researchAbstract
Students have difficulty assessing the quality of information. They often rely on content-focused criteria to make reliability assessments and, as a result, may accept inaccurate information. Despite the impact of poor source evaluation skills, educational researchers have not widely examined source evaluation behaviours in authentic environments or tasks. Students’ epistemic cognition, or their thinking about the epistemic properties of specific knowledge claims and sources, is one promising avenue to better understand their source evaluation behaviours. Two studies were conducted to explore students’ epistemic thinking. In Study 1, college students (n = 12) reported their reliability criteria in focus group interviews. Four of these participants (n = 4) also examined the reliability of an online news article. Grounded theory was used to infer students’ epistemic ideals and reliable epistemic processes. In Study 2, students (n = 43) rank-ordered two news articles and justified how they assigned each article’s rank in a written response. Most students were able to accurately rank-order the articles using relevant epistemic processes. Cluster analysis was used to characterize the evaluation criteria used. Surprisingly, more participants who justified their decisions using relevance criteria accurately rank-ordered the articles. The role of direct and indirect indicators of reliability are discussed through the lens of the Apt-AIR framework of epistemic thinking.
References
Aribarg, A., & Schwartz, E. M. (2020). Native advertising in online news: Trade-offs among clicks, brand recognition, and website trustworthiness. Journal of Marketing Research, 57(1), 20–34. https://doi.org/10.1177/0022243719879711
Barnes, M. D., Penrod, C., Neiger, B. L., Merrill, R. M., Thackeray, R., Eggett, D. L., & Thomas, E. (2003). Measuring the relevance of evaluation criteria among health information seekers on the Internet. Journal of Health Psychology, 8(1), 71–82. https://doi.org/10.1177/1359105303008001436
Barzilai, S., & Chinn, C. A. (2018). On the goals of epistemic education: Promoting apt epistemic performance. Journal of the Learning Sciences, 27(3), 353–389. https://doi.org/10.1080/10508406.2017.1392968
Barzilai, S., & Eshet-Alkalai, Y. (2015). The role of epistemic perspectives in comprehension of multiple author viewpoints. Learning and Instruction, 36, 86–103.
Barzilai, S., Mor-Hagani, S., Zohar, A. R., Shlomi-Elooz, T., & Ben-Yishai, R. (2020). Making sources visible: Promoting multiple document literacy with digital epistemic scaffolds. Computers & Education, 157, 103980. https://doi.org/10.1016/j.compedu.2020.103980
Barzilai, S., & Zohar, A. (2012). Epistemic thinking in action: Evaluating and integrating online sources. Cognition and Instruction, 30(1), 39–85. https://doi.org/10.1080/07370008.2011.636495
Barzilai, S., & Zohar, A. (2014). Reconsidering personal epistemology as metacognition: A multifaceted approach to the analysis of epistemic thinking. Educational Psychologist, 49(1), 13–35. https://doi.org/10.1080/00461520.2013.863265
Barzilai, S., & Zohar, A. (2016). Epistemic (meta) cognition: Ways of thinking about knowledge and knowing. In J. A. Greene, W. A. Sandoval, & I. Bråten (Eds.), Handbook of epistemic cognition (pp. 409–424). Routledge.
Braasch, J. L. G., Bråten, I., Strømsø, H. I., Anmarkrud, Ø., & Ferguson, L. E. (2013). Promoting secondary school students’ evaluation of source features of multiple documents. Contemporary Educational Psychology, 38(3), 180–195. https://doi.org/10.1016/j.cedpsych.2013.03.003
Bråten, I., Strømsø, H. I., & Andreassen, R. (2016). Sourcing in professional education: Do text factors make any difference? Reading and Writing, 29(8), 1599–1628.
Bråten, I., Strømsø, H. I., & Britt, M. A. (2009). Trust matters: Examining the role of source evaluation in students’ construction of meaning within and across multiple texts. Reading Research Quarterly, 44(1), 6–28.
Breakstone, J., McGrew, S., Smith, M., Ortega, T., & Wineburg, S. (2018). Why we need a new approach to teaching digital literacy. Phi Delta Kappan, 99(6), 27–32.
Britt, M. A., & Aglinskas, C. (2002). Improving students’ ability to identify and use source information. Cognition and Instruction, 20(4), 485–522.
Chinn, C. A., Buckland, L. A., & Samarapungavan, A. (2011). Expanding the dimensions of epistemic cognition: Arguments from philosophy and psychology. Educational Psychologist, 46(3), 141–167. https://doi.org/10.1080/00461520.2011.587722
Chinn, C. A., & Rinehart, R. W. (2016). Epistemic cognition and philosophy: Developing a new framework for epistemic cognition. In J. A. Greene, W. A. Sandoval, & I. Bråten (Eds.), Handbook of epistemic cognition (pp. 460–478). Routledge.
Chinn, C. A., Rinehart, R. W., & Buckland, L. A. (2014). Epistemic cognition and evaluating information: Applying the AIR model of epistemic cognition. In D. N. Rapp & J. L.G. Braasch (Eds.), Processing inaccurate information: Theoretical and applied perspectives from cognitive science and the educational sciences (pp. 425–453). MIT Press.
Chinn, C., & Sandoval, W. A. (2018). Epistemic cognition and epistemic development. In F. Fischer, C. E. Hmelo-Silver, S. R. Goldman, & P. Resiman (Eds.), International handbook of the learning sciences (pp. 46–55). Routledge.
Cho, B.-Y., Woodward, L., & Li, D. (2018). Epistemic processing when adolescents read online: A verbal protocol analysis of more and less successful online readers. Reading Research Quarterly, 53(2), 197–221.
Cresswell, J. W., & Plano Clark, V. L. (2011). Designing and conducting mixed methods research (2nd ed.). Sage Publications.
Cresswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). Sage Publications.
Cunningham, A., & Johnson, F. (2016). Exploring trust in online health information: A study of user experiences of patients.co.uk. Health Information & Libraries Journal, 33(4), 323–328. https://doi.org/10.1111/hir.12163
Elby, A., & Hammer, D. (2010). Epistemological resources and framing: A cognitive framework for helping teachers interpret and respond to their students’ epistemologies. I In L. D. Bendixen & F. C. Feucht (Eds.), Personal epistemology in the classroom: Theory, research, and implications for practice (Vol. 4, pp. 409–434). Cambridge University Press.
Eshet-Alkalai, Y., & Chajut, E. (2009). Changes over time in digital literacy. CyberPsychology & Behavior, 12(6), 713–715.
Eysenbach, G. (2002). How do consumers search for and appraise health information on the world wide web? Qualitative study using focus groups, usability tests, and in-depth interviews. BMJ, 324(7337), 573–577. https://doi.org/10.1136/bmj.324.7337.573
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive–developmental inquiry. American Psychologist, 34(10), 906.
Flavell, J. H., Miller, P. H., & Miller, S. A. (2002). Cognitive development (4th ed., Vol. 338). Prentice-Hall Englewood Cliffs, NJ.
Gerjets, P., Kammerer, Y., & Werner, B. (2011). Measuring spontaneous and instructed evaluation processes during Web search: Integrating concurrent thinking-aloud protocols and eye-tracking data. Learning and Instruction, 21(2), 220–231. https://doi.org/10.1016/j.learninstruc.2010.02.005
Glaser, B. G. (1978). Theoretical sensitivity: Advances in the methodology of grounded theory. The Sociology Press.
Glaser, B. G., & Strauss, A. L. (2017). Discovery of grounded theory: Strategies for qualitative research. Routledge.
Greene, J. A. (2016). Interacting epistemic systems within and beyond the classroom. In J. A. Greene, W. A. Sandoval, & I. Bråten (Eds.), Handbook of epistemic cognition (pp. 265–278). Routledge.
Greene, J. A., Copeland, D. Z., Deekens, V. M., & Seung, B. Y. (2018). Beyond knowledge: Examining digital literacy’s role in the acquisition of understanding in science. Computers & Education, 117, 141–159.
Greene, J. A., & Yu, S. B. (2016). Educating critical thinkers: The role of epistemic cognition. Policy Insights from the Behavioral and Brain Sciences, 3(1), 45–53. https://doi.org/10.1177/2372732215622223
Greene, J. A., Yu, S. B., & Copeland, D. Z. (2014). Measuring critical components of digital literacy and their relationships with learning. Computers & Education, 76, 55–69. https://doi.org/10.1016/j.compedu.2014.03.008
Halverson, K. L., Siegel, M. A., & Freyermuth, S. K. (2010). Non-science majors’ critical evaluation of websites in a biotechnology course. Journal of Science Education and Technology, 19(6), 612–620. https://doi.org/10.1007/s10956-010-9227-6
Hammer, D., & Elby, A. (2002). On the form of a personal epistemology. In B. K. Hofer and P. R. Pintrich (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing (pp. 169–190). Lawrence Erlbaum Associates Publishers.
Hart, W., Albarracín, D., Eagly, A. H., Brechan, I., Lindberg, M. J., & Merrill, L. (2009). Feeling validated versus being correct: A meta-analysis of selective exposure to information. Psychological Bulletin, 135(4), 555.
Hofer, B. K. (2016). Epistemic cognition as a psychological construct: Advancements and challenges. In J. A. Greene, W. A. Sandoval, & I. Bråten (Eds.), Handbook of epistemic cognition (pp. 31–50). Routledge.
Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Researcher, 33(7), 14–26. https://doi.org/10.3102/0013189X033007014
Jung, E. H., Walsh-Childers, K., & Kim, H.-S. (2016). Factors influencing the perceived credibility of diet-nutrition information web sites. Computers in Human Behavior, 58, 37–47. https://doi.org/10.1016/j.chb.2015.11.044
Jung, W. S., Chung, M.-Y., & Rhee, E. S. (2018). The effects of attractiveness and source expertise on online health sites. Health Communication, 33(8), 962–971. https://doi.org/10.1080/10410236.2017.1323364
Kąkol, M., Nielek, R., & Wierzbicki, A. (2017). Understanding and predicting Web content credibility using the Content Credibility Corpus. Information Processing & Management, 53(5), 1043–1061. https://doi.org/10.1016/j.ipm.2017.04.003
Kiili, C., Laurinen, L., & Marttunen, M. (2008). Students evaluating internet sources: From versatile evaluators to uncritical readers. Journal of Educational Computing Research, 39(1), 75–95. https://doi.org/10.2190/EC.39.1.e
Krouwer, S., Poels, K., & Paulussen, S. (2020). Moving towards transparency for native advertisements on news websites: A test of more detailed disclosures. International Journal of Advertising, 39(1), 51–73. https://doi.org/10.1080/02650487.2019.1575107
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159. https://doi.org/10.2307/2529310
Lucassen, T., & Schraagen, J. M. (2013). The influence of source cues and topic familiarity on credibility evaluation. Computers in Human Behavior, 29(4), 1387–1392. https://doi.org/10.1016/j.chb.2013.01.036
Luttrell, W. (2010). Qualitative educational research: Readings in reflexive methodology and transformative practice. Routledge.
Machackova, H., & Smahel, D. (2018). The perceived importance of credibility cues for the assessment of the trustworthiness of online information by visitors of health-related websites: The role of individual factors. Telematics and Informatics, 35(5), 1534–1541. https://doi.org/10.1016/j.tele.2018.03.021
Mason, L., Ariasi, N., & Boldrin, A. (2011). Epistemic beliefs in action: Spontaneous reflections about knowledge and knowing during online information searching and their influence on learning. Learning and Instruction, 21(1), 137–151. https://doi.org/10.1016/j.learninstruc.2010.01.001
Mason, L., Boldrin, A., & Ariasi, N. (2010). Searching the Web to learn about a controversial topic: Are students epistemically active? Instructional Science, 38(6), 607–633.
Mason, L., Junyent, A. A., & Tornatora, M. C. (2014). Epistemic evaluation and comprehension of web-source information on controversial science-related topics: Effects of a short-term instructional intervention. Computers & Education, 76, 143–157. https://doi.org/10.1016/j.compedu.2014.03.016
Mason, L., Scrimin, S., Tornatora, M. C., Suitner, C., & Moè, A. (2018). Internet source evaluation: The role of implicit associations and psychophysiological self-regulation. Computers & Education, 119, 59–75. https://doi.org/10.1016/j.compedu.2017.12.009
Morgan, D. L. (1988). Focus groups as qualitative research (Vol. 16). Sage Publications.
Morse, J. M. (2003). Principles of mixed methods and multimethod research design. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of mixed methods in social and behavioral research (pp. 189–208). Sage Publications.
Muis, K. R., Pekrun, R., Sinatra, G. M., Azevedo, R., Trevors, G., Meier, E., & Heddy, B. C. (2015). The curious case of climate change: Testing a theoretical model of epistemic beliefs, epistemic emotions, and complex learning. Learning and Instruction, 39, 168–183.
Nunkoosing, K. (2005). The problems with interviews. Qualitative Health Research, 15(5), 698–706.
Press, S. J., & Wilson, S. (1978). Choosing between logistic regression and discriminant analysis. Journal of the American Statistical Association, 73(364), 699–705.
Richter, T., & Schmid, S. (2010). Epistemological beliefs and epistemic strategies in self-regulated learning. Metacognition and Learning, 5(1), 47–65.
Robertson-Lang, L., Major, S., & Hemming, H. (2011). An exploration of search patterns and credibility Issues among older adults seeking online health information. Canadian Journal on Aging / La Revue Canadienne Du Vieillissement, 30(4), 631–645. https://doi.org/10.1017/S071498081100050X
Rouet, J.-F., & Britt, M. A. (2011). In M. T. McCrudden, J. P. Magliano, & G. Schraw (Eds.), Text relevance and learning from text (p. 19–52). IAP Information Age Publishing.
Sandoval, W. A. (2014). Science education’s need for a theory of epistemological development. Science Education, 98(3), 383–387.
Sandoval, W. A. (2017). Situating practices of epistemic cognition. In T. G. Amin & O. Levrini (Eds.), Converging Perspectives on Conceptual Change (1st ed., pp. 253–260). Routledge. https://doi.org/10.4324/9781315467139-30
Sandoval, W. A., & Redman, E. H. (2015). The contextual nature of scientists’ views of theories, experimentation, and their coordination. Science & Education, 24(9–10), 1079–1102.
Sinatra, G. M., & Chinn, C. A. (2012). Thinking and reasoning in science: Promoting epistemic conceptual change. In APA educational psychology handbook: Application to learning and teaching (Vol. 3, pp. 257–282). American Psychological Association.
Subramaniam, M., Taylor, N. G., St. Jean, B., Follman, R., Kodama, C., & Casciotti, D. (2015). As simple as that?: Tween credibility assessment in a complex online world. Journal of Documentation, 71(3), 550–571. https://doi.org/10.1108/JD-03-2014-0049
Thon, F. M., & Jucks, R. (2017). Believing in expertise: How authors’ credentials and language use influence the credibility of online health information. Health Communication, 32(7), 828–836. https://doi.org/10.1080/10410236.2016.1172296
Ulyshen, T. Z., Koehler, M. J., & Gao, F. (2015). Understanding the connection between epistemic beliefs and Internet searching. Journal of Educational Computing Research, 53(3), 345–383.
White, A. M., DeCuir-Gunby, J. T., & Kim, S. (2019). A mixed methods exploration of the relationships between the racial identity, science identity, science self-efficacy, and science achievement of African American students at HBCUs. Contemporary Educational Psychology, 57, 54–71. https://doi.org/10.1016/j.cedpsych.2018.11.006
Wiley, J., Goldman, S. R., Graesser, A. C., Sanchez, C. A., Ash, I. K., & Hemmerich, J. A. (2009). Source evaluation, comprehension, and learning in Internet science inquiry tasks. American Educational Research Journal, 46(4), 1060–1106. https://doi.org/10.3102/0002831209333183
Wineburg, S. S. (1991). On the reading of historical texts: Notes on the breach between school and academy. American Educational Research Journal, 28(3), 495–519.
Zhang, S., & Duke, N. K. (2011). The impact of instruction in the WWWDOT framework on students’ disposition and ability to evaluate web sites as sources of information. The Elementary School Journal, 112(1), 132–154. https://doi.org/10.1086/660687
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2022 Courtney A. Denton, Krista R. Muis, Adam K. Dubé, Skylar Armstrong
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors wishing to include figures, tables, or text passages that have already been published elsewhere are required to obtain permission from the copyright owner(s) for both the print and online format and to include evidence that such permission has been granted when submitting their papers. Any material received without such evidence will be assumed to originate from the authors.
