2020 Results

Participants

Sixteen teams from around the world (representing 16 different countries and 6 continents) participated in the data challenge. You can see the team and some of their preliminary findings below.

About QE Methods

There are many ways to conduct quantitative ethnography (QE) research, but QE analyses have five interdependent elements that distinguish them from other approaches to modeling rich qualitative data.

1. Grounded Analysis
 

QE researchers begin with a thorough qualitative analysis of their data (or sample of their data). This facilitates informed interpretation of the data, hypothesis generation, and identification of constructs that may explain the cognitive, behavioral, or interactive phenomena of interest.

2. Data Coding
 

To locate constructs in their data, researchers develop a system for determining which constructs are present in which parts of the data. That system can be as simple as a human rater reading each line to determine whether a given construct is in evidence.

3. Automated Coding Process
 

When researchers work with large datasets, it is not possible to make coding decisions manually for every excerpt. To identify key constructs in data at scale, researchers often develop automated coding algorithms.

4. Coding Validation
 

Regardless of the coding process used, researchers need to validate their codes to establish that the coding decisions are consistent and accurate. This requires, at a minimum, establishing sufficient agreement between two independent human raters (for manual coding) or between two independent human raters and the automated coding algorithms (for automated coding).

5. Model Specification
 

Once a dataset has been coded for the constructs of interest, researchers develop models of those codes to conduct exploratory or confirmatory analyses. This involves selecting a technique that can model the relationships among codes found in the qualitative analysis, and choosing variables that define the unit of analysis and other model parameters. Here, a particular emphasis in QE is aligning the quantitative model with the pattern observed in the qualitative analysis, so that the quantitative model can warrant that the grounded pattern observed is consistent throughout the data.

6. Qualitative Regrounding (Loop Closing)
 

Finally, researchers confirm that a result from their model is meaningful through a process of qualitative regrounding. That is, they confirm that the outputs and explanatory properties of a model are aligned with the findings of their original grounded analysis, thus closing the interpretive loop.

Findings

Because the Data Challenge lasted only a week, teams did not complete analyses to the point of being ready for peer review. Any findings should thus be treated with appropriate caution.

To aid in the interpretation of results, teams indicated which of the steps of their QE analysis were completed, in progress, or planned.

Investigating the different responses to the Coronavirus situation between Norway, Denmark and Sweden

Examining the national debate and international perceptions, with a focus on the people’s relationship to the state, as well as the relationship between epidemic, healthcare system, biology and illness.

Daniel Spikol Sweden/Denmark

Daniel Spikol is an associate professor at Malmö University’s department of Computer Science and Media Technology. He is the group leader of Smart Learning at the Internet of Things and People Research Center. Current research explores how to use multimodal learning analytics to support small-group collaboration. He is interested in using QE to help make sense of how LA can benefit teachers and students.

Morten Misfeldt Denmark

Morten is professor of digital education and director of the Center for Digital Education at the University of Copenhagen.

Jonas Dreyøe Denmark

Ben Allsopp Denmark

Florian Meier Denmark

Karoline Schnaider Sweden

Karoline Schnaider is a second-year Ph.D. student from the Department of Education, Umeå University. In her doctoral studies she is specializing in digital technologies in education and is part of the interdisciplinary research school GRADE. Her research concerns technology use in sign-making activities from a social semiotic multimodal perspective. From a visual approach to meaning-making through technology use, equal focus is put on the technologies (hardware/software in combination), sign systems and modes of representation, and how these components come together in different activities. In her current work, she has engaged in the development of a theoretical framework called multimodal layers, which strives to take a comprehensive and detailed approach to technology use to analyze future technology use and implementation in educational environments.

Stefano Schiavetto Amancio Sweden

Barbara Wasson Norway

Barbara Wasson is Professor in Information Science and Director for the Centre for the Science of Learning & Technology, University of Bergen, Norway. Her research interests span from artificial intelligence in education, distributed collaboration, socio-cultural theories of learning, teacher inquiry, learning design, to learning analytics.

MENTOR: Ariel Fogel United States

Ariel Fogel is a first year PhD student at the University of Wisconsin-Madison, working at the Epistemic Analytics lab. He is interested in collaborative problem solving, learning analytics, and the development of new quantitative ethnographic methodologies.

Status:

In progress:

Grounded Analysis

Data coded

Automated classifier

Codes validated

Model completed

Planned:

Qualitative regrounding

See preliminary results

The evolution of guidance to individuals on risk factors and mitigation factors from early January to late March

Eric Hamilton United States

Eric Hamilton is Professor of Education at Pepperdine’s Graduate School of Education and Psychology, and courtesy mathematics faculty. He currently leads what is known as the IC4 project – the International Community for Collaborative Content Creation (ic4.site) involving digital makerspace networking between school-aged learners in the US, Africa, Europe, and South America. He recently completed a one year term at UNESCO’s International Bureau of Education, following a multiyear Fulbright based in Namibia exploring the use of learning technologies in formal education settings.

Seung Lee United States

David Stern United Kingdom

Lin Zhu United States

Zachariahe Mbasu Kenya

Zach Mbasu is a passionate educator and the founder of African Maths Initiative (AMI) in Kenya. He is very passionate about Integrating technology and turning data into useful information in Education. He is a country leader in the IC4 project – the International Community for Collaborative Content Creation (ic4.site) involving digital makerspace networking between school-aged learners in the US, Africa, Europe, and South America.

Ateamate Mukabana Kenya

Ateamate is an undergraduate student at Pwani University in Kenya pursuing a degree in Mathematics. She is so passionate about mathematics and is looking forward to influencing more female gender in the mathematics field. Ateamate is an IC4 project- International Community for Collaborative Content Creation, alumni and currently a facilitator and Kenyan student leader in the same project

Tiffany Wright United States

Tiffany Wright is a doctoral student in the Global Leadership and Change, Ph.D. program at Pepperdine University. She has served in various roles of Student Services in Higher Education as well as an adjunct faculty of Education. Her current research in girl’s leadership development examines the use of educational technology in leadership development programs as a catalyst to promote equity and access.

Christopher Sokolov United States

Christopher K. Sokolov, Ed.D., teaches at University of San Francisco School of Education and the Bay Area Teacher Training Institute. He is also the Technology and Research Consultant at Town School for Boys. He received his doctorate from Pepperdine University Graduate School of Education & Psychology and wrote his dissertation on teacher engagement. In addition to his teaching and research interests, he serves as an Orthodox Christian priest caring for Holy Trinity Cathedral in San Francisco, California.

Matthew Sweeney United States

Victoria Brown United States

Claudine Hudson United States

Claudine is a PhD student at Pepperdine University, her research interest surrounds political leadership and corruption. Claudine’s passion for social justice led her to establishing a nonprofit called Project Arete Inc. The organization works to provide social and economic support to vulnerable groups.

Danielle Espino United States

MENTOR: Amanda Siebert-Evenstone United States

Amanda Siebert-Evenstone is graduating this summer with a PhD from the Department of Educational Psychology at the University of Wisconsin – Madison focusing on Learning Sciences. Amanda works in the Epistemic Analytics Group to develop innovative learning technologies and statistical tools to improve the assessment of complex thinking. Amanda’s dissertation uses quantitative ethnography to develop a method to measure and model 3-dimensional learning in STEM classrooms.

Exploring the discourse of news outlets during the time of COVID by political bias

Ariel Fogel United States

Ariel Fogel is a first year PhD student at the University of Wisconsin-Madison, working at the Epistemic Analytics lab. He is interested in collaborative problem solving, learning analytics, and the development of new quantitative ethnographic methodologies.

JooYoung Seo United States

Doy Kim United States

Pauley Tedoff Canada

Soyoung Choi United States

MENTOR: Zhiqiang Cai United States

MENTOR: Simon Buckingham Shum Australia

Simon Buckingham Shum is Professor of Learning Informatics at the University of Technology Sydney, where he directs the Connected Intelligence Centre (CIC). A career-long fascination with how computers help make thinking visible has seen his teams develop a range of computational tools to “augment human intellect” (after Doug Engelbart). His current focus, and CIC’s missions, is the future of education, specifically, the contribution of human-centred design of Analytics/AI-powered tools to close the feedback loop to learners and educators. http://Simon.BuckinghamShum.net

Comparison of discussion of telehealth by healthcare professionals on Twitter in April 2019 and April 2020, during the Covid-19 pandemic

Sarah Jung United States

Sarah Jung Ph.D. is an Assistant Professor in Education Research and Development in the Department of Surgery at the University of Wisconsin-Madison. She has studied the incorporation and impact of digital technologies in multiple learning environments. She is currently involved in numerous studies in the areas of undergraduate, graduate, and continuing surgical education. Her Educational Psychology background allows her to apply theories of learning to understand how people become expert physicians and how we can support this process to facilitate quality patient care.

Azliza Mohd Ali Malaysia

Azliza Mohd Ali is a Senior Lecturer at Faculty of Computer and Mathematical Sciences, Universiti Teknologi MARA, Malaysia. She dedicates herself to university teaching and conducting research. Currently her research interest on anomaly detection, data mining, machine learning and knowledge-based systems. Quantitate Ethnography is still new for her and this is her first experience joining QE Data Challenge.

Sarah Larson United States

Haiqin Yu China

Vitaliy Popov United States

Vitaliy Popov is an Assistant Professor of Learning Health Sciences at the University of Michigan Medical School. In his current work, Vitaliy utilizes evidence in education science, simulation-based training and learning analytics to understand how health professionals can better work in teams and how we can support these processes to foster health care delivery and health outcomes.

MENTOR: Parameswaran Ramanathan United States

Status:

Completed:

Grounded Analysis

In progress:

Data coded

Planned:

Automated classifier

Codes validated

Model completed

Qualitative regrounding

See preliminary results

The sentimental manifestations of people regarding the covid19 scenario and its implications for mental health

Roberto Martinez-Maldonado Australia

Roberto Martinez-Maldonado is a Senior Lecturer in the Faculty of Information Technologies at Monash University where he applies quantitative ethnography principles to study relationships between multimodal interaction traces captured from physical learning spaces such as collaborative classrooms, team training rooms and laboratory spaces.

Mahir Akgun United States

Irni Eliana Khairuddin Malaysia

Adam Morishita United States

Pedro Lealdino Filho Brazil

Pedro Lealdino Filho has a PhD degree in Mathematics Education from Université Claude Bernard, Lyon – France. He is currently working at the Federal Technological University of Brazil (UTFPR) as a postdoctoral researcher developing digital technologies for teaching mathematics for visually impaired students.

MENTOR: Karin Frey United States

MENTOR: Brendan Eagan United States

Brendan R. Eagan is Associate Director for Partnerships and Community Engagement in the Epistemic Analytics lab at the Wisconsin Center for Education Research. His work focuses on cultivating the evolving culture and growing community of Quantitative Ethnographers. His research focuses on the reliability and validity of tools and methods for Quantitative Ethnography. He has been heartened to see the exciting, creative, and welcoming atmosphere of this communal challenge response.

Exploring the ways in which political leaders shape their public discourse in response to the COVID-19 pandemic: A case study from the United States of America and New Zealand

Angie Frabasilio United States

Angie Frabasilio, Utah State University Doctoral Student in Mathematics Education and Leadership. Full-time mathematics teacher. Utilizing Epistemic Network Analysis to investigate adaptive reasoning of middle school students engaged in mathematical tasks.

Mike Phillips Australia

Michael Phillips is a Senior Lecturer in the Faculty of Education at Monash University where he uses quantitative ethnography to explore the relationships between teachers skills, knowledge, identity, values and epistemologies and their pedagogical decision-making in technology rich contexts.

Aileen Owens United States

Nora’ayu Ahmad Uzir Malaysia

Nora’ayu Ahmad Uzir is a final-year PhD candidate at the School of Informatics, The Edinburgh University and a lecturer at the Faculty of Information Management, Universiti Teknologi MARA (UiTM Shah Alam). Her research involves using learning analytics methods to explore the time management and learning strategies in online learning environments. In her current study, she and her colleagues propose a new method that integrates multiple data analytic techniques (unsupervised machine learning, process mining and Epistemic Network Analysis) to detect time management and learning strategies in trace data and also demonstrates relevance and applicability to diverse learning contexts.

Amanda Barany United States

Amanda Barany is a PhD Candidate in the Educational Leadership and Learning Technologies program at Drexel University’s School of Education. Her research explores the identities and patterns of participation enacted by learners in online communities of practice. Other areas of research interest and expertise include: design-based research, game-based learning, interest and motivation, social network analysis and epistemic network analysis, and design thinking strategies.

Xavier Ochoa United States

MENTOR: Jun Oshima Japan

Network analysis of COVID19 Tweets between Donald Trump and Center for Disease Control Twitter

Szilvia Zörgő Hungary

Szilvia Zörgő is a medical anthropologist working as an assistant professor at the Institute of Behavioral Sciences at Semmelweis University, Faculty of Medicine. She is also a lecturer at the Institute of Intercultural Psychology and Education at Eötvös Loránd University. Her research focuses on the sociocultural factors of therapy choice and decision-making related to health.

Woodson Hobbs United States

David Sanchez United States

Getachew Tarekegn Ethiopia

MENTOR: Melanie Peffer United States

Melanie Peffer has a BS and PhD in molecular biology from the University of Pittsburgh and completed a postdoctoral appointment in learning sciences from Georgia State University. She holds a research faculty appointment at University of Colorado Boulder. Dr. Peffer combines her expertise in molecular biology and the learning sciences to study how people learn, understand, and engage with biology content. In particular, she’s interested in how we can use educational technology like simulations and serious games and learning analytics to assess what people know and believe about the nature of science knowledge. Melanie recently published Biology Everywhere: How the Science of Life Matters to Everyday Life, which focuses on where we see applications of biology as part of our everyday experience.

Status:

Completed:

Grounded Analysis

Data coded

Automated classifier

Codes validated

Model completed

Qualitative regrounding

See preliminary results

Team 8

Joshua Rosenberg United States

Samvel Mkhitaryan Netherlands

Samvel Mkhitaryan is a PhD candidate at the School for Public Health and Prim Care, at Maastricht University Faculty of Health, Medicine and Life Sciences. His research focuses on complex systems modeling in health behavior interventions and evaluations. Prior to his PhD studies, he obtained a Masters of Science degree from Tilburg University (The Netherlands) and a Masters of Public Health degree from the American University of Armenia.

Daniel Tetrick United States

Daniel Tetrick is a data scientist at Microsoft in Seattle, WA. He works in the field of Information Security focusing on methods for identifying and classifying ephemeral behavior in data. Prior to Microsoft, Daniel built Boeing’s spare parts pricing software, taught R programming and data science to international corporations, and provided statistical evidence for US federal court cases.

MENTOR: Zach Swiecki United States

An exploration of the discourse around the efficacy of mask use against COVID-19 between communication from health organisations and citizen science conversations on Twitter

Ming Liu Australia

Natasha Arthars Australia

Aneesha Bakharia Australia

Linda Corrin Australia

Linda Corrin is Associate Professor, Learning Analytics at Swinburne University of Technology in Melbourne, Australia. She is currently working on several research projects exploring how learning analytics can be used to provide meaningful and timely feedback to academics and students. While still new to QE, she sees many possibilities for its use within the context of her research which prompted her to join this challenge.

Hazel Vega Quesada United States

Hazel Vega is a third-year PhD student in the Learning Sciences program at Clemson University. She is part of the IDEA Lab, directed by Golnaz Arastoppour Irgens. Her research explores language teacher identity construction among pre-service teachers of English as a second/foreign language. Her work is situated within theoretical perspectives of communities of practice and expansive learning and predominantly uses equity-oriented and participatory methods. Through her work at the lab and with her advisor, she is getting familiar with QE and sees it as a potential tool to make better sense of data and tell grounded stories.

MENTOR: Golnaz Arastoopour Irgens United States

Golnaz Arastoopour Irgens is Assistant Professor of Learning Sciences, Director of the IDEA Lab, and Co-Director of the Digital Media and Learning (DML) Lab in the College of Education at Clemson University. In her learning analytics work, she uses quantitative ethnography to make sense of how learners engage with digital technologies in ways that serve their communities and are meaningful to them, focusing within the domains of computer science and engineering.

Mitigation measures against coronavirus: a case of the UK and the US

Mariana Castro United States

Mariana serves as the current Interm Director at the Wisconsin Center for Education Research at the University of Wisconsin Madison. In her research, Mariana focuses on language practices of multilingual students, curriculum and instruction in dual language immersion programs, teacher professional learning and family engagement.

Erkan Er Spain

Erkan Er received his PhD degree in Learning, Design, and Technology from the University of Georgia, USA. He is currently working as a postdoctoral researcher in GSIC-EMIC research group in the University of Valladolid, Spain. His research interests include using machine learning and educational data mining techniques to understand and support student learning in massive contexts. In his current project, he investigates peer feedback and learning analytics. He is the developer of the Synergy platform.

Stephen Aguilar United States

Marta Jackowska Denmark

Marta Jackowska is a PhD Candidate at Aarhus University, School of Business and Social Sciences, in Denmark. She studies organizational behavior, team dynamics and cross-boundary teams, particularly those spanning national borders. She is a former participant in the Doctoral Consortium at ICQE, and she uses Quantitative Ethnography to investigate phenomena related to collaboration in novel forms of organizational teamwork.

MENTOR: Andrew Ruis United States

Andrew R. Ruis is Associate Director for Research in the Epistemic Analytics lab at the Wisconsin Center for Education Research and a fellow in the Department of Medical History and Bioethics at the University of Wisconsin-Madison. He uses quantitative ethnography to study medical communication and public health. He is the author of Eating to Learn, Learning to Eat: The Origins of School Lunch in the United States, as well as numerous articles on the history of food, nutrition, and health; learning analytics and digital humanities; and STEM and medical education.

Status:

Completed:

Grounded Analysis

Data coded

Automated classifier

Model completed

Qualitative regrounding

In progress:

Codes validated

See preliminary results

An Initial Investigation into the Discourses of Blogpost Misinformation

Jais Brohinsky United States

Roziya Abu Malaysia

Mazlina Pati Khan Malaysia

Claire Brainard United States

Claire Brainard is a researcher and the Lab Manager at Wisconsin’s Equity and Inclusion Laboratory at the Wisconsin Center for Education Research. Her research interests focus on socio-cultural anthropology, specifically looking at refugee and immigrant communities and the cultural interplay between migrant and dominant national cultures through qualitative research methods and ethnography. She is new to quantitative ethnography and excited to be part of the QE Data Challenge!

Dilrukshi Gamage Sri Lanka

Dilrukshi is a final year Ph.D. student at the CSE Dept, University of Moratuwa, Sri Lanka. Her research is focusing on behaviors in large scale online learning, especially in MOOCs. She also possesses an interest in user experience/ behaviors of misinformation and works with a community of researchers, journalists, academics, policy-makers, technologists in an organization named Credibility Coalition. She has experience in many open science research projects such as Stanford Crowdsourced research Collective and Mozilla Open Leader. She was introduced to Epistemic Network Analysis at the EATEL – JTEL Summer Shcool held in Bari, Italy, and ever since then she tries to incorporate such methods to make more knowledge out of the large amount of learning behaviors she collected through platforms and building new socio-technical tools. She is excited to work with the QE team where once they were just citations to her research and now, she engages with them directly and became live citations.

MENTOR: Srecko Joksimovic Australia

Srecko Joksimovic is a Research Fellow (Data Scientist) at the Education Futures, University of South Australia. His research is centered around augmenting abilities of individuals to solve complex problems in collaborative settings. Srecko is particularly interested in evaluating the influence of contextual, social, cognitive, and affective factors on groups and individuals as they solve complex real-world problems.

Status:

Completed:

Grounded Analysis

Data coded

Model completed

Qualitative regrounding

In progress:

Automated classifier

Codes validated

See preliminary results

Team 12

Wenmo AAAA United States

Jeff Lee United States

Paul Sparks United States

Stephanie Alvarez United States

Haiying Lee United States

MENTOR: Vitomir Kovanovic Australia

Vitomir Kovanović is a Research Fellow at Education Futures, University of South Australia. His research focuses on the development of novel learning analytics systems, with a particular interested in students’ self-regulation of learning, use of personalised feedback, and adoption of learning analytics in primary and secondary settings.

Team 13

Madhuri Manjunath United States

Hamideh Talafian United States

Hamideh is graduating from her Ph.D. program in Educational Leadership and Learning Technologies. Her concentration in STEM education and she learned about quantitative ethnography and ENA during her program at Drexel University. Her research interest is in patterns of learning as changes in their individual or social perceptions conceptualized as perceived STEM career identities. QE has helped her research interests grow and she is very excited to further nourish this growth.

John Lunalo Kenya

Niraj Pathak United States

MENTOR: Abigail Wooldridge United States

Abigail R. Wooldridge is an Assistant Professor in the Department of Industrial and Enterprise Systems Engineering and PI of the Human Factors in Sociotechnical Systems Laboratory. Dr. Wooldridge’s research focuses on understanding, modeling and improving complex sociotechnical systems, in particular those found in health care. Her current work is focused on integrating her expertise in human factors, team cognition and mixed methods research to inform technology-based efforts to patient and health care professional outcomes associated with care transitions.

A Brief Examination of Twitter Feed on #RemoteLearning To Explore the Discourse Around Teaching and Learning During the COVID-19 Pandemic

The team investigated Twitter text of 1941 users with unique roles (e.g. teachers, schools, educational technology companies and institutes, families, informal learning providers, professors, and consultants) to examine the emergent nature of behavior exhibited from April 20-28, 2020.

Mohamad Noorman Masrek Malaysia

Mohamad Noorman Masrek, Malaysia, is an associate professor of information management system at the Faculty of Information Management, Universiti Teknologi MARA. His research interests revolves around social informatics that focus on user behavior. The majority of his research employed quantitative approach and used structural equation modelling for the data analysis. QE is something new to him and he is very excited to explore and learn in this collaboration platform.

Lisa Cuevas Shaw United States

Sukie Wang United States

Sukie Wang is a first-year master student in UW-Madison, Education school, who is interested in design-based research, online learning product design, learning outcome assessment, and informal learning environment building.

Mamta Shah United States

Mamta Shah is a Learning Scientist at Elsevier focusing on the design and delivery of impactful digital learning solutions for nursing and health education. She was introduced to the Quantitative Ethnography (QE) methodology and techniques in her former role as a postdoctoral scholar at Drexel University Games and Learning in Interactive Digital Environments (GLIDE) Lab. As a passion project, Mamta is examining what her 6-year old son is QErious about!

Meixi United States

Meixi grew up across lands of Singapore and Thailand and is interested in learning at the intersection of lands-school-family/community. Her work focuses on how education systems advance the restoration and flourishing of Indigenous communities in Southeast Asia and the Americas. Meixi is a learning scientist by training and currently a postdoctoral fellow in American Indian Studies at the University of Minnesota.

MENTOR: Yeyu Wang United States

Yeyu Wang is a PhD student in Epistemic Analytics Lab, University of Wisconsin-Madison. She is interested in theory-based analytics-driven research to explore how people learn. She focuses both on methodology and application of quantitative ethnography in the field of learning analytics. (https://www.yeyu-wang.com/)

A Divided Public: A Brief Analysis of Public Response to US Leader Policy on Personal & Political Levels

Jennifer Scianna United States

Jenn Scianna is an incoming PhD student at the University of Wisconsin Madison in Curriculum and Instruction. Her research passion involves using interaction data from educational games to better understand student thinking, particularly in relation to science. Jenn’s interest in QE stems from a desire to tell better stories with data that engage people from seemingly disparate fields.

Karl Vachuska United States

Kamila Misiejuk Norway

Kamila Misiejuk is a PhD candidate at the University of Bergen in Norway and works at the Centre for the Science of Learning and Technology (SLATE). Her research revolves around understanding peer assessment using learning analytics. She uses QE to explore students’ reactions to peer feedback.

Rogers Kaliisa Norway

Rogers Kaliisa is a PhD student at the Department of Education, University of Oslo, Norway. His research involves using learning analytics to support and evaluate learning design and individualised student support. He applies Quantitative Ethnography approaches to gain richer insights into students’ online social learning processes to inform data-informed learning and teaching decisions.

MENTOR: David Williamson Shaffer United States

David Williamson Shaffer is the Vilas Distinguished Professor of Learning Sciences at the University of Wisconsin-Madison in the Department of Educational Psychology and a Data Philosopher at the Wisconsin Center for Education Research. His work focuses on the development of tools and methods for Quantitative Ethnography, and he is excited to be part of this Data Challenge!