• Home
  • Archived Webinars

Integrating QE methods into a qualitative workflow

Jonathon Sun
Jonathon Sun

University of Pennsylvania

Abstract

Qualitative methods can be daunting because of the sheer number of themes that arise from the data; however, quantitative ethnography (QE) has provided resources to begin making meaning. This workshop will introduce participants to methods to assist in making meaningful qualitative themes, such as a co-occurrence network. First, this workshop will show participants how to clean and import data from other qualitative software like Dedoose. Next, a co-occurrence matrix will be made using the R package/plugin SNAHelper. Finally, based on the selection of themes, an ENA will be built based on the themes from the selected network.

Resources

Video: Transcript

Presentation: Slides




Building Your Own QE Workshop

Mariah A. Knowles
Mariah A. Knowles

University of Wisconsin-Madison

Abstract

This webinar will be an interactive discussion about creating workshops, lessons, and other hands-on activities that make our research community the collaborative community that it is. This entails everything from teaching new students in your lab how to conduct reproducible research, to developing new QE tools and now needing to train others how to use them. In this discussion, we'll share our experiences and best practices about how we design QE lessons, how we design lessons collaboratively, and how we share lessons with others to remix for their own use. We will draw on The Carpentries's Collaborative Lesson Development Training, though primarily we will discuss our own lesson development aims and experiences.

Resources

Video: Transcript




Leveraging Epistemic Network Analysis to Discern the Development of Shared Understanding between Physicians and Nurses

Vitaliy Popov
Vitaliy Popov

University of Michigan

Abstract

In healthcare settings, poor communication between physicians and nurses is one of the most common causes of adverse events. This study used Epistemic Network Analysis to help identify communication patterns in physician-nurse dyad interactions. We used existing video data where physicians made patient care rounds on two oncology patient units at a large academic medical center, and video recordings captured conversations physicians had with nurses on the plan of care. All data was transcribed, segmented and annotated using the Verbal Response Mode (VRM) taxonomy. The results showed that the relationship between Edification and Disclosure was strongest for the dyads that reached a shared understanding, suggesting the importance of these two modes to reaching shared understanding during patient care rounds. Reflection and Interpretation were the least used VRM codes, and this might be one possible area for intervention development. This pilot study provided new insight into how to improve communication between physicians and nurses using ENA coupled with VRM taxonomy.

Resources

Video: Transcript




Teaching a QE Course: Triumphs and Tensions

David Williamson Shaffer
David Williamson Shaffer

University of Wisconsin-Madison

Golnaz Arastoopour Irgens
Golnaz Arastoopour Irgens

Clemson University

Amanda Barany
Amanda Barany

Drexel University

Abstract

Join Amanda Barany, Golnaz Arastoopour Irgens, and David Wiliamson Shaffer as they discuss their experiences teaching Quantitative Ethnography courses at their respective universities. The QE instructors will share successes and challenges developing an introductory QE course and what these courses can mean for shaping the QE community. This webinar will be in a "podcast" discussion style. In the first half, the three participants will ask each other questions and engage in a discussion. In the second half, the participants will take questions from the audience.

Resources

Video: Transcript




Upstream Sources of Bias in Adaptive Learning Systems

Shamya Karumbaiah
Shamya Karumbaiah

Carnegie Mellon University & joining University of Wisconsin-Madison 2023

Abstract

Adaptive systems in education need to ensure population validity to meet the needs of all students for an equitable outcome. Recent research highlights how these systems encode societal biases leading to discriminatory behaviors towards specific student subpopulations. However, the focus has mostly been on investigating bias in predictive modeling, particularly its downstream stages like model development and evaluation. In this talk, I hypothesize that the upstream sources (i.e., theory, design, training data collection method) in the development of adaptive systems also contribute to the bias in these systems, highlighting the need for a nuanced approach to conducting fairness research. By empirically analyzing student data previously collected from various virtual learning environments, I investigate demographic disparities in three cases representative of the aspects that shape technological advancements in education: 1) non-conformance of data to a widely-accepted theoretical model of emotion, 2) differing implications of technology design on student outcomes, and 3) varying effectiveness of methodological improvements in annotated data collection. In doing so, I challenge implicit assumptions of generalizability and provide an evidence-based commentary on future research and design practices in adaptive and artificially intelligent educational systems surrounding how we consider diversity in our investigations.

Resources

Video: Transcript

Presentation: Slides




Dialogues and Digital Education

Rupert Wegerif
Rupert Wegerif

University of Cambridge

David Williamson Shaffer
David Williamson Shaffer

Unviversity of Wisconsin - Madison

Abstract

Dialogue is important to education, and digital technologies changes the conditions for both dialogue and education, locally and globally. This conversation between Rupert Wegerif and David Williamson Shaffer zooms in on this development by discussing three questions about the importance of dialogue to education, the potentials and pitfalls of digital technologies to support dialogues in education, and the way that digital methods can support evaluation frameworks and heuristics for educational dialogues

Resources

Video: Transcript




Advancing JEDI (Justice, Equity, Diversity, Inclusion) Efforts Through QE

Danielle P. Espino

Pepperdine University

Bryan C. Keene

Riverside City College

Abstract

This webinar will explore how research using QE can help to advance dialogue on JEDI (Justice, Equity, Diversity, Inclusion) topics. The presenters will highlight one study as an example of scholarly insight and analysis to informal sharing of testimonies that would otherwise be ignored. Through documentation and dissemination, such studies give voice to stories that long to be heard in ways that make it harder for structures of power to ignore and, hopefully, inform tangible change.

Resources

Video: Transcript

Presentation: Slides




Moral and Political Discussion and Epistemic Networks

Peter Levine
Peter Levine

Associate Dean for Academic Affairs and Lincoln Filene Professor of Citizenship & Public Affairs in Tufts University's Jonathan M. Tisch College of Civic Life

Abstract

Studies using ENA and other research supports the following theory: An individual holds linked beliefs about political or moral issues, which we can model as a network. How these ideas are connected influences the person’s actions and opinions. When individuals discuss, they share portions of their networks. While interacting, we are conscious of only some aspects of our own networks. Some network structures are better than others for discussion; overly centralized or scattered networks are problematic. Individuals tend to demonstrate similar network structures on different issues. People, with their respective networks of ideas, are also embedded in social networks. An idea is more likely to spread depending on features of both the social network and the idea network. As a whole, a population may develop a shared network structure. An idea that is widely shared and frequently central in individuals’ networks becomes a norm. Institutions are partly composed of such norms. A community or a culture is a single network with disagreement. Ultimately, all such networks interconnect.

Resources

Presentation: Slides




Building Quantitative Ethnography Hubs: Facilitating and Fostering QE in New Spaces

Hendrik Drachsler
Hendrik Drachsler

Goethe-University Frankfurt am Main, & the Open University of the Netherlands

Golnaz Arastoopour Irgens
Golnaz Arastoopour Irgens

Clemson University

Abstract

This webinar will focus on building new hubs of Quantitative Ethnography research around the world. The session will include discussion of introducing QE to new institutions, building QE labs, introducing QE to existing labs, and connecting across institutions. Our presenters, Hendrik Drachsler and Golnaz Arastoopour Irgens will share their own experiences and perspectives on potential opportunities and challenges related to this topic. We also plan for this session to engage others from around the world who are also involved and/or interested in these efforts while we explore how they might take shape in a variety of contexts.

Resources

Video: Transcript




Implications for Giving Participants Voice in QE

Michael Phillips
Michael Phillips

Monash University

Hazel Vega Quesada
Hazel Vega Quesada

Clemson University

Abstract

This webinar will introduce the idea of Participatory Quantitative Ethnography (PQE) in which research participants are given active roles in meaning making. The session will draw on examples from a number of different research projects which have begun to use different PQE techniques. From these examples opportunities and challenges associated with new tools and methods along with ethical issues associated with participant involvement in the QE process will be raised in the hope they spark fruitful conversations and collaborations that advance explorations and understandings of PQE.

Resources

Video: Transcript




QE Fireside Chat: A Conversation about Designing Tools for QE

Cody Marquart
Cody Marquart

University of Wisconsin-Madison

César Hinojosa
César Hinojosa

University of Wisconsin-Madison

David Williamson Shaffer
David Williamson Shaffer

University of Wisconsin-Madison

Abstract

A fireside chat with Cody Marquart and Cesar Hinojosa about designing tools for QE. This is an opportunity to talk with two of the lead designers of ENA and nCoder, and see how they integrate technical and aesthetic perspectives with the mathematical and theoretical foundations of QE. In the session, Cody and Cesar will talk with David Williamson Shaffer about their design processes, the trials and tribulations of building tools for large audiences, and the need to develop a more community-based participatory design process going forward --- with plenty of time, as usual, for questions and discussion with the audience.

Resources

Video: Transcript




Putting the E in QE: How Quantitative Ethnography can Enrich Qualitative Analyses

Yotam Hod
Yotam Hod

University of Haifa

Abstract

In this session, we will take a deep look at a case study where Epistemic Network Analysis (ENA) was employed after the completion of a qualitative analysis that showed the emotional development of a classroom learning community, ultimately leading to new findings and insights. Based on this example, our session will explore how ENA can be used to both confirm the results of qualitative investigations, as well as to extend on their findings by suggesting new ways to qualitatively explore data. As an additional feature of this presentation, the exploration of this case study will highlight ways in which ENA can be used to examine complex socioemotional phenomena.




QE Data ChallengeField Report: Reflecting on the benefits of multidisciplinary research teams

Daniel Spikol
Daniel Spikol

University of Copenhagen, DK

Stefano Schiavetto
Stefano Schiavetto

State University of Campinas, Brazil

Barbara Wasson
Barbara Wasson

University of Bergen, Norway

Karoline Schnaider
Karoline Schnaider

University of Umeå, Sweden

Abstract

For our seminar, we unpack our experiences and the resulting paper presented at the ISQE 2021 conference. The seminar looks at how a diverse team from multiple countries and research disciplines worked together with the QE tools to investigate how the different countries in Scandinavia responded at the beginning of 2020 (January to March). We use our paper, "Governmental Response to the COVID-19 Pandemic-A Quantitative Ethnographic Comparison of Public Health Authorities' Communication in Denmark, Norway, and Sweden", as a case to explore and reflect on the tools and team building. The seminar explores the challenges and opportunities of working with these tools.

Resources

Video: Transcript




The Bellwether Problem: Publishing the First QE Studies in a New Field

Abigail Wooldridge
Abigail Wooldridge

University of Illinois at Urbana-Champaign

Andrew Ruis
Andrew Ruis

University of Wisconsin-Madison

Sarah Jung
Sarah Jung

University of Wisconsin-Madison

Abstract

Publishing QE research in fields that are not familiar with the theories, techniques, and research designs that QE scholars use can be a significant barrier to adoption, especially for junior scholars. In this webinar, three members of the QE in Healthcare SIG discuss their experiences presenting and publishing QE work in medical contexts and reflect on the strategies that helped successfully introduce QE to healthcare research.

Resources

Video: Transcript

Presentation: Slides




Making Sense of Collocated Teamwork Activity: The Multimodal Matrix as a Quantitative Ethnography Methodology

Roberto Martinez-Maldonado
Roberto Martinez-Maldonado

Senior Lecturer, Monash University

Gloria Milena Fernandez-Nieto
Gloria Milena Fernandez-Nieto

Doctoral Researcher, University of Technology Sydney

Simon Buckingham Shum
Simon Buckingham Shum

Professor, University of Technology Sydney

Vanessa Echeverria
Vanessa Echeverria

Profesor Ocasional, Escuela Superior Politécnica del Litoral, Guayaquil

Abstract

Collocated, face-to-face teamwork remains a pervasive mode of working and learning, which is hard to replicate online. In team-based situations, learners’ embodied, multimodal interaction with each other and with digital and material resources has been studied by researchers, but due to its complexity, has remained opaque to automated analysis. The ready availability of sensors makes it increasingly affordable to instrument work spaces to automatically capture activity traces to study teamwork and groupwork. Yet, a key challenge is the enrichment of these multiple and intertwined quantitative data streams with the qualitative insights needed to make sense of them. In this seminar, we will discuss our inroads into giving meaning to multimodal group data. We have followed a human-centred approach to design meaningful end-user interfaces that convert multimodal data into data stories. Based on Quantitative Ethnography principles, we developed a modelling technique, termed the Multimodal Matrix, to grounding quantitative data in the semantics derived from a qualitative interpretation of the context from which it arises. We will present practical examples in the context of high-fidelity clinical simulations in which multimodal data (physiological, positioning, and logged actions) have been transformed into learning analytics interfaces that support teachers’ and learners’ reflection.




Epistemic network modeling of pre-service teacher’s STEM teaching competence through collaborative learning design

Bian Wu
Bian Wu

Associate Professor, East China Normal University

Abstract

Pre-service teacher training is crucial to guarantee the quality of STEM education. However, how teachers develop STEM teaching competence through collaborative learning design remains unclear. This study investigated high and low-performing groups who worked together to draft STEM lesson plans in tutor-guided online design meetings. An epistemic network modeling approach was employed to compared their STEM learning competence networks drawn from conversation. The findings revealed that there are significant differences in the networks between high and low-performing groups, which may contribute to the quality of their STEM lesson plans. Implications of fostering STEM teaching competence development are discussed.




We’re All in this Together: Modeling Interdependence in Collaborative Settings

Zachari Swiecki
Zachari Swiecki

Lecturer, Monash University

Abstract

When individuals collaborate their actions are not isolated. Instead, they respond to and build upon the actions of others, creating interdependent systems. This suggests that valid models of collaborative processes, which inform research, assessments, and education, should account for interdependence. However, many models treat the collaborative actions of individuals as independent from the context in which they occur. In this talk, I discuss an approach for determining the conditions under which models that account for interdependence are more appropriate. The approach estimates the difference between independent and interdependent models using information about the social and cognitive structure of the collaborative context. Using simulation studies, I show that the estimates are reliable under a variety of conditions. This work furthers our understanding of the social and cognitive interactions that characterize collaboration, and provides guidance for researchers as to which kind of model (independent vs interdependent) may be more appropriate for their data.




Where social and epistemic networks meet

Srecko Joksimovic
Srecko Joksimovic

Senior Lecturer, University of South Australia

Abstract

In addition to the commonly used data sources about learners and their knowledge practices, data about learners’ social interactions have also attracted significant attention of learning analytics researchers. Social network analysis (SNA) emerged early as one of the cornerstones of the learning analytics research, providing the opportunity to automatically extract large-scale networks from learners' interactions across various environments, such as LMSs and different social media platforms. Epistemic network analysis (ENA), on the other hand, recently emerged as a technique to analyse coded data of individual and collaborative learning. ENA is a graph-based method for analysing associations between coded data and represents an operationalization of the learning science theory of epistemic frames. As such, these two methods represent complementary approaches that, combined, provide comprehensive understanding of knowledge processes at the individual and group level.

In this talk, we will review the strengths and opportunities that SNA and ENA provide methodologically for learning analytics. In so doing, we will evaluate various educational contexts and demonstrate the depth of analytical insights obtained when these two network-based approaches are utilised together. Finally, although both methods build on strong theoretical underpinnings, we will discuss the role of Connectivism as a theoretical base to further develop social and epistemic network signature – SENS, as coined by Gašević and colleagues.

Resources

Video: Transcript

Presentation: Slides




Using quantitative ethnography to tell stories that have not been told before

Golnaz Arastoopour Irgens
Golnaz Arastoopour Irgens

Clemson University

Abstract

Quantitative ethnography (QE) leverages the computational abilities of automation and precision and the human abilities of interpretation and analysis to tell powerful stories about human behavior and learning. The computational tools highlight potentially interesting findings, while the researcher works with the computer to curate meaningful results and re-examine biases. Working together using QE methods, researcher and computer can reveal findings that may otherwise not have been revealed without QE methods and tell data-based stories from people whose voices have not been historically included in scientific findings. In this talk, I will share an example of how QE is an inclusive methodology that can reveal untold stories about how people see themselves and others in the world. The study examines pre-service teachers in Costa Rica and how they negotiate their identities as non-native English speakers who are teaching English as a foreign language. This example illustrates the use of the QE methodology to directly confront biases and assumptions in ourselves and our tools, visualize data to tell the participants' stories that may not have been told before, and co-create understandings with participants and others.

Resources

Video: Transcript

Presentation: Slides

Paper: Identity Negotiation of Pre-Service Teachers of English as a Foreign Language

Learn more about the IDEA Lab: Website




A Linguistic Ethnographic Perspective on Classroom Identities and Participation (And Some Challenges for Quantitative Ethnography)

Adam Lefstein
Adam Lefstein

Professor, Ben-Gurion University of the Negev

Hadar Netz
Hadar Netz

Tel Aviv University

Aviv Orner
Aviv Orner

Tel Aviv University

Eran Hakim
Eran Hakim

Ben-Gurion University of the Negev

Abstract

Most quantitative research on classroom discourse focuses on structural and cognitive dimensions of the interaction. For example, researchers have examined teacher questions, student argumentation, sequential structures, and the distribution of participation. For good reason: such variables are central to many of our conceptualizations of effective pedagogy, and they readily lend themselves to systematic observation and quantitative measurement. Nevertheless, more happens in classroom discourse and interaction than is captured in such measures. Students negotiate their own and one another's identities, make sense of lesson content and expectations, manage relationships with peers and teacher, struggle to assert their voices, and find creative ways of passing the time while also staying out of trouble. Likewise, teachers are occupied with managing these student concerns, classroom power relations, and institutional pressures, while also living up to institutional and ideological expectations. Linguistic ethnography offers a powerful set of tools for making sense of such forces and issues, which critically shape learning processes and outcomes. On the other hand, these tools are not well-suited to quantifying variables or working with large data sets. In this talk we will (a) provide a brief introduction to a linguistic ethnographic perspective on classroom discourse analysis; (b) demonstrate this perspective through the analysis of identity, peer relations and participation in a brief classroom episode; and (c) present some of our initial attempts to transform this object of inquiry into a set of variables that would allow us to engage in quantitative ethnographic analysis.

Resources

Video: Transcript

Presentation: Slides




The influence of discipline on teachers’ knowledge and decision making

Michael Phillips
Michael Phillips

Senior Lecturer, Monash University

Abstract

The knowledge required by teachers has long been a focus of public and academic attention. Following a period of intense research interest in teachers’ knowledge in the 1980’s and 1990’s, many researchers have adopted Shulman’s (1987) suggestion that expert teaching practice is based on seven forms of knowledge which collectively are referred to as a knowledge base for teaching. Shulman’s work also offered a decision-making framework known as pedagogical reasoning and action which allows teachers to use their seven forms of knowledge to make effective pedagogical decisions. Despite the widespread acceptance of these ideas, no empirical evidence exploring the connections between knowledge and decision-making are evident in the research literature. This paper reports on a pilot study in which the connections between knowledge and decisions in science, mathematics and information technology teachers’ lesson plans are quantified and represented using epistemic network analysis. Findings reveal and levels of complexity that have been intimated but, until now, not supported with empirical evidence.

Resources

Video: Transcript

Presentation: Slides




Visualizing patient decision-making processes regarding choice of therapy with Epistemic Network Analysis: A worked example of manual coding and segmentation

Szilvia Zörgő
Szilvia Zörgő

Assistant professor, Semmelweis University

Gjalt-Jorn Peters
Gjalt-Jorn Peters

Assistant Professor, Open University of the Netherlands

Abstract

Our research initiative aimed to explore patient decision-making concerning their choice of therapy: biomedicine, non-conventional medicine, or both. These decisions, occurring throughout the patient journey, are intricately tied to the patient’s previous experiences, their trusted sources of information, and what they think caused their illness. We employed Epistemic Network Analysis (ENA) as an analytical system that enabled us to handle large amounts of data and capture the systemic nature of many variables involved. Yet applying Quantitative Ethnography (QE) techniques to continuous narratives (e.g. semi-structured interviews) in an inquiry where manual segmentation with a multitude of codes is preferred poses several challenges. In order to address these issues, we developed the Reproducible Open Coding Kit (ROCK) – convention, open source software, and interface – that eases manual coding, enables researchers to reproduce the coding process, compare results, and collaborate. Our aim is to broaden the usage of QE, while facilitating Open Science principles and transparency. Our webinar will elaborate our research, address issues surrounding the QE treatment of continuous narratives, and introduce the basic functionality of the ROCK. We hope to see you there!

Visualizing patient decision-making processes regarding choice of therapy with Epistemic Network Analysis: A worked example of manual coding and segmentation

Resources

Video: Transcript

Presentation: Szilvia Zörgő | Gjalt-Jorn Peters

Paper: ICQE19 Proceedings

Learn more about ROCK: ROCK Book | iROCK | ROCK Tutorial