QE Fireside Chat: A Conversation about Designing Tools for QE
University of Wisconsin-Madison
David Williamson Shaffer
University of Wisconsin-Madison
University of Wisconsin-Madison
A fireside chat with Cody Marquart and Cesar Hinojosa about designing tools for QE. This is an opportunity to talk with two of the lead designers of ENA and nCoder, and see how they integrate technical and aesthetic perspectives with the mathematical and theoretical foundations of QE. In the session, Cody and Cesar will talk with David Williamson Shaffer about their design processes, the trials and tribulations of building tools for large audiences, and the need to develop a more community-based participatory design process going forward --- with plenty of time, as usual, for questions and discussion with the audience.
Putting the E in QE: How Quantitative Ethnography can Enrich
University of Haifa
In this session, we will take a deep look at a case study where Epistemic Network Analysis (ENA) was employed after the completion of a qualitative analysis that showed the emotional development of a classroom learning community, ultimately leading to new findings and insights. Based on this example, our session will explore how ENA can be used to both confirm the results of qualitative investigations, as well as to extend on their findings by suggesting new ways to qualitatively explore data. As an additional feature of this presentation, the exploration of this case study will highlight ways in which ENA can be used to examine complex socioemotional phenomena.
QE Data ChallengeField Report: Reflecting on the benefits of multidisciplinary research teams
University of Copenhagen, DK
University of Bergen, Norway
State University of Campinas, Brazil
University of Umeå, Sweden
For our seminar, we unpack our experiences and the resulting paper presented at the ISQE 2021 conference. The seminar looks at how a diverse team from multiple countries and research disciplines worked together with the QE tools to investigate how the different countries in Scandinavia responded at the beginning of 2020 (January to March). We use our paper, "Governmental Response to the COVID-19 Pandemic-A Quantitative Ethnographic Comparison of Public Health Authorities' Communication in Denmark, Norway, and Sweden", as a case to explore and reflect on the tools and team building. The seminar explores the challenges and opportunities of working with these tools.
The Bellwether Problem: Publishing the First QE Studies in a New Field
University of Illinois at Urbana-Champaign
University of Wisconsin-Madison
University of Wisconsin-Madison
Publishing QE research in fields that are not familiar with the theories, techniques, and research designs that QE scholars use can be a significant barrier to adoption, especially for junior scholars. In this webinar, three members of the QE in Healthcare SIG discuss their experiences presenting and publishing QE work in medical contexts and reflect on the strategies that helped successfully introduce QE to healthcare research.
Making Sense of Collocated Teamwork Activity: The Multimodal Matrix as a Quantitative Ethnography Methodology
Collocated, face-to-face teamwork remains a pervasive mode of working and learning, which is hard to replicate online. In team-based situations, learners’ embodied, multimodal interaction with each other and with digital and material resources has been studied by researchers, but due to its complexity, has remained opaque to automated analysis. The ready availability of sensors makes it increasingly affordable to instrument work spaces to automatically capture activity traces to study teamwork and groupwork. Yet, a key challenge is the enrichment of these multiple and intertwined quantitative data streams with the qualitative insights needed to make sense of them. In this seminar, we will discuss our inroads into giving meaning to multimodal group data. We have followed a human-centred approach to design meaningful end-user interfaces that convert multimodal data into data stories. Based on Quantitative Ethnography principles, we developed a modelling technique, termed the Multimodal Matrix, to grounding quantitative data in the semantics derived from a qualitative interpretation of the context from which it arises. We will present practical examples in the context of high-fidelity clinical simulations in which multimodal data (physiological, positioning, and logged actions) have been transformed into learning analytics interfaces that support teachers’ and learners’ reflection.
Epistemic network modeling of pre-service teacher’s STEM teaching competence through collaborative learning design
Pre-service teacher training is crucial to guarantee the quality of STEM education. However, how teachers develop STEM teaching competence through collaborative learning design remains unclear. This study investigated high and low-performing groups who worked together to draft STEM lesson plans in tutor-guided online design meetings. An epistemic network modeling approach was employed to compared their STEM learning competence networks drawn from conversation. The findings revealed that there are significant differences in the networks between high and low-performing groups, which may contribute to the quality of their STEM lesson plans. Implications of fostering STEM teaching competence development are discussed.
We’re All in this Together: Modeling Interdependence in Collaborative Settings
When individuals collaborate their actions are not isolated. Instead, they respond to and build upon the actions of others, creating interdependent systems. This suggests that valid models of collaborative processes, which inform research, assessments, and education, should account for interdependence. However, many models treat the collaborative actions of individuals as independent from the context in which they occur. In this talk, I discuss an approach for determining the conditions under which models that account for interdependence are more appropriate. The approach estimates the difference between independent and interdependent models using information about the social and cognitive structure of the collaborative context. Using simulation studies, I show that the estimates are reliable under a variety of conditions. This work furthers our understanding of the social and cognitive interactions that characterize collaboration, and provides guidance for researchers as to which kind of model (independent vs interdependent) may be more appropriate for their data.
Where social and epistemic networks meet
In addition to the commonly used data sources about learners and their knowledge practices, data about learners’ social interactions have also attracted significant attention of learning analytics researchers. Social network analysis (SNA) emerged early as one of the cornerstones of the learning analytics research, providing the opportunity to automatically extract large-scale networks from learners' interactions across various environments, such as LMSs and different social media platforms. Epistemic network analysis (ENA), on the other hand, recently emerged as a technique to analyse coded data of individual and collaborative learning. ENA is a graph-based method for analysing associations between coded data and represents an operationalization of the learning science theory of epistemic frames. As such, these two methods represent complementary approaches that, combined, provide comprehensive understanding of knowledge processes at the individual and group level.
In this talk, we will review the strengths and opportunities that SNA and ENA provide methodologically for learning analytics. In so doing, we will evaluate various educational contexts and demonstrate the depth of analytical insights obtained when these two network-based approaches are utilised together. Finally, although both methods build on strong theoretical underpinnings, we will discuss the role of Connectivism as a theoretical base to further develop social and epistemic network signature – SENS, as coined by Gašević and colleagues.
Using quantitative ethnography to tell stories that have not been told before
A Linguistic Ethnographic Perspective on Classroom Identities and Participation (And Some Challenges for Quantitative Ethnography)
Most quantitative research on classroom discourse focuses on structural and cognitive dimensions of the interaction. For example, researchers have examined teacher questions, student argumentation, sequential structures, and the distribution of participation. For good reason: such variables are central to many of our conceptualizations of effective pedagogy, and they readily lend themselves to systematic observation and quantitative measurement. Nevertheless, more happens in classroom discourse and interaction than is captured in such measures. Students negotiate their own and one another's identities, make sense of lesson content and expectations, manage relationships with peers and teacher, struggle to assert their voices, and find creative ways of passing the time while also staying out of trouble. Likewise, teachers are occupied with managing these student concerns, classroom power relations, and institutional pressures, while also living up to institutional and ideological expectations. Linguistic ethnography offers a powerful set of tools for making sense of such forces and issues, which critically shape learning processes and outcomes. On the other hand, these tools are not well-suited to quantifying variables or working with large data sets. In this talk we will (a) provide a brief introduction to a linguistic ethnographic perspective on classroom discourse analysis; (b) demonstrate this perspective through the analysis of identity, peer relations and participation in a brief classroom episode; and (c) present some of our initial attempts to transform this object of inquiry into a set of variables that would allow us to engage in quantitative ethnographic analysis.
The influence of discipline on teachers’ knowledge and decision making
The knowledge required by teachers has long been a focus of public and academic attention. Following a period of intense research interest in teachers’ knowledge in the 1980’s and 1990’s, many researchers have adopted Shulman’s (1987) suggestion that expert teaching practice is based on seven forms of knowledge which collectively are referred to as a knowledge base for teaching. Shulman’s work also offered a decision-making framework known as pedagogical reasoning and action which allows teachers to use their seven forms of knowledge to make effective pedagogical decisions. Despite the widespread acceptance of these ideas, no empirical evidence exploring the connections between knowledge and decision-making are evident in the research literature. This paper reports on a pilot study in which the connections between knowledge and decisions in science, mathematics and information technology teachers’ lesson plans are quantified and represented using epistemic network analysis. Findings reveal and levels of complexity that have been intimated but, until now, not supported with empirical evidence.
Visualizing patient decision-making processes regarding choice of therapy with Epistemic Network Analysis: A worked example of manual coding and segmentation
Our research initiative aimed to explore patient decision-making concerning their choice of therapy: biomedicine, non-conventional medicine, or both. These decisions, occurring throughout the patient journey, are intricately tied to the patient’s previous experiences, their trusted sources of information, and what they think caused their illness. We employed Epistemic Network Analysis (ENA) as an analytical system that enabled us to handle large amounts of data and capture the systemic nature of many variables involved. Yet applying Quantitative Ethnography (QE) techniques to continuous narratives (e.g. semi-structured interviews) in an inquiry where manual segmentation with a multitude of codes is preferred poses several challenges. In order to address these issues, we developed the Reproducible Open Coding Kit (ROCK) – convention, open source software, and interface – that eases manual coding, enables researchers to reproduce the coding process, compare results, and collaborate. Our aim is to broaden the usage of QE, while facilitating Open Science principles and transparency. Our webinar will elaborate our research, address issues surrounding the QE treatment of continuous narratives, and introduce the basic functionality of the ROCK. We hope to see you there!