Project description: The Local Artificial Intelligence for Qualitative Analysis project aims to explore how exlores how open-source LLMs, running locally withiDigital Media Lab infrastructure can be meaningfully integrated into qualitative analysis. Researchers have been increasingly enabled in the collection of data through technological advances, however this surplus of structured qualitative data requires an equal amount of manual analysis and interpretation, if not more.
The question is whether LLMs can augment analysis or even automate parts of the workload. We will systematically investigate GenAI’s potential across different analytical phases including coding, thematic analysis, and analysis of metaphors and linguistic devices in interview interview transcripts, survey responses, or open-ended feedback. This has both practical, methodological, and epistemological implications for humanities and social science.
Working exploratively and systematically through different tests and workshops, we will document both possibilities and limitations of various approaches, which can also be critically evaluated by colleagues affiliated with the lab. The analysis focuses on three main aspects: 1) Evaluation of different LLMs’ suitability for qualitative analysis, 2) Methodological development of workflows and best practices, 3) Critical assessment of AI’s output quality and validity as a research tool.
While commercial tech-companies provide access to their LLMs and researchers can think of these as harmless and effective helpers/assistants, the concerns around privacy and data protection compliance are difficult to ignore, particularly when working with human participants, GDPR, and sensitive data
As these models generate outputs based on provided data, questions of data ownership, jurisdictional control and regulatory accountability are likewise increasingly complex and subject to geopolitical factors.
Likewise, the general lack of explainability (XAI-methodology) afforded by commercial LLMs raises further concerns about the transparency and trustworthiness of the output.
Ultimately, the project addresses the challenge of leveraging LLMs for research while maintaining ownership over data, interpretability, and trustworthiness. The project builds bridges between traditional qualitative research and cutting-edge AI technology by demonstrating practical applications that respect existing methodological traditions and ethical standards, rather than seeking to replace them with AI hype and automation.
Status: Prototyping-phase where we are designing the overarching architecture.
Contact: Mark Friis Hau and Frederik Møller Henriksen