Affording Process Auditability with QualAnalyzer: An Atomistic LLM Analysis Tool for Qualitative Research

Large language models are seeping into qualitative social science. But how do you know what the model actually did? A new preprint on arXiv (arXiv:2604.03820) introduces QualAnalyzer, an open‑source Chrome extension for Google Workspace that aims to make LLM‑assisted qualitative analysis auditable by taking an atomistic approach to each data segment.

What the tool does

QualAnalyzer reportedly processes each segment of qualitative data independently and records the full analytic transaction — prompt, input and output — creating a persistent trail that reviewers or auditors can inspect. Built as a Chrome extension that integrates with Google Workspace, the project’s stated goal is to preserve the micro‑decisions behind coding and interpretation so that claims based on LLM workstreams can be traced and, in principle, reproduced.

Why it matters — and the trade‑offs

The push for transparency around automated decision‑making is global. Regulators in the EU and US are increasingly focused on model explainability and documentation; China’s own algorithm governance regime — notably rules introduced since 2022 that demand greater transparency for recommendation systems — has pushed organizations to document how automated outputs are produced. Tools like QualAnalyzer could help researchers meet audit and documentation expectations, but there are trade‑offs: recording prompts and outputs creates sensitive logs, and a Chrome/Google Workspace implementation raises questions about data residency and platform dependence, especially in contexts where Google services are restricted.

The paper is a preprint and not peer‑reviewed. It has been reported that the authors have released the code as open source, enabling community inspection and further development. Whether QualAnalyzer becomes a standard part of qualitative LLM workflows will depend on how teams balance reproducibility, participant privacy, and cross‑border data constraints.