DiSCoKit: An Open-Source Toolkit for Deploying Live LLM Experiences in Survey Research

Abstract: Advancing social-scientific research of human-AI interaction dynamics and outcomes often requires researchers to deliver experiences with live large-LLMs to participants through online survey platforms. However, technical and practical challenges (from logging chat data to manipulating AI behaviors for experimental designs) often inhibit survey-based deployment of AI stimuli. We developed DiSCoKit--an open-source toolkit for deploying live LLM experiences (e.g., ones based on models delivered through Microsoft Azure portal) through JavaScript-enabled survey platforms (e.g., Qualtrics). This paper introduces that toolkit, explaining its scientific impetus, describes its architecture and operation, as well as its deployment possibilities and limitations.