LLMimic icon

LLMimic

An interactive and gamified AI literacy tutorial

LLMimic is a role-play-based tutorial where users assume the role of an LLM and progress through three key stages of the training pipeline — pretraining, SFT, and RLHF. Human study results show it significantly improved participants' AI literacy (p < .001), reduced persuasion success across three realistic scenarios (p < .05), and enhanced truthfulness and social responsibility in the hotel scenario (p < 0.01).

This work has been accepted as a poster at COLM 2026.

LLMimic overview
Paper Demo Human Study Data Code

Demo Stats

— completions of the LLMimic demo.

Citation

@misc{fan2026trainllmexploringeffects,
      title={Train Yourself as an LLM: Exploring Effects of AI Literacy on Persuasion via Role-playing LLM Training}, 
      author={Qihui Fan and Min Ge and Chenyan Jia and Weiyan Shi},
      year={2026},
      eprint={2604.02637},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2604.02637}, 
}