Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

History

39 Commits

Repository files navigation

Creativity in LLM-based Multi-Agent Systems: A Survey

arXiv BadgeLicense Badge

Large language model (LLM)-driven multi-agent systems (MAS) are transforming how humans and AIs collaboratively generate ideas and artifacts. While existing surveys provide comprehensive overviews of MAS infrastructures, they largely overlook the dimension of creativity, including how novel outputs are generated and evaluated, how creativity informs agent personas, and how creative workflows are coordinated.

This survey offers a structured framework and roadmap for advancing the development, evaluation, and standardization of creative MAS.

🔥 Update

🤝 Citation

@inproceedings{lin2025creativity,
title = "Creativity in LLM-based Multi-Agent Systems: A Survey",
author = "Yi-Cheng Lin and Kang-Chieh Chen and Zhe-Yan Li and Tzu-Heng Wu and Tzu-Hsuan Wu and Kuan-Yu Chen and Hung-yi Lee and Chen, Yun-Nung",
booktitle = "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing",
month = nov,
year = "2025",
address = "Suzhou, China",
publisher = "Association for Computational Linguistics"
}

📃 Introduction

MAS comprises multiple autonomous entities: software agents, robots, or human-AI hybrids. This structure enables emergent collaboration and richer exploration of open-ended creative spaces, significantly underlying the practice for computational creativity. Here, computational creativity refers to the creation of artifacts—such as ideas, behaviors, or solutions—that are both novel and valuable, demonstrating clear usefulness or appeal rather than being random.

This survey focuses on systems whose inputs and outputs span text and images. We aim to map the current landscape of techniques, datasets, evaluations, and challenges to foster and measure creativity in such multimodal and heterogeneous systems. By analyzing how different agents interact, we reveal how collaborative structures can unlock creative potentials that exceed what isolated LLMs or individuals can achieve.

🌟 MAS Workflow and Proactivity

Overall Proactivity across Process and Decision-Making

MethodYearProcessDecision-Making
Rapid AIdeation: Generating Ideas With the Self and in Collaboration With Large Language Modelsarxiv(2024)lowlow
AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual CanvasCHIWork 2024lowlow
"It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Modelsarxiv(2024)lowlow
StoryDrawer: A Child–AI Collaborative Drawing System to Support Children's Creative Visual StorytellingCHI 2022lowlow
CharacterMeet: Supporting Creative Writers' Entire Story Character Construction Processes Through Conversation with LLM-Powered Chatbot AvatarsCHI 2024lowlow
SPARKIT: A Mind Map-Based MAS for Idea Generation SupportEMAS 2024mediumlow
Human–machine co-creation: a complementary cognitive approach to creative character design process using GANsThe Journal of Supercomputing 2024mediumlow
Towards Inclusive Co-creative Child-robot Interaction: Can Social Robots Support Neurodivergent Children's Creativity?HRI 2025mediumlow
MAxPrototyper: A Multi-Agent Generation System for Interactive User Interface Prototypingarxiv(2024)mediumlow
ContextCam: Bridging Context Awareness with Creative Human-AI Image Co-Creation2024mediumlow
Scideator: Human-LLM Scientific Idea Generation Grounded in Research-Paper Facet Recombinationarxiv(2025)mediumlow
C2Ideas: Supporting Creative Interior Color Design Ideation with a Large Language ModelCHI 2024mediumlow
Colin: A Multimodal Human-AI Co-Creation Storytelling System To Support Children's Multi-Level Narrative Skillsarxiv(2025)mediumlow
Creative Wand: A System to Study Effects of Communications in Co-Creative Settingsarxiv(2023)mediumlow
AI-Augmented Brainwriting: Investigating the use of LLMs in group ideationCHI 2024mediumlow
ChainBuddy: An AI-assisted Agent System for Generating LLM PipelinesCHI 2025mediumlow
CoQuest: Exploring Research Question Co-Creation with an LLM-based AgentGitHubCHI 2024mediumlow
An Interactive Co-Pilot for Accelerated Research IdeationHCINLP 2024mediumlow
A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative DesignIEEE Transactions on Visualization and Computer Graphics 2023mediummedium
Mathemyths: Leveraging Large Language Models to Teach Mathematical Language through Child-AI Co-Creative StorytellingCHI 2024mediummedium
PersonaFlow: Boosting Research Ideation with LLM-Simulated Expert Personasarxiv(2024)highlow
DesignGPT: Multi-Agent Collaboration in DesignISCID 2023mediummedium
Towards an AI co-scientistarxiv(2025)mediumhigh
LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultationarxiv(2024)mediumhigh
StoReys: A neurosymbolic approach to human-AI co-creation of novel action-oriented narratives in known story worldsICCC 2024mediumhigh
AgentReview: Exploring Peer Review Dynamics with LLM AgentsGitHubEMNLP 2024mediumhigh
StoryVerse: Towards Co-authoring Dynamic Plot with LLM-based Character Simulation via Narrative PlanningFDG 2024mediumhigh
PersonaGym: Evaluating Persona Agents and LLMsGitHubarXiv(2025)highmedium
AgentCoord: Visually Exploring Coordination Strategy for LLM-based Multi-Agent Collaborationarxiv(2024)highmedium
CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysisarxiv(2025)mediumhigh
MaCTG: Multi-Agent Collaborative Thought Graph for Automatic Programmingarxiv(2025)highhigh
Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent SystemGitHubarxiv(2025)highhigh
Generative Agents: Interactive Simulacra of Human BehaviorGitHubUIST 2023highhigh
HoLLMwood: Unleashing the Creativity of Large Language Models in Screenwriting via Role PlayingEMNLP 2024highhigh
LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-PlayGitHubarxiv(2024)highhigh
Encouraging Divergent Thinking in Large Language Models through Multi-Agent DebateGitHubEMNLP 2024highhigh
Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-CollaborationGitHubarxiv(2024)highhigh
Systematic Idea Refinement for Machine Learning Research AgentsTsinghua University Course:AML 2024highhigh
Unlocking AI Creativity: A Multi-Agent Approach with CrewAIJournal of Trends in Computer Science and Smart Technology 2024highhigh
On Generative Agents in RecommendationSIGIR 2024highhigh
MARG: Multi-Agent Review Generation for Scientific PapersGitHubarxiv(2024)highhigh

🌟 MAS Techniques for Creativity

Divergent Exploration

MethodYearTaskCode Link
Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking2024AUT and RAT
PersonaGym: A Reinforcement Learning Environment for Training Multi-Persona Agents2024Character Design
Creativity Support in the Age of Large Language Models2024Creative Writing
One Does Not Simply “Meme” Alone: Self-Dialogue for Creative Ideation2024Humor Co-Creation
Rapid AIdeation: Generating Ideas With the Self and in Collaboration With LLMs2024Idea Generation
AI-Augmented Brainwriting: Exploring Idea Flow Between Groups and Large Language Models2024Idea Generation
AI and the Future of Collaborative Work: Group Ideation With an LLM in a Virtual Canvas2024Idea Generation
ContextCam: Bridging Context Awareness With Creative Human-AI Image Co-creation2024Image Generation
C2Ideas: Supporting Creative Interior Color Design Ideation With an LLM2024Interior Color Design Ideation
IRIS: An Interactive Co-pilot for Accelerated Research Ideation2023Research Ideation
PersonaFlow: Dynamic Multi-persona Interaction Framework2024Research Ideation
VirSci: Autonomous Team-Leader Agent for Virtual Scientific Collaboration2025Scientific Research Co-creation
StoryDrawer: Sketch-Based Storyboarding With Multi-agent Discussion2024Sketches Generation
ICCRI: Inclusive Co-creative Child-Robot Interaction2025Story Generation
Sparkit: Mind-Map-Based Multi-Agent System for Idea Generation Support2024Story Generation

Iterative Refinement

MethodYearTaskCode Link
CharacterMeet: Supporting Creative Writers' Entire Story Character Construction Processes Through Conversation with LLM-Powered Chatbot Avatars2024Character Design
Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debate2024Debating (Fairness)
Interpreting and Mitigating Hallucination in MLLMs through Multi-agent Debate2024Hallucination Mitigation
LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultation2024Legal Consultation
ChainBuddy: An AI-assisted Agent System for Generating LLM Pipelines2025LLM Pipeline Generation
DesignGPT: Multi-Agent Collaboration in Design2023Product Design
Systematic Idea Refinement for Machine Learning Research Agents2024Scientific Research Co-creation
HoLLMwood: Unleashing the Creativity of Large Language Models in Screenwriting via Role Playing2024Screenwriting
A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative Design2024Sketches Generation
Generative Agents: Interactive Simulacra of Human Behavior2023Social Simulation

Collaborative Synthesis

MethodYearTaskCode Link
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks2024Agent Benchmarking
AgentCoord: Visually Exploring Coordination Strategy for LLM-based Multi-Agent Collaboration2024Agent Collaboration Visualization
Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration2024Cognitive Synergy
"It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models2024Creative Writing
Creative Agents: Simulating the Systems Model of Creativity with Generative Agents2024Creativity Simulation
ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematics2023Formal Proof
Beyond Code Generation: LLM-supported Exploration of the Program Design Space2025Program Design
Agent4Rec: On Generative Agents in Recommendation2024Recommendation
How AI Processing Delays Foster Creativity: Exploring Research Question Co-Creation with an LLM-based Agent2024Research Ideation
MARG: Multi-Agent Review Generation for Scientific Papers2024Research Peer Review
AgentReview: Exploring Peer Review Dynamics with LLM Agents2024Scientific Peer Review
Unlocking AI Creativity: A Multi-Agent Approach with CrewAI2024Scientific Research Co-creation
Towards an AI co-scientist2025Scientific Research Co-creation
Human--machine co-creation: a complementary cognitive approach to creative character design process using GANs2024Silhouette Generation
ChatDev: Communicative Agents for Software Development2024Software Engineering
MaCTG: Multi-Agent Collaborative Thought Graph for Automatic Programming2025Software Engineering
CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysis2024Story Generation
Colin: A Multimodal Human-AI Co-Creation Storytelling System To Support Children's Multi-Level Narrative Skills2024Story Generation
Mathemyths: Leveraging Large Language Models to Teach Mathematical Language through Child-AI Co-Creative Storytelling2024Story Generation
StoReys: A neurosymbolic approach to human-AI co-creation of novel action-oriented narratives in known story worlds2024Story Generation
StoryVerse: Towards Co-authoring Dynamic Plot with LLM-based Character Simulation via Narrative Planning2024Story Generation
MAxPrototyper: A Multi-Agent Generation System for Interactive User Interface Prototyping2024UI Prototyping

🌟 Persona and Agent Profile

Coarse-Grained

MethodPublicationProfiling MethodAgent Persona
Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-CollaborationGitHubarXiv(2024)Model-GeneratedSelf-Defined
LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-PlayGitHubCOLM 2024Model-GeneratedSelf-Defined
PersonaGym: Evaluating Persona Agents and LLMsGitHubarXiv(2025)Human-definedSelf-Defined
Systematic Idea Refinement for Machine Learning Research AgentsTsinghua University Course:AML 2024Human-DefinedAssitant
Sparkit: A mind map-based mas for idea generation support.EMAS 2024Human-DefinedSelf-Defined
Encouraging Divergent Thinking in Large Language Models through Multi-Agent DebateGitHubEMNLP 2024Human-DefinedDebater
An Interactive Co-Pilot for Accelerated Research IdeationHCINLP 2024Human-DefinedMentor & Colleague
ChainBuddy: An AI-assisted Agent System for Generating LLM PipelinesCHI 2025Human-DefinedMentor & Planner

Medium-Coarse-Grained

MethodPublicationProfiling MethodAgent Persona
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World TasksGitHubarXiv(2025)Model-GeneratedCompany Employee
HOLLMWOOD: Unleashing the Creativity of Large Language Models in Screenwriting via Role PlayinEMNLP 2024Human-DefinedArtist
TRIZ Agents: A Multi-Agent LLM Approach for TRIZ-Based InnovationICAART 2025Human-DefinedProblem Solver
Towards an AI co-scientistarxiv(2025)Human-DefinedResearcher
MaCTG: Multi-Agent Collaborative Thought Graph for Automatic Programmingarxiv(2025)Human-DefinedProgrammer
DesignGPT: Multi-Agent Collaboration in DesignISCID 2023Human-DefinedSelf-Defined
CoQuest: Exploring Research Question Co-Creation with an LLM-based AgentGitHubCHI 2024Human-DefinedResearcher
LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultationarxiv(2024)Human-DefinedLawyer
MARG: Multi-Agent Review Generation for Scientific PapersGitHubarxiv(2024)Human-DefinedExpert

Fine-Grained

MethodPublicationProfiling MethodAgent Persona
PersonaFlow: Boosting Research Ideation with LLM-Simulated Expert Personasarxiv(2024)Data-DerivedResearcher
Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent SystemGitHubarxiv(2025)Data-DerivedResearcher
On Generative Agents in RecommendationSIGIR 2024Data-DerivedMedia User
[PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traits]GitHubNAACL 2024Model-Generated
Self-Defined
The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model AgentsarXiv(2025)Model-GeneratedSelf-Defined
Creative Agents: Simulating the Systems Model of Creativity with Generative AgentsarXiv(2024)Model-GeneratedArtist
Synergizing Human-AI Agency: A Guide of 23 Heuristics for Service Co-Creation with LLM-Based AgentsarXiv(2023)Model-GeneratedSelf-Defined
Generative Agents: Interactive Simulacra of Human BehaviorGitHubUIST 2023Conditional Video GenerationHuman-defined
AgentReview: Exploring Peer Review Dynamics with LLM AgentsGitHubEMNLP 2024Human-definedReviewer

🌟 Evaluation

Text Artifact Evaluation

PapersYearTaskSubjectiveObjectiveCode Link
Human Creativity in the Age of LLMs: Randomized Experiments on Divergent and Convergent Thinking2025AUT and RATTTCT, Boden’s Criterion, and othersSemantic similarity
The Power of Personality: A Human Simulation Perspective to Investigate Large Language Model Agents2025AUT and othersTTCT-
LLMDiscussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play2024AUT and othersTTCT-Github
An Innovative Solution to Design Problems: Applying the Chain-of-Thought Technique to Integrate LLM-Based Agents With Concept Generation Methods2025Conceptual DesignTTCT-
Rapid AIdeation: Generating Ideas With the Self and in Collaboration With Large Language Models2024Idea GenerationTTCT-
AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation2024Idea GenerationInnovation, Insightfullness, and othersSemantic Similarity
LawLuo: A Multi-Agent Collaborative Framework for Multi-Round Chinese Legal Consultation2024Legal ConsultationPersonalization and Professionalism-
ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematics2023Mathematical Proving-BLEU ScoreGithub
DesignGPT: Multi-Agent Collaboration in Design2023Product DesignNovelty, Completeness, and Feasibility-
Using Generative AI Personas Increases Collective Diversity in Human Ideation2025Plot Generation-Semantic Similarity
Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging Writers2024Poem WritingFluency and Creativity-
MARG: Multi-Agent Review Generation for Scientific Papers2024Paper Review GenerationSpecificity and Overall Rating-Github
CoQuest: Exploring Research Question Co-Creation with an LLM-based Agent2024Research IdeationBoden’s criterion-Github
PersonaFlow: Boosting Research Ideation with LLM-Simulated Expert Personas2024Research IdeationCreativity, Usefulness, and Helpfulness-
HOLLMWOOD:Unleashing the Creativity of Large Language Models in Screenwriting via Role Playing2024ScreenwritingInterestingness, Relevance, and othersEntropy-n, Self-BLEU, and othersGithub
Towards an AI co-scientist2025Scientific Research Co-creationNovelty and Impact-Github
Mixed-Initiative Methods for Co-Creation in Scientific Research2024Scientific Research Co-creationNovelty, Specificity, and othersSemantic Similarity
Many Heads Are Better Than One: Improved Scientific Idea Generation by A LLM-Based Multi-Agent System2025Scientific Research Co-creationNovelty, Clarity, FeasibilitySemantic Euclidean DistanceGithub
Mathemyths: Leveraging Large Language Models to Teach Mathematical Language through Child-AI Co-Creative Storytelling2024Story GenerationReadability, Perceived Creativity, and others-Github
StoReys: A neurosymbolic approach to human-AI co-creation of novel action-oriented narratives in known story worlds2024Story GenerationInteractivity, Coherence, and othersSelf-BLEU
Creative Wand: A System to Study Effects of Communications in Co-creative Settings2022Story GenerationGoal completion and Satisfication-Github
CollabStory: Multi-LLM Collaborative Story Generation and Authorship Analysis2024Story GenerationCreativityEntropy and othersGithub
Towards Inclusive Co-creative Child-robot Interaction: Can Social Robots Support Neurodivergent Children's Creativity?2025Story GenerationTTCT-

Image Artifact Evaluation

PapersYearTaskSubjectiveObjectiveCode Link
C2Ideas: Supporting Creative Interior Color Design Ideation with a Large Language Model2024Interior Color Design IdeationInspiring, Reasonableness, and others-
CREA:ACollaborative Multi-Agent Framework for Creative Content Generation with Diffusion Models2025Image Editing & GenerationExpressiveness, Aesthetic appeal, and othersCLIP scores and others
One Does Not Simply Meme Alone: Evaluating Co-Creativity Between LLMs and Humans in the Generation of Humor2025Meme GenerationFunniness, Creativity, and Shareability-
StoryDrawer: A Child–AI Collaborative Drawing System to Support Children's Creative Visual Storytelling2022Sketches GenerationTTCTFID, TIE ,and Semantic loss
A Collaborative, Interactive and Context-Aware Drawing Agent for Co-Creative Design2024Sketches GenerationTTCT-Github
Drawing with Reframer: Emergence and Control in Co-Creative AI2023Sketches GenerationNovelty and Surprise within MICSI-Github
Human–machine co-creation: a complementary cognitive approach to creative character design process using GANs2024Silhouette GenerationDesigner’s reviewFID
MAxPrototyper: A Multi-Agent Generation System for Interactive User Interface Prototyping2024UI Prototype Generation-FID and Generation Diversity

Interaction Evaluation with User Study

PapersYearTaskCode Link
ContextCam: Bridging Context Awareness with Creative Human-AI Image Co-Creation2024Image Generation
AI and the Future of Collaborative Work: Group Ideation with an LLM in a Virtual Canvas2024Idea Generation
How AI Processing Delays Foster Creativity: Exploring Research Question Co-Creation with an LLM-based Agent2024Research Ideation
"It Felt Like Having a Second Mind": Investigating Human-AI Co-creativity in Prewriting with Large Language Models2024Creative Writing

🌟 Dataset

Psychological Test Datasets

DatasetYearTaskData Link

Task Specific Datasets

PaperTaskYearData Link
MacGyver: Are Large Language Models Creative Problem Solvers?Problem-Solving in Physically Grounded Scenarios2023Link
Creativity Support in the Age of Large Language Models: An Empirical Study Involving Emerging WritersCreative Writing2023Link
Large Language Models for Human-AI Co-Creation of Robotic Dance PerformancesRobot Dance Creation2024Link
Human–machine co-creation: a complementary cognitive approach to creative character design process using GANsCharacter Design2024Link
TriviaQAOpen-ended question task.2017Link
Generative Agents: Interactive Simulacra of Human BehaviorOpen-ended question task.2023Link
GPQA: A Graduate-Level Google-Proof Q&A BenchmarkOpen-ended question task2023Link
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelsCodenames Task2022Link
ProofNet: Autoformalizing and Formally Proving Undergraduate-Level MathematicsMathematical Formal Proof Generation and Verification2023Link
AI Idea Bench 2025: AI Research Idea Generation BenchmarkIdea Generation2025Link
LiveIdeaBench: Evaluating LLMs' Divergent Thinking for Scientific Idea Generation with Minimal ContextIdea Generation2024Link
COMPAS datasetFairness-Aware Debating2016Link

License

This repository is released under the Apache 2.0 license.

About

Survey Paper for Multi-Agent Interaction and Collaboration

Resources

Stars

14 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors