Skip to content
BranchesTags

Repository files navigation

Benchmarking Large Language Models for Drug Combination Alerts: Achieving Expert-Level Reliability via Knowledge Grounding and Contextual Reasoning

  • doi: 10.1021/acs.jmedchem.5c03511

Python 3.8+PyPI versionLicense: BSD-2-Clause

⚠️IMPORTANT DISCLAIMER

🚨 CRITICAL NOTICE: This tool is designed for research and educational purposes only.

CoMed is NOT intended for direct clinical use and should NOT be used as the sole basis for clinical decision-making.

  • ✅ Intended Use: Research, education, and clinical decision support for healthcare professionals
  • ❌ NOT for: Direct patient care, automated clinical decisions, or replacing professional medical judgment
  • 🔬 Target Users: Clinical researchers, healthcare professionals, and medical students
  • ⚖️ Responsibility: Always consult qualified healthcare professionals for clinical decisions

By using this software, you acknowledge that it is for research and educational purposes only.

🎯 Overview

CoMed is a comprehensive framework for analyzing drug co-medication risks using advanced AI techniques including Retrieval-Augmented Generation (RAG), Chain-of-Thought (CoT) reasoning, and multi-agent collaboration. It automates the process of searching medical literature, analyzing drug interactions, and generating detailed risk assessment reports.

Why CoMed?

  • Research Efficiency: Automate literature review for drug interaction studies
  • Comprehensive Analysis: Combine multiple AI approaches for thorough risk assessment
  • Reproducible Results: Standardized methodology for consistent analysis
  • Scalable Processing: Handle multiple drug combinations efficiently
  • Evidence-Based: Ground analysis in peer-reviewed medical literature

🔧 Key Features

1. Literature Retrieval & Analysis

  • Automated PubMed search with intelligent query construction
  • Relevance filtering and ranking of medical literature
  • Statistical analysis of retrieved papers
  • Support for custom search parameters

2. Advanced AI Reasoning

  • Chain-of-Thought reasoning for step-by-step analysis
  • Multi-agent collaboration for comprehensive assessment
  • Conflict resolution and consensus building
  • Confidence-weighted decision making

3. Comprehensive Risk Assessment

  • Multi-dimensional risk analysis (pharmacokinetic, pharmacodynamic, clinical)
  • Evidence-based risk scoring
  • Detailed interaction mechanism analysis
  • Clinical recommendation generation

4. Flexible Architecture

  • Modular design for easy customization
  • Support for different LLM providers
  • Configurable analysis parameters
  • Extensible agent system

📦 Installation

Prerequisites

  • Python 3.8 or higher
  • OpenAI API key (or compatible LLM API)

Install from PyPI

pip install comed

Install from Source

git clone https://github.com/studentiz/comed.git
cd comed
pip install -e .

🚀 Quick Start

Basic Setup

importosimportcomed# Configure your LLM APIos.environ["MODEL_NAME"] ="gpt-4o"# or "gpt-3.5-turbo", "qwen2.5-32b-instruct"os.environ["API_BASE"] ="https://api.openai.com/v1"os.environ["API_KEY"] ="your-api-key-here"# Initialize with drug listdrugs= ["warfarin", "aspirin", "ibuprofen"]
com=comed.CoMedData(drugs)
# Run complete analysisreport_path=com.run_full_analysis(retmax=30, verbose=True)
print(f"Report generated at: {report_path}")

Step-by-Step Analysis

# 1. Search medical literaturecom.search(retmax=20, email="your.email@example.com")
# 2. Analyze drug associationscom.analyze_associations()
# 3. Assess riskscom.analyze_risks()
# 4. Generate reportcom.generate_report("Drug_Interaction_Report.html")

📚 Usage Examples

Example 1: Cardiovascular Drug Interactions

importosimportcomed# Set up API credentialsos.environ["MODEL_NAME"] ="gpt-4o"os.environ["API_BASE"] ="https://api.openai.com/v1"os.environ["API_KEY"] ="your-api-key-here"# Analyze cardiovascular drug combinationscardiovascular_drugs= ["warfarin", "aspirin", "clopidogrel", "metoprolol"]
com=comed.CoMedData(cardiovascular_drugs)
# Run analysis with method chainingcom.search(retmax=50) \
.analyze_associations() \
.analyze_risks() \
.generate_report("Cardiovascular_Interactions.html")

Example 2: Diabetes Medication Analysis

# Analyze diabetes medication interactionsdiabetes_drugs= ["metformin", "insulin", "glipizide", "pioglitazone"]
com=comed.CoMedData(diabetes_drugs)
# Incremental analysiscom.search(retmax=30)
com.analyze_associations()
# Add more drugs and continue analysiscom.add_drugs(["sitagliptin", "canagliflozin"])
com.search(retmax=30)
com.analyze_risks()
com.generate_report("Diabetes_Medication_Analysis.html")

Example 3: Multi-Agent Collaboration

fromcomedimportMultiAgentSystem# Initialize multi-agent systemagent_system=MultiAgentSystem(
model_name="gpt-4o",
api_key="your-key",
api_base="https://api.openai.com/v1"
)
# Analyze specific drug combinationdrug1, drug2="warfarin", "aspirin"abstract="Literature abstract content..."# Use consensus collaborationresult=agent_system.process_drug_combination(
drug1, drug2, abstract, collaboration_mode="consensus"
)
print(f"Risk Assessment: {result['risk_analysis']}")
print(f"Safety Recommendation: {result['safety_assessment']}")
print(f"Clinical Guidance: {result['clinical_recommendation']}")

Example 4: Batch Processing

# Process multiple drug combinationsdrug_combinations= [
["warfarin", "aspirin"],
["metformin", "lisinopril"],
["atorvastatin", "amlodipine"]
]
fori, drugsinenumerate(drug_combinations):
com=comed.CoMedData(drugs)
com.search(retmax=20)
com.analyze_associations()
com.analyze_risks()
com.generate_report(f"Combination_{i+1}_Report.html")

🎮 Demo Scripts

Run the included demo scripts to see CoMed in action:

# Basic functionality demo
python examples/basic_demo.py
# Quick start tutorial
python examples/quick_start.py
# Multi-agent system demo
python examples/simple_agent_test.py
# Load existing data and analyze
python examples/load_and_analyze.py

🔧 Advanced Configuration

Environment Variables

export MODEL_NAME="gpt-4o"export API_BASE="https://api.openai.com/v1"export API_KEY="your-api-key"export LOG_DIR="logs"export OLD_OPENAI_API="No"# Set to "Yes" for older OpenAI API format

Custom LLM Configuration

# Configure different LLM providerscom=comed.CoMedData(["drug1", "drug2"])
com.set_config({
'model_name': 'qwen2.5-32b-instruct',
'api_base': 'https://your-llm-api.com/v1',
'api_key': 'your-api-key'
})

Analysis Parameters

# Customize search parameterscom.search(
retmax=50, # Number of papers to retrieveemail="your.email@example.com", # Required for PubMeddate_range=("2020/01/01", "2024/12/31") # Optional date filter
)
# Customize analysis depthcom.analyze_associations(
confidence_threshold=0.7, # Minimum confidence for associationsinclude_mechanisms=True# Include interaction mechanisms
)

🏗️ Architecture

Core Components

CoMed Framework
├── RAG Module (rag.py)
│ ├── Literature retrieval from PubMed
│ ├── Relevance scoring and filtering
│ └── Statistical analysis
├── CoT Module (cot.py)
│ ├── Chain-of-thought reasoning
│ ├── Step-by-step analysis
│ └── Result interpretation
├── Multi-Agent Module (agents.py)
│ ├── RiskAnalysisAgent
│ ├── SafetyAgent
│ ├── ClinicalAgent
│ └── Collaboration protocols
└── Core Module (core.py)
├── Component integration
├── Configuration management
└── Result aggregation

Data Flow

Drug Combinations → Literature Search → Association Analysis → Risk Assessment → Report Generation
↓ ↓ ↓ ↓
Input Validation PubMed Retrieval CoT Reasoning Multi-Agent Analysis
↓ ↓ ↓ ↓
Query Construction Relevance Filtering Evidence Analysis Consensus Building
↓ ↓ ↓ ↓
Search Execution Statistical Analysis Risk Scoring Final Report

📊 Performance & Evaluation

CoMed provides built-in performance monitoring and evaluation capabilities:

# Monitor analysis performancecom=comed.CoMedData(["warfarin", "aspirin"])
com.search(retmax=20)
# Get performance statisticsstats=com.get_performance_stats()
print(f"Papers retrieved: {stats['papers_retrieved']}")
print(f"Analysis time: {stats['analysis_time']:.2f}s")
print(f"Success rate: {stats['success_rate']:.2%}")

📚 API Reference

Core Classes

  • CoMedData: Main analysis class
  • RAGSystem: Literature retrieval system
  • CoTReasoner: Chain-of-thought reasoning
  • MultiAgentSystem: Multi-agent collaboration

Key Methods

Analysis Methods

  • search(retmax, email, date_range): Search medical literature
  • analyze_associations(confidence_threshold): Analyze drug associations
  • analyze_risks(risk_dimensions): Assess interaction risks
  • generate_report(filename): Generate HTML report

Multi-Agent Methods

  • process_drug_combination(drug1, drug2, abstract, mode): Process with agents
  • get_agent_stats(): Get agent performance statistics
  • set_collaboration_mode(mode): Set collaboration strategy

Configuration Methods

  • set_config(config_dict): Set system configuration
  • add_drugs(drug_list): Add drugs to analysis
  • save_data(filename): Save analysis data
  • load_data(filename): Load existing data

🛠️ Development

Adding Custom Agents

fromcomed.agentsimportAgentclassCustomAnalysisAgent(Agent):
def__init__(self, model_name, api_key, api_base):
super().__init__(
name="CustomAnalysisAgent",
description="Custom drug analysis",
model_name=model_name,
api_key=api_key,
api_base=api_base
)
def_execute_task(self, input_data):
# Implement custom analysis logicreturn {"custom_result": "Analysis result"}

Extending Analysis Modules

fromcomed.ragimportRAGSystemclassCustomRAGSystem(RAGSystem):
def__init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
defcustom_search_method(self, query):
# Implement custom search logicpass

🤝 Contributing

We welcome contributions in various forms:

  1. Code Contributions: New features, bug fixes, performance optimizations
  2. Documentation: Better examples, tutorials, API documentation
  3. Testing: Unit tests, integration tests, benchmark tests
  4. Research: New evaluation metrics, test scenarios, and performance studies

Development Setup

# Clone the repository
git clone https://github.com/studentiz/comed.git
cd comed
# Install in development mode
pip install -e .# Install development dependencies (optional)
pip install -r requirements-dev.txt
# Set up environment variablesexport MODEL_NAME="gpt-4o"export API_BASE="https://api.openai.com/v1"export API_KEY="your-api-key-here"

📄 License

This project is licensed under the BSD-2-Clause License. See LICENSE file for details.

🙏 Acknowledgments

Thanks to the research community for valuable feedback and contributions. Special thanks to reviewers who helped improve the framework's modularity and evaluation capabilities.

📞 Contact

Note: This framework is for research purposes only and should not be used for clinical decision-making. Any medical decisions should be made in consultation with qualified healthcare professionals.

About

CoMed is a comprehensive framework for analyzing drug co-medication risks using Chain-of-Thought (CoT) reasoning and large language models. It automates the process of searching medical literature, analyzing drug interactions, and generating detailed risk assessment reports.

Resources

Readme
BSD-2-Clause license
Activity

Stars

22 stars

Watchers

1 watching

Forks

4 forks
Report repository

Releases

Packages

Used by

Contributors

Languages