Feat: Add multi-provider LLM support (OpenAI + Gemini)

[dzakwanalifi]

Problem

PageIndex currently only supports OpenAI models, limiting user choice and potentially increasing costs.

Solution

Add unified interface supporting both OpenAI GPT-4 and Gemini 2.5 Flash models with structured output capabilities.

Key Changes

• Add LLM provider abstraction layer in utils.py
• Support structured output with Pydantic models for Gemini
• Add provider configuration in config.yaml
• Fix JSON parsing error handling in page_index.py
• Update function signatures for better model parameter handling

Code Changes

• utils.py: Added LLMProvider class with unified interface
• config.yaml: Added provider configuration option
• page_index.py: Enhanced error handling and Gemini integration

Use Case

Users can now choose between providers:

# OpenAI (default)
python run_pageindex.py --pdf_path doc.pdf
# Gemini
python run_pageindex.py --pdf_path doc.pdf --provider gemini

This provides flexibility for different cost/performance requirements while maintaining full compatibility.

[dgallitelli]

Great work on this multi-provider abstraction! 🎉

This pattern would make it straightforward to add additional cloud providers. I've opened two related feature requests that could build on this PR:

• Feature Request: Add Amazon Bedrock support for indexing and retrieval #104 - Amazon Bedrock support (Claude, Amazon Nova, Llama via AWS)
• Feature Request: Add Amazon SageMaker AI support for indexing and retrieval #105 - Amazon SageMaker AI support (self-hosted/custom models)

Both would follow the same LLMProvider interface pattern you've established here. The main implementation considerations:

For Bedrock:

• Uses boto3 with the Converse API
• Stop reason mapping: end_turn → stop, max_tokens → length
• Message format: {"content": [{"text": "..."}]}

For SageMaker:

• Uses invoke_endpoint with custom payload format (varies by model container)
• Enables self-hosted open models (Llama, Mistral, etc.)

Would you be open to expanding this PR to include AWS providers, or would separate PRs be preferred? Happy to contribute the Bedrock/SageMaker implementations if helpful.

[KylinMountain]

Thanks @dzakwanalifi! Multi-provider support (Gemini included) now ships via LiteLLM (#168, merged) — Gemini works with a gemini/<model> model name, no extra code needed. Closing as superseded, but thanks for pushing on this early! 🙏

[dzakwanalifi]

Problem

PageIndex currently only supports OpenAI models, limiting user choice and potentially increasing costs.

Solution

Add unified interface supporting both OpenAI GPT-4 and Gemini 2.5 Flash models with structured output capabilities.

Key Changes

• Add LLM provider abstraction layer in utils.py
• Support structured output with Pydantic models for Gemini
• Add provider configuration in config.yaml
• Fix JSON parsing error handling in page_index.py
• Update function signatures for better model parameter handling

Code Changes

• utils.py: Added LLMProvider class with unified interface
• config.yaml: Added provider configuration option
• page_index.py: Enhanced error handling and Gemini integration

Use Case

Users can now choose between providers:

# OpenAI (default)
python run_pageindex.py --pdf_path doc.pdf
# Gemini
python run_pageindex.py --pdf_path doc.pdf --provider gemini

This provides flexibility for different cost/performance requirements while maintaining full compatibility.

[dgallitelli]

Great work on this multi-provider abstraction! 🎉

This pattern would make it straightforward to add additional cloud providers. I've opened two related feature requests that could build on this PR:

• Feature Request: Add Amazon Bedrock support for indexing and retrieval #104 - Amazon Bedrock support (Claude, Amazon Nova, Llama via AWS)
• Feature Request: Add Amazon SageMaker AI support for indexing and retrieval #105 - Amazon SageMaker AI support (self-hosted/custom models)

Both would follow the same LLMProvider interface pattern you've established here. The main implementation considerations:

For Bedrock:

• Uses boto3 with the Converse API
• Stop reason mapping: end_turn → stop, max_tokens → length
• Message format: {"content": [{"text": "..."}]}

For SageMaker:

• Uses invoke_endpoint with custom payload format (varies by model container)
• Enables self-hosted open models (Llama, Mistral, etc.)

Would you be open to expanding this PR to include AWS providers, or would separate PRs be preferred? Happy to contribute the Bedrock/SageMaker implementations if helpful.

[KylinMountain]

Thanks @dzakwanalifi! Multi-provider support (Gemini included) now ships via LiteLLM (#168, merged) — Gemini works with a gemini/<model> model name, no extra code needed. Closing as superseded, but thanks for pushing on this early! 🙏

[dzakwanalifi]

Problem

PageIndex currently only supports OpenAI models, limiting user choice and potentially increasing costs.

Solution

Add unified interface supporting both OpenAI GPT-4 and Gemini 2.5 Flash models with structured output capabilities.

Key Changes

• Add LLM provider abstraction layer in utils.py
• Support structured output with Pydantic models for Gemini
• Add provider configuration in config.yaml
• Fix JSON parsing error handling in page_index.py
• Update function signatures for better model parameter handling

Code Changes

• utils.py: Added LLMProvider class with unified interface
• config.yaml: Added provider configuration option
• page_index.py: Enhanced error handling and Gemini integration

Use Case

Users can now choose between providers:

# OpenAI (default)
python run_pageindex.py --pdf_path doc.pdf
# Gemini
python run_pageindex.py --pdf_path doc.pdf --provider gemini

This provides flexibility for different cost/performance requirements while maintaining full compatibility.

[dgallitelli]

Great work on this multi-provider abstraction! 🎉

This pattern would make it straightforward to add additional cloud providers. I've opened two related feature requests that could build on this PR:

• Feature Request: Add Amazon Bedrock support for indexing and retrieval #104 - Amazon Bedrock support (Claude, Amazon Nova, Llama via AWS)
• Feature Request: Add Amazon SageMaker AI support for indexing and retrieval #105 - Amazon SageMaker AI support (self-hosted/custom models)

Both would follow the same LLMProvider interface pattern you've established here. The main implementation considerations:

For Bedrock:

• Uses boto3 with the Converse API
• Stop reason mapping: end_turn → stop, max_tokens → length
• Message format: {"content": [{"text": "..."}]}

For SageMaker:

• Uses invoke_endpoint with custom payload format (varies by model container)
• Enables self-hosted open models (Llama, Mistral, etc.)

Would you be open to expanding this PR to include AWS providers, or would separate PRs be preferred? Happy to contribute the Bedrock/SageMaker implementations if helpful.

[KylinMountain]

Thanks @dzakwanalifi! Multi-provider support (Gemini included) now ships via LiteLLM (#168, merged) — Gemini works with a gemini/<model> model name, no extra code needed. Closing as superseded, but thanks for pushing on this early! 🙏

[dzakwanalifi]

Problem

PageIndex currently only supports OpenAI models, limiting user choice and potentially increasing costs.

Solution

Add unified interface supporting both OpenAI GPT-4 and Gemini 2.5 Flash models with structured output capabilities.

Key Changes

• Add LLM provider abstraction layer in utils.py
• Support structured output with Pydantic models for Gemini
• Add provider configuration in config.yaml
• Fix JSON parsing error handling in page_index.py
• Update function signatures for better model parameter handling

Code Changes

• utils.py: Added LLMProvider class with unified interface
• config.yaml: Added provider configuration option
• page_index.py: Enhanced error handling and Gemini integration

Use Case

Users can now choose between providers:

# OpenAI (default)
python run_pageindex.py --pdf_path doc.pdf
# Gemini
python run_pageindex.py --pdf_path doc.pdf --provider gemini

This provides flexibility for different cost/performance requirements while maintaining full compatibility.

[dgallitelli]

Great work on this multi-provider abstraction! 🎉

This pattern would make it straightforward to add additional cloud providers. I've opened two related feature requests that could build on this PR:

• Feature Request: Add Amazon Bedrock support for indexing and retrieval #104 - Amazon Bedrock support (Claude, Amazon Nova, Llama via AWS)
• Feature Request: Add Amazon SageMaker AI support for indexing and retrieval #105 - Amazon SageMaker AI support (self-hosted/custom models)

Both would follow the same LLMProvider interface pattern you've established here. The main implementation considerations:

For Bedrock:

• Uses boto3 with the Converse API
• Stop reason mapping: end_turn → stop, max_tokens → length
• Message format: {"content": [{"text": "..."}]}

For SageMaker:

• Uses invoke_endpoint with custom payload format (varies by model container)
• Enables self-hosted open models (Llama, Mistral, etc.)

Would you be open to expanding this PR to include AWS providers, or would separate PRs be preferred? Happy to contribute the Bedrock/SageMaker implementations if helpful.

[KylinMountain]

Thanks @dzakwanalifi! Multi-provider support (Gemini included) now ships via LiteLLM (#168, merged) — Gemini works with a gemini/<model> model name, no extra code needed. Closing as superseded, but thanks for pushing on this early! 🙏

[dzakwanalifi]

Problem

PageIndex currently only supports OpenAI models, limiting user choice and potentially increasing costs.

Solution

Add unified interface supporting both OpenAI GPT-4 and Gemini 2.5 Flash models with structured output capabilities.

Key Changes

• Add LLM provider abstraction layer in utils.py
• Support structured output with Pydantic models for Gemini
• Add provider configuration in config.yaml
• Fix JSON parsing error handling in page_index.py
• Update function signatures for better model parameter handling

Code Changes

• utils.py: Added LLMProvider class with unified interface
• config.yaml: Added provider configuration option
• page_index.py: Enhanced error handling and Gemini integration

Use Case

Users can now choose between providers:

# OpenAI (default)
python run_pageindex.py --pdf_path doc.pdf
# Gemini
python run_pageindex.py --pdf_path doc.pdf --provider gemini

This provides flexibility for different cost/performance requirements while maintaining full compatibility.

[dgallitelli]

Great work on this multi-provider abstraction! 🎉

This pattern would make it straightforward to add additional cloud providers. I've opened two related feature requests that could build on this PR:

• Feature Request: Add Amazon Bedrock support for indexing and retrieval #104 - Amazon Bedrock support (Claude, Amazon Nova, Llama via AWS)
• Feature Request: Add Amazon SageMaker AI support for indexing and retrieval #105 - Amazon SageMaker AI support (self-hosted/custom models)

Both would follow the same LLMProvider interface pattern you've established here. The main implementation considerations:

For Bedrock:

• Uses boto3 with the Converse API
• Stop reason mapping: end_turn → stop, max_tokens → length
• Message format: {"content": [{"text": "..."}]}

For SageMaker:

• Uses invoke_endpoint with custom payload format (varies by model container)
• Enables self-hosted open models (Llama, Mistral, etc.)

Would you be open to expanding this PR to include AWS providers, or would separate PRs be preferred? Happy to contribute the Bedrock/SageMaker implementations if helpful.

[KylinMountain]

Thanks @dzakwanalifi! Multi-provider support (Gemini included) now ships via LiteLLM (#168, merged) — Gemini works with a gemini/<model> model name, no extra code needed. Closing as superseded, but thanks for pushing on this early! 🙏

[dzakwanalifi]

Problem

PageIndex currently only supports OpenAI models, limiting user choice and potentially increasing costs.

Solution

Add unified interface supporting both OpenAI GPT-4 and Gemini 2.5 Flash models with structured output capabilities.

Key Changes

• Add LLM provider abstraction layer in utils.py
• Support structured output with Pydantic models for Gemini
• Add provider configuration in config.yaml
• Fix JSON parsing error handling in page_index.py
• Update function signatures for better model parameter handling

Code Changes

• utils.py: Added LLMProvider class with unified interface
• config.yaml: Added provider configuration option
• page_index.py: Enhanced error handling and Gemini integration

Use Case

Users can now choose between providers:

# OpenAI (default)
python run_pageindex.py --pdf_path doc.pdf
# Gemini
python run_pageindex.py --pdf_path doc.pdf --provider gemini

This provides flexibility for different cost/performance requirements while maintaining full compatibility.

[dgallitelli]

Great work on this multi-provider abstraction! 🎉

This pattern would make it straightforward to add additional cloud providers. I've opened two related feature requests that could build on this PR:

• Feature Request: Add Amazon Bedrock support for indexing and retrieval #104 - Amazon Bedrock support (Claude, Amazon Nova, Llama via AWS)
• Feature Request: Add Amazon SageMaker AI support for indexing and retrieval #105 - Amazon SageMaker AI support (self-hosted/custom models)

Both would follow the same LLMProvider interface pattern you've established here. The main implementation considerations:

For Bedrock:

• Uses boto3 with the Converse API
• Stop reason mapping: end_turn → stop, max_tokens → length
• Message format: {"content": [{"text": "..."}]}

For SageMaker:

• Uses invoke_endpoint with custom payload format (varies by model container)
• Enables self-hosted open models (Llama, Mistral, etc.)

Would you be open to expanding this PR to include AWS providers, or would separate PRs be preferred? Happy to contribute the Bedrock/SageMaker implementations if helpful.

[KylinMountain]

Thanks @dzakwanalifi! Multi-provider support (Gemini included) now ships via LiteLLM (#168, merged) — Gemini works with a gemini/<model> model name, no extra code needed. Closing as superseded, but thanks for pushing on this early! 🙏

[dzakwanalifi]

Problem

PageIndex currently only supports OpenAI models, limiting user choice and potentially increasing costs.

Solution

Add unified interface supporting both OpenAI GPT-4 and Gemini 2.5 Flash models with structured output capabilities.

Key Changes

• Add LLM provider abstraction layer in utils.py
• Support structured output with Pydantic models for Gemini
• Add provider configuration in config.yaml
• Fix JSON parsing error handling in page_index.py
• Update function signatures for better model parameter handling

Code Changes

• utils.py: Added LLMProvider class with unified interface
• config.yaml: Added provider configuration option
• page_index.py: Enhanced error handling and Gemini integration

Use Case

Users can now choose between providers:

# OpenAI (default)
python run_pageindex.py --pdf_path doc.pdf
# Gemini
python run_pageindex.py --pdf_path doc.pdf --provider gemini

This provides flexibility for different cost/performance requirements while maintaining full compatibility.

[dgallitelli]

Great work on this multi-provider abstraction! 🎉

This pattern would make it straightforward to add additional cloud providers. I've opened two related feature requests that could build on this PR:

• Feature Request: Add Amazon Bedrock support for indexing and retrieval #104 - Amazon Bedrock support (Claude, Amazon Nova, Llama via AWS)
• Feature Request: Add Amazon SageMaker AI support for indexing and retrieval #105 - Amazon SageMaker AI support (self-hosted/custom models)

Both would follow the same LLMProvider interface pattern you've established here. The main implementation considerations:

For Bedrock:

• Uses boto3 with the Converse API
• Stop reason mapping: end_turn → stop, max_tokens → length
• Message format: {"content": [{"text": "..."}]}

For SageMaker:

• Uses invoke_endpoint with custom payload format (varies by model container)
• Enables self-hosted open models (Llama, Mistral, etc.)

Would you be open to expanding this PR to include AWS providers, or would separate PRs be preferred? Happy to contribute the Bedrock/SageMaker implementations if helpful.

[KylinMountain]

Thanks @dzakwanalifi! Multi-provider support (Gemini included) now ships via LiteLLM (#168, merged) — Gemini works with a gemini/<model> model name, no extra code needed. Closing as superseded, but thanks for pushing on this early! 🙏

[dzakwanalifi]

Problem

PageIndex currently only supports OpenAI models, limiting user choice and potentially increasing costs.

Solution

Add unified interface supporting both OpenAI GPT-4 and Gemini 2.5 Flash models with structured output capabilities.

Key Changes

• Add LLM provider abstraction layer in utils.py
• Support structured output with Pydantic models for Gemini
• Add provider configuration in config.yaml
• Fix JSON parsing error handling in page_index.py
• Update function signatures for better model parameter handling

Code Changes

• utils.py: Added LLMProvider class with unified interface
• config.yaml: Added provider configuration option
• page_index.py: Enhanced error handling and Gemini integration

Use Case

Users can now choose between providers:

# OpenAI (default)
python run_pageindex.py --pdf_path doc.pdf
# Gemini
python run_pageindex.py --pdf_path doc.pdf --provider gemini

This provides flexibility for different cost/performance requirements while maintaining full compatibility.

[dgallitelli]

Great work on this multi-provider abstraction! 🎉

This pattern would make it straightforward to add additional cloud providers. I've opened two related feature requests that could build on this PR:

• Feature Request: Add Amazon Bedrock support for indexing and retrieval #104 - Amazon Bedrock support (Claude, Amazon Nova, Llama via AWS)
• Feature Request: Add Amazon SageMaker AI support for indexing and retrieval #105 - Amazon SageMaker AI support (self-hosted/custom models)

Both would follow the same LLMProvider interface pattern you've established here. The main implementation considerations:

For Bedrock:

• Uses boto3 with the Converse API
• Stop reason mapping: end_turn → stop, max_tokens → length
• Message format: {"content": [{"text": "..."}]}

For SageMaker:

• Uses invoke_endpoint with custom payload format (varies by model container)
• Enables self-hosted open models (Llama, Mistral, etc.)

Would you be open to expanding this PR to include AWS providers, or would separate PRs be preferred? Happy to contribute the Bedrock/SageMaker implementations if helpful.

[KylinMountain]

Thanks @dzakwanalifi! Multi-provider support (Gemini included) now ships via LiteLLM (#168, merged) — Gemini works with a gemini/<model> model name, no extra code needed. Closing as superseded, but thanks for pushing on this early! 🙏