outlines-haystack

---

Table of Contents

• 🛠️ Installation
• 📃 Description
• 💻 Usage

🛠️ Installation

pip install outlines-haystack

📃 Description

Outlines is a Python library that allows you to use Large Language Model in a simple and robust way (with structured generation). It is built by .txt.

-- Outlines docs

This library allow you to use outlines generators in your Haystack pipelines!

This library currently supports the following generators:

• JSON: generate a JSON object with a given schema
• Choices: generate text from a list of options. Useful for classification tasks!
• Text: simply generate text
• Regex: ⚠️ coming soon
• Format: ⚠️ coming soon
• Grammar: ⚠️ coming soon

outlines supports a wide range of models and frameworks, we are currently supporting:

• OpenAI/Azure OpenAI
• 🤗 Transformers
• llama-cpp
• mlx-lm

💻 Usage

Tip

See the Example Notebooks for complete examples.

All below examples only use the transformers models.

JSON Generation

>>>frompydanticimportBaseModel>>>fromoutlines_haystack.generators.transformersimportTransformersJSONGenerator>>>classUser(BaseModel):
... name: str
... last_name: str>>>generator=TransformersJSONGenerator(
... model_name="microsoft/Phi-3-mini-4k-instruct",
... schema_object=User,
... device="cuda",
... )
>>>generator.warm_up()
>>>generator.run(prompt="Create a user profile with the fields name, last_name")
{'structured_replies': ['{"name": "John", "last_name": "Doe"}']}

Choice Generation

>>>fromoutlines_haystack.generators.transformersimportTransformersChoiceGenerator>>>generator=TransformersChoiceGenerator(
... model_name="microsoft/Phi-3-mini-4k-instruct",
... choices=["Positive", "Negative"],
... device="cuda",
... )
>>>generator.warm_up()
>>>generator.run(prompt="Classify the following statement: 'I love pizza'")
{'choice': 'Positive'}

Text Generation

Tip

While outlines supports classic text generation, it excels at structured generation. For text generation, consider using Haystack's built-in text generators that offer more features.

>>>fromoutlines_haystack.generators.transformersimportTransformersTextGenerator>>>generator=TransformersTextGenerator(
... model_name="microsoft/Phi-3-mini-4k-instruct",
... device="cuda",
... )
>>>generator.warm_up()
>>>generator.run(prompt="What is the capital of Italy?")
{'replies': ['The capital of Italy is Rome.']}

License

outlines-haystack is distributed under the terms of the MIT license.