Instructor: Structured LLM Outputs

Instructor is a Python library that makes it a breeze to work with structured outputs from large language models (LLMs). Built on top of Pydantic, it provides a simple, transparent, and user-friendly API to manage validation, retries, and streaming responses. Get ready to supercharge your LLM workflows!

Key Features

• Response Models: Specify Pydantic models to define the structure of your LLM outputs
• Retry Management: Easily configure the number of retry attempts for your requests
• Validation: Ensure LLM responses conform to your expectations with Pydantic validation
• Streaming Support: Work with Lists and Partial responses effortlessly
• Flexible Backends: Seamlessly integrate with various LLM providers beyond OpenAI

Get Started in Minutes

Install Instructor with a single command:

pip install -U instructor

Now, let's see Instructor in action with a simple example:

importinstructorfrompydanticimportBaseModelfromopenaiimportOpenAI# Define your desired output structureclassUserInfo(BaseModel):
name: strage: int# Patch the OpenAI clientclient=instructor.from_openai(OpenAI())
# Extract structured data from natural languageuser_info=client.chat.completions.create(
model="gpt-3.5-turbo",
response_model=UserInfo,
messages=[{"role": "user", "content": "John Doe is 30 years old."}],
)
print(user_info.name)
#> John Doeprint(user_info.age)
#> 30

Using Anthropic Models

importinstructorfromanthropicimportAnthropicfrompydanticimportBaseModelclassUser(BaseModel):
name: strage: intclient=instructor.from_anthropic(Anthropic())
# note that client.chat.completions.create will also workresp=client.messages.create(
model="claude-3-opus-20240229",
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
assertisinstance(resp, User)
assertresp.name=="Jason"assertresp.age==25

Using Cohere Models

Make sure to install cohere and set your system environment variable with export CO_API_KEY=<YOUR_COHERE_API_KEY>.

pip install cohere

importinstructorimportcoherefrompydanticimportBaseModelclassUser(BaseModel):
name: strage: intclient=instructor.from_cohere(cohere.Client())
# note that client.chat.completions.create will also workresp=client.chat.completions.create(
model="command-r-plus",
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
assertisinstance(resp, User)
assertresp.name=="Jason"assertresp.age==25

Using Litellm

importinstructorfromlitellmimportcompletionfrompydanticimportBaseModelclassUser(BaseModel):
name: strage: intclient=instructor.from_litellm(completion)
resp=client.chat.completions.create(
model="claude-3-opus-20240229",
max_tokens=1024,
messages=[
{
"role": "user",
"content": "Extract Jason is 25 years old.",
}
],
response_model=User,
)
assertisinstance(resp, User)
assertresp.name=="Jason"assertresp.age==25

Type are inferred correctly

This was the dream of instructor but due to the patching of openai, it wasnt possible for me to get typing to work well. Now, with the new client, we can get typing to work well! We've also added a few create_* methods to make it easier to create iterables and partials, and to access the original completion.

Calling create

importopenaiimportinstructorfrompydanticimportBaseModelclassUser(BaseModel):
name: strage: intclient=instructor.from_openai(openai.OpenAI())
user=client.chat.completions.create(
model="gpt-4-turbo-preview",
messages=[
{"role": "user", "content": "Create a user"},
],
response_model=User,
)

Now if you use a IDE, you can see the type is correctly inferred.

Handling async: await create

This will also work correctly with asynchronous clients.

importopenaiimportinstructorfrompydanticimportBaseModelclient=instructor.from_openai(openai.AsyncOpenAI())
classUser(BaseModel):
name: strage: intasyncdefextract():
returnawaitclient.chat.completions.create(
model="gpt-4-turbo-preview",
messages=[
{"role": "user", "content": "Create a user"},
],
response_model=User,
)

Notice that simply because we return the create method, the extract() function will return the correct user type.

Returning the original completion: create_with_completion

You can also return the original completion object

importopenaiimportinstructorfrompydanticimportBaseModelclient=instructor.from_openai(openai.OpenAI())
classUser(BaseModel):
name: strage: intuser, completion=client.chat.completions.create_with_completion(
model="gpt-4-turbo-preview",
messages=[
{"role": "user", "content": "Create a user"},
],
response_model=User,
)

Streaming Partial Objects: create_partial

In order to handle streams, we still support Iterable[T] and Partial[T] but to simply the type inference, we've added create_iterable and create_partial methods as well!

importopenaiimportinstructorfrompydanticimportBaseModelclient=instructor.from_openai(openai.OpenAI())
classUser(BaseModel):
name: strage: intuser_stream=client.chat.completions.create_partial(
model="gpt-4-turbo-preview",
messages=[
{"role": "user", "content": "Create a user"},
],
response_model=User,
)
foruserinuser_stream:
print(user)
#> name=None age=None#> name=None age=None#> name=None age=None#> name=None age=None#> name=None age=25#> name=None age=25#> name=None age=25#> name=None age=25#> name=None age=25#> name=None age=25#> name='John Doe' age=25# name=None age=None# name='' age=None# name='John' age=None# name='John Doe' age=None# name='John Doe' age=30

Notice now that the type inferred is Generator[User, None]

Streaming Iterables: create_iterable

We get an iterable of objects when we want to extract multiple objects.

importopenaiimportinstructorfrompydanticimportBaseModelclient=instructor.from_openai(openai.OpenAI())
classUser(BaseModel):
name: strage: intusers=client.chat.completions.create_iterable(
model="gpt-4-turbo-preview",
messages=[
{"role": "user", "content": "Create 2 users"},
],
response_model=User,
)
foruserinusers:
print(user)
#> name='John' age=30#> name='Jane' age=25# User(name='John Doe', age=30)# User(name='Jane Smith', age=25)

Evals

We invite you to contribute to evals in pytest as a way to monitor the quality of the OpenAI models and the instructor library. To get started check out the evals for anthropic and OpenAI and contribute your own evals in the form of pytest tests. These evals will be run once a week and the results will be posted.

Contributing

If you want to help, checkout some of the issues marked as good-first-issue or help-wanted found here. They could be anything from code improvements, a guest blog post, or a new cookbook.

CLI

We also provide some added CLI functionality for easy convinience:

• instructor jobs : This helps with the creation of fine-tuning jobs with OpenAI. Simple use instructor jobs create-from-file --help to get started creating your first fine-tuned GPT3.5 model

• instructor files : Manage your uploaded files with ease. You'll be able to create, delete and upload files all from the command line

• instructor usage : Instead of heading to the OpenAI site each time, you can monitor your usage from the cli and filter by date and time period. Note that usage often takes ~5-10 minutes to update from OpenAI's side

License

This project is licensed under the terms of the MIT License.

Contributors