CreatingAnAgent.md

Creating an Agent

An agent is a code library that interfaces with the AI Shell to talk to a specific large language model or other assistance provider. Users chat with the agents using natural language to get the desired output or assistance. Agents are implemented as C# classes that implement the ILLMAgent interface from the AIShell.Abstraction package.

For details about the AIShell.Abstraction layer and AIShell.Kernel, see the AI Shell architecture documentation.

Prerequisites

• .NET 8 SDK or newer
• PowerShell 7.4 or newer

Steps to create an agent

For this example we create an agent to communicate with the language model phi3 by utilizing Ollama, a CLI tool for managing and using locally built LLM/SLMs. The complete source code of the agent can be found in the shell/agents/AIShell.Ollama.Agent folder of the repository. the repository.

Step 1: Create a new project

Currently, the only way to import an agent is for it to be included in the folder structure of this repository. We suggest creating an agent under the shell/agents folder. Create a new folder with the prefix AIShell.<AgentName>. Within that folder, create a new C# project with the same name. Run the following command from the folder where you want to create the agent:

dotnet new classlib

Step 2: Add the necessary packages

Within the newly created project, add a reference to the AIShell.Abstraction package. To reduce the number of files created by the build, you can disable the generation of PDB and deps.json for release builds.

Your .csproj file should contain the following elements:

<ProjectSdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net8.0</TargetFramework>
<ImplicitUsings>enable</ImplicitUsings>
<SuppressNETCoreSdkPreviewMessage>true</SuppressNETCoreSdkPreviewMessage>
</PropertyGroup>
<ItemGroup>
<PackageReferenceInclude="AIShell.Abstraction"Version="0.1.0-alpha.11">
<ExcludeAssets>contentFiles</ExcludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
</ItemGroup>
</Project>

Important

Be sure to replace the version number with the latest version of the package. That can be found in the shell/shell.common.props file.

Step 3: Modify the build script

Modify the build script so that you can build and test your agent during development. The build.ps1 script is located in the root of the repository. This script builds the kernel and all agents. The following lines were added to the script to build the new agent.

$ollama_agent_dir=Join-Path$agent_dir"AIShell.Ollama.Agent"$ollama_out_dir=Join-Path$app_out_dir"agents""AIShell.Ollama.Agent"if ($LASTEXITCODE-eq0-and$AgentToInclude-contains'ollama') {
Write-Host"`n[Build the Ollama agent ...]`n"-ForegroundColor Green
$ollama_csproj= GetProjectFile $ollama_agent_dir
dotnet publish $ollama_csproj-c $Configuration-o $ollama_out_dir
}

Be sure to put this code after definition of the $agent_dir, $app_out_dir, and $AgentToInclude. Also add the name of the agent to the $AgentToInclude array and parameter validation.

$AgentToInclude??=@('openai-gpt','interpreter','ollama')

Step 3: Implement the agent class

To being the creation of the agent, modify the Class1.cs file to implement the ILLMAgent interface. We suggest renaming the file to OllamaAgent.cs and then rename class to OllamaAgent. We've also added some packages that are used by the code in the implementation.

usingSystem.Diagnostics;usingSystem.Text;usingSystem.Text.Json;usingAIShell.Abstraction;namespaceAIShell.Ollama.Agent;publicsealedclassOllamaAgent:ILLMAgent{}

Step 4: Add necessary class members and methods

Next, implement the necessary variables and methods of the agent class. The comments provide descriptions of the members of the OllamaAgent class. The _chatService member is an instance of the OllamaChatService class. The implementation of the OllamaChatService class is show in later steps.

publicsealedclassOllamaAgent:ILLMAgent{/// <summary>/// The name of the agent/// </summary>publicstringName=>"ollama";/// <summary>/// The description of the agent to be shown at start up/// </summary>publicstringDescription=>"This is an AI assistant that utilizes the Ollama CLI tool. Be sure to follow all prerequisites in aka.ms/ollama/readme";/// <summary>/// This is the company added to /like and /dislike verbiage for who the telemetry helps./// </summary>publicstringCompany=>"Microsoft";/// <summary>/// These are samples that are shown at start up for good questions to ask the agent/// </summary>publicList<string>SampleQueries=>["How do I list files in a given directory?"];/// <summary>/// These are any optional legal/additional information links you want to provide at start up/// </summary>publicDictionary<string,string>LegalLinks{privateset;get;}/// <summary>/// This is the chat service to call the API from/// </summary>privateOllamaChatService_chatService;/// <summary>/// A string builder to render the text at the end/// </summary>privateStringBuilder_text;/// <summary>/// Dispose method to clean up the unmanaged resource of the chatService/// </summary>publicvoidDispose(){_chatService?.Dispose();}/// <summary>/// Initializing function for the class when the shell registers an agent/// </summary>/// <param name="config">Agent configuration for any configuration file and other settings</param>publicvoidInitialize(AgentConfigconfig){_text=newStringBuilder();_chatService=newOllamaChatService();LegalLinks=new(StringComparer.OrdinalIgnoreCase){["Ollama Docs"]="https://github.com/ollama/ollama",["Prerequisites"]="https://aka.ms/ollama/readme"};}/// <summary>/// Get commands that an agent can register to the shell when being loaded/// </summary>publicIEnumerable<CommandBase>GetCommands()=>null;/// <summary>/// Gets the path to the setting file of the agent./// </summary>publicstringSettingFile{privateset;get;}=null;/// <summary>/// Refresh the current chat by starting a new chat session./// An agent can reset chat states in this method./// </summary>publicvoidRefreshChat(){}/// <summary>/// Gets a value indicating whether the agent accepts a specific user action feedback./// </summary>/// <param name="action">The user action.</param>publicboolCanAcceptFeedback(UserActionaction)=>false;/// <summary>/// A user action was taken against the last response from this agent./// </summary>/// <param name="action">Type of the action.</param>/// <param name="actionPayload"></param>publicvoidOnUserAction(UserActionPayloadactionPayload){}/// <summary>/// Main chat function that takes/// </summary>/// <param name="input">The user input from the chat experience</param>/// <param name="shell">The shell that provides host functionality</param>/// <returns>Task Boolean that indicates whether the query was served by the agent.</returns>publicasyncTask<bool>Chat(stringinput,IShellshell){}}

For the initial implementation, we want the agent to return "Hello World!" to prove that we have create the correct interfaces. We will also add a try-catch block to catch any expections to handle when the user tries to cancel the operation.

Add the following code to your Chat method.

publicasyncTask<bool>Chat(stringinput,IShellshell){// Get the shell hostIHosthost=shell.Host;// get the cancelation tokenCancellationTokentoken=shell.CancellationToken;try{host.RenderFullResponse("Hello World!");}catch(OperationCanceledExceptione){_text.AppendLine(e.ToString());host.RenderFullResponse(_text.ToString());returnfalse;}returntrue;}

Step 5: Build your agent

At this point its good to try building and testing the agent. See if you get Hello World! when you ask a question.

Use the following command to build the agent:

../../build.ps1

To test the agent, run the aish you just built. The build script puts the path to aish on the clipboard. Paste the path from the clipboard into your terminal application. Select your agent from the list of agents presented by aish.

AI Shell
v0.1.0-alpha.11
Please select an agent to use:
interpreter
>ollama
openai-gpt

After selecting the agent, enter a question in the chat window. You should see the response "Hello World!".

Step 6: Add ollama check

Next, we want to add a check to make sure ollama is running.

publicasyncTask<bool>Chat(stringinput,IShellshell){// Get the shell hostIHosthost=shell.Host;// get the cancellation tokenCancellationTokentoken=shell.CancellationToken;if(Process.GetProcessesByName("ollama").Lengthis0){host.RenderFullResponse("Please be sure the ollama is installed and server is running. Check all the prerequisites in the README of this agent are met.");returnfalse;}// Calls to the API will go herereturntrue;}

Step 7: Create data structures to exchange data with the Chat Service

Before we can use the ollama API, we need to create classes that handle input to and responses from the ollama API. To find more information about the API call we're going to make see, this The following ollama example shows the format of the input and the response from the agent.

For this example we call the ollama API with streaming disabled. This generates a single, fixed response. In the future we could add streaming capabilities so that responses could be rendered in real time, as the agent receives them.

To defined the data structures, create a new file called OllamaSchema.cs in the same folder.

namespaceAIShell.Ollama.Agent;// Query class for the data to send to the endpointinternalclassQuery{publicstringprompt{get;set;}publicstringmodel{get;set;}publicboolstream{get;set;}}// Response data schemainternalclassResponseData{publicstringmodel{get;set;}publicstringcreated_at{get;set;}publicstringresponse{get;set;}publicbooldone{get;set;}publicstringdone_reason{get;set;}publicint[]context{get;set;}publicdoubletotal_duration{get;set;}publiclongload_duration{get;set;}publicintprompt_eval_count{get;set;}publicintprompt_eval_duration{get;set;}publicinteval_count{get;set;}publiclongeval_duration{get;set;}}internalclassOllamaResponse{publicintStatus{get;set;}publicstringError{get;set;}publicstringApi_version{get;set;}publicResponseDataData{get;set;}}

Now we have the pieces we need to construc a chat service that communicates using the ollama API. A separate chat service class isn't required but can be helpful to abstract the calls to the API.

Create a new file called OllamaChatService.cs in the same folder as the agent. For this example, we are using a hard coded endpoint and model for the ollama API. In the future, we could add these as parameters in an agent configuration file.

usingSystem.Net.Http.Headers;usingSystem.Text;usingSystem.Text.Json;usingAIShell.Abstraction;namespaceAIShell.Ollama.Agent;internalclassOllamaChatService:IDisposable{/// <summary>/// Ollama endpoint to call to generate a response/// </summary>internalconststringEndpoint="http://localhost:11434/api/generate";/// <summary>/// Http client/// </summary>privatereadonlyHttpClient_client;/// <summary>/// Initialization method to initialize the http client/// </summary>internalOllamaChatService(){_client=newHttpClient();}/// <summary>/// Dispose of the http client/// </summary>publicvoidDispose(){_client.Dispose();}/// <summary>/// Preparing chat with data to be sent/// </summary>/// <param name="input">The user input from the chat experience</param>/// <returns>The HTTP request message</returns>privateHttpRequestMessagePrepareForChat(stringinput){// Main data to send to the endpointvarrequestData=newQuery{model="phi3",prompt=input,stream=false};varjson=JsonSerializer.Serialize(requestData);vardata=newStringContent(json,Encoding.UTF8,"application/json");varrequest=newHttpRequestMessage(HttpMethod.Post,Endpoint){Content=data};returnrequest;}/// <summary>/// Getting the chat response async/// </summary>/// <param name="context">Interface for the status context used when displaying a spinner.</param>/// <param name="input">The user input from the chat experience</param>/// <param name="cancellationToken">The cancellation token to exit out of request</param>/// <returns>Response data from the API call</returns>internalasyncTask<ResponseData>GetChatResponseAsync(IStatusContextcontext,stringinput,CancellationTokencancellationToken){try{HttpRequestMessagerequest=PrepareForChat(input);HttpResponseMessageresponse=await_client.SendAsync(request,cancellationToken);response.EnsureSuccessStatusCode();context?.Status("Receiving Payload ...");Console.Write(response.Content);varcontent=awaitresponse.Content.ReadAsStreamAsync(cancellationToken);returnJsonSerializer.Deserialize<ResponseData>(content);}catch(OperationCanceledException){// Operation was cancelled by user.}returnnull;}}

Step 8: Call the chat service

Now we can call the chat service in the main agent class.

Modify the Chat method to call the chat service and render the response to the user. The following code shows the completed Chat method.

publicasyncTask<bool>Chat(stringinput,IShellshell){// Get the shell hostIHosthost=shell.Host;// get the cancellation tokenCancellationTokentoken=shell.CancellationToken;if(Process.GetProcessesByName("ollama").Lengthis0){host.RenderFullResponse("Please be sure the ollama is installed and server is running. Check all the prerequisites in the README of this agent are met.");returnfalse;}ResponseDataollamaResponse=awaithost.RunWithSpinnerAsync(status:"Thinking ...",func:async context =>await_chatService.GetChatResponseAsync(context,input,token)).ConfigureAwait(false);if(ollamaResponseis not null){// render the contenthost.RenderFullResponse(ollamaResponse.response);}returntrue;}

Congratulations! The agent is now complete. You can build and test the agent to confirm it's working. Compare your code to the example code in the shell/agents/AIShell.Ollama.Agent folder to see if you missed a step.

How can I share my own agent?

Currently there is no way to share your agents in a centralized repository. We suggest forking this repository for development of your own agent. You can share a link your fork in the Agent Sharing section of the Discussions tab of this repository. To use an agent, if you put its dll files in the agents folder of the base directory of aish.exe, the agent will be loaded by aish.