Setting up a project and model in AI Foundry

Let’s start by creating a project in https://ai.azure.com/

Note: I’m going to create a very simple, pretty standard chatbot for a pizza delivery service, so my project is going be called pizza, so you’re see this in the code but ofcourse replace with your preferred example or real code as this is the same setup that you’ll do for your own chatbot anyway.

• Navigate to https://ai.azure.com/
• Click Create new (if not available go to the Management Center | All Resources and the option should be there)
• Select the Azure AI Foundry resource, then click Next
• Supply a project name, resource group (or create one) and region – I left this as Sweden Central as I’m sure I read that it was a little more advanced than some regions, but do pick one which suites.
• Click Create

Once completed you’ll be presented with the project page.

We’re not quite done as we need to deploy a model…

• From the left hand nav. bar, locate My assets and click on Models + endpoints.
• Click + Deploy model
• Select Deploy base model from the drop down
• From the Select a model popup, choose a mode, I’ve selected gpt-4o-mini which is a good model for chat completion.
• Click Confirm
• Give it a Deployment name and I’ve using the Deployment type as Standard and leaving all the new fields that appear as default
• Click Deploy to assign the model to the project

We should now see some source code samples listed. We’ll partially be using in the code part of this, but before we move on we need an endpoint and an api key.

• From this page on the Details tab copy the Endpoint Target URI – but we don’t need the whole this, from the project overview we can get the cutdown version but it’s basically this https://{your project}.cognitiveservices.azure.com/
• From below the Target URI copy the Key

Writing the code

Create a Console application using Visual Studio.

Let’s begin be adding the following NuGet packages

dotnet add package Microsoft.SemanticKernel
dotnet add package Microsoft.Extensions.Configuration
dotnet add package Microsoft.Extensions.Configuration.Json

We’re using (as you can see) Semantic Kernel, now the versions seem to change pretty quickly at the moment so hopefully the code below will work but if not check against the version you’re using. For completeness here’s my versions

<ItemGroup>
  <PackageReference Include="Microsoft.Extensions.Configuration" Version="9.0.10" />
  <PackageReference Include="Microsoft.Extensions.Configuration.Json" Version="9.0.10" />
  <PackageReference Include="Microsoft.SemanticKernel" Version="1.66.0" />
</ItemGroup>

Create yourself an appsettings.json file, which should look like this

{
  "AI": {
    "Endpoint": "<The Endpoint>",
    "ApiKey": "<Api Key>",
    "ApiVersion": "2024-12-01-preview",
    "DeploymentName":  "pizza"
  }
}

Obviously you’ll need to supply your endpoint and API key that we copied after creating our AI Foundry project.

Now before we go onto look at implementing the Program.cs… I’m wanting this LLM to use some custom functions to fulfil a couple of tasks such as returning the menu and placing and order.

The AI Foundry project is an LLM which is our chat bot and it can answer questions and also generate hallucinations etc. For example without supplying my functions it will try to create a pizza menu for me, but that’s not a lot of use to our pizza place.

What I want is the Natural Language Processing (NLP) as well as the model’s “knowledge” to work with my functions – we implement this using Plugins.

What I want to happens is this

• The customer connects to the chatbot/LLM
• The customer then asks to either order a Pizza or for information on what Pizza’s we make, i.e. the menu
• The LLM then needs to pass information to the PizzaPlugin which then returns information to the LLM to respond to the customer

Our PizzaPlugin is a standard C# class and we’re going to keep things simple, but you can imagine that this could call into a database or whatever you like to to get a menu and place an order.

public class PizzaPlugin
{
    [KernelFunction]
    [Description("Use this function to list the pizza's a customer can order")]
    public string ListMenu() => "We offer Meaty Feasty, Pepperoni, Veggie, and Cheese pizzas.";

    [KernelFunction]
    public string PlaceOrder(string pizzaType)
        => $"Order placed for: {pizzaType}. It will be delivered in 30 minutes.";
}

The KernelFunctionAttribute is registered/added to the Semantic Kernal to supply callable plugin functions. The DescriptionAttribute is optional, but recommended if you want the LLM to understand what the function does during auto function calling (which we will be using). I’ve left the other function without this DescriptionAttribute just to demonstrate it’s not required in this case, yet our function will/should still be called. If we have many similar functions this would be a helpful addition.

Note: Try to also function names that are clearly stating their usage, i.e. use action oriented naming.

Now let’s implement the Program.cs where we’ll, read in our configuration from the appsettings.json and then create the Semantic Kernel, add the Azure Open AI Chat services, add the plugin we just created then call into the AI Foundry LLM model we created earlier.

We’re NOT going create all the code for an actual console based chat app, hence we’ll just predefine the “chat” part with a ChatHistory object. In a real world you may wish to keep track of the chat history.

using Microsoft.Extensions.Configuration;
using Microsoft.SemanticKernel;
using Microsoft.SemanticKernel.ChatCompletion;
using OpenAI.Chat;
using SemanticKernelTest.PizzaPlugin;

var config = new ConfigurationBuilder()
    .AddJsonFile("appsettings.json")
    .Build();

var endpoint = config["AI:Endpoint"];
var apiKey = config["AI:ApiKey"];
var apiVersion = config["AI:ApiVersion"];
var deploymentName = config["AI:DeploymentName"];

var builder = Kernel.CreateBuilder();

builder.AddAzureOpenAIChatCompletion(
    deploymentName: deploymentName,
    endpoint: endpoint,
    apiKey: apiKey,
    apiVersion: apiVersion
);

var kernel = builder.Build();

var plugin = new PizzaPlugin();
kernel.Plugins.AddFromObject(plugin);

var chatCompletion = kernel.GetRequiredService<IChatCompletionService>();

var chatHistory = new ChatHistory();
chatHistory.AddAssistantMessage("How can I help you?");
chatHistory.AddUserMessage("Can I order a plan Pepperoni pizza?");

var result = await chatCompletion.GetChatMessageContentAsync(chatHistory, new PromptExecutionSettings
{
    FunctionChoiceBehavior = FunctionChoiceBehavior.Auto()
}, kernel);

Console.WriteLine(result.Content);

Before you run this code, place breakpoints on the Kernel Functions in the plugin and then run the code. Hopefully all run’s ok and you’ll notice that the LLM (via Semantic Kernel) calls into the plugin methods. As you’ll hopefully see – it calls the menu to check whether the pizza supplied is one we make then orders it, if it does exist. Change the pizza to one we do not make (for example Chicken) and watch the process and output.

More settings

In the code above we’re using the PromptExecutionSettings but we can also use OpenAIPromptExecutionSettings instead, from this we can configure Open AI by setting the Temperature, MaxTokens and others, for example

var result = await chatCompletion.GetChatMessageContentAsync(chatHistory, new OpenAIPromptExecutionSettings
{
  Temperature = 0.7,
  FunctionChoiceBehavior = FunctionChoiceBehavior.Auto(),
  MaxTokens = 100
}, kernel);

These options are also settable in the AI Foundry. Temperature controls the randomness of the model, for example a lower value is more deterministic whereas the higher is more random the results are, the default is 1.0.

• 0.2-0.5 is more deterministic and produces more focused outputs
• 0.8-1.0 allows for more diverse and creative responses