Posted in:

AI opens up a lot of exciting possibilities for developers to use its capabilities within their own applications, but it can feel a bit overwhelming getting started. You have a lot of choices around which models to use and how best to interact with them.

In this post I want to go through the steps to implement a basic chat with an LLM hosted in Azure using C#, by taking advantage of the new Microsoft.Extensions.AI NuGet package, which aims to simplify the process.

There's a helpful tutorial available here, but there were a few permissions-related issues I ran into along the way, so I thought I'd document how I got it working myself.

Step 1 - Create an Azure Open AI Model Deployment

To start with, we need to create an AI model "deployment" in Azure. This can be done fairly simply in the Azure portal.

First, create an "Azure OpenAI Service" resource, and then within it, you need to create a model "deployment". This is one of the first points where it gets a bit confusing as there are several models to choose from and its not at all obvious if you are new to AI how to decide which one is the most appropriate. Essentially you're trading off cost, performance and capabilities, and for this tutorial, gpt-4o-mini would be a good choice.

You'll need to take note of the Open AI service endpoint, which will be something like https://my-openai.openai.azure.com/, as well as the name of your deployment, e.g. "my-gpt-4o-mini".

Step 2 - Grant yourself the necessary permissions to use the model

If you want to use EntraID for authenticating, then you'll need to grant yourself the necessary permissions to use the model deployment. This can be done by visiting the "Access control (IAM)" section of the portal for the OpenAI service and adding a role assignment of Cognitive Services OpenAI Contributor to your user (I originally tried the "Cognitive Services OpenAI User" role but that didn't seem to be enough although I'm not sure why not).

Step 3 - Call the model from C#

In a C# console app, add references to the latest versions of the following packages:

<PackageReference Include="Microsoft.Extensions.AI.OpenAI" Version="9.0.1-preview.1.24570.5" />
<PackageReference Include="Azure.AI.OpenAI" Version="2.1.0" />
<PackageReference Include="Azure.Identity" Version="1.13.1" />

And then to actually connect to our Azure OpenAI service we can use DefaultAzureCredential. I find I often need to explicitly specify which tenant my resources are in, so I've shown how to do that with DefaultAzureCredentialOptions. You may also need to log into Azure with the Azure CLI using az login before this credential will work for you.

var endpoint = "https://my-openai.openai.azure.com/";
var modelId = "my-gpt-4o-mini"; // this is the deployment name
var credential = new DefaultAzureCredential(new DefaultAzureCredentialOptions() {  
    TenantId = "123e4567-e89b-12d3-a456-426614174000" });
IChatClient client =
    new AzureOpenAIClient(
        new Uri(endpoint),
        credential)
            .AsChatClient(modelId);

And now submitting a question to the chat client is very straightforward:

var response = await client.CompleteAsync("Should I learn about AI in 2025?");
Console.WriteLine(response.Message);

I'll leave you to guess what the AI said in response!

Using key-based authentication

If you'd prefer to use key-based authentication, then you can access the key for your OpenAI service from the Azure portal. Dotnet secrets would be a good way to make your key available for local development, which you can set up by running the following commands in your project folder.

dotnet user-secrets init
dotnet user-secrets set "AzureOpenAI:ApiKey" "your-api-key-here"
dotnet add package Microsoft.Extensions.Configuration
dotnet add package Microsoft.Extensions.Configuration.UserSecrets
dotnet add package Microsoft.Extensions.Configuration.EnvironmentVariables

And then in your Program.cs file, you can get the secret from configuration and construct an ApiKeyCredential with it.

var builder = new ConfigurationBuilder()
    .AddUserSecrets<Program>()
    .AddEnvironmentVariables();
var configuration = builder.Build();
var apiKey = configuration["AzureOpenAI:ApiKey"];
var apiKeyCredential = new ApiKeyCredential(apiKey);

This can then be used instead of the DefaultAzureCredential, and would be useful if you needed to run your application from somewhere that you can't use managed identities.

System prompt and chat history

Finally for this tutorial, you probably want to be able to have a full conversation with the model, and might want to also change the "system" prompt, which allows you to give instructions to your model about what it is intended to do and how it should respond to questions.

This is achieved by building up a collection of ChatMessage objects, which includes the system prompt, along with all of the user's messages, but also should include the response objects generated by the model itself as they form an important part of the context of the ongoing conversation.

Here we're initializing a list of chat history messages with a system prompt.

List<ChatMessage> chatHistory = new()
    {
        new ChatMessage(ChatRole.System, """
            You are an enthusiastic Arsenal supporter.
        """)
    };

And now, in a loop, we create a new ChatRole.User message from the user's console input, and pass the whole thing into CompleteStreamingAsync which allows us to print the output from the model as it returns. This improves the perceived responsiveness, as the user can start reading the model's response while it is still being generated.

Finally, before looping round again, we put the whole response into the chat history list as a ChatRole.Assistant message.

while (true)
{
    // Get user prompt and add to chat history
    Console.WriteLine("Your prompt:");
    var userPrompt = Console.ReadLine();
    chatHistory.Add(new ChatMessage(ChatRole.User, userPrompt));

    // Stream the AI response and add to chat history
    Console.WriteLine("AI Response:");
    var response = "";
    await foreach (var item in
        chatClient.CompleteStreamingAsync(chatHistory))
    {
        Console.Write(item.Text);
        response += item.Text;
    }
    chatHistory.Add(new ChatMessage(ChatRole.Assistant, response));
    Console.WriteLine();
}

If we test this out, we can see that it does indeed take into account the system prompt.

Your prompt:
Who is the greatest striker?
AI response:
As an Arsenal supporter, I have to give a special nod to Thierry Henry when discussing the greatest strikers! His combination of skill, pace, and goal-scoring ability made him a legend at the club. With over 200 goals for Arsenal and numerous accolades, including two Premier League titles and a stint as the club's all-time leading scorer, he undoubtedly left an indelible mark on both Arsenal and the Premier League as a whole. However, opinions may vary, and players like Pele, Diego Maradona, and more recently Lionel Messi and Cristiano Ronaldo often come up in the conversation as well. But for me, Henry will always be the greatest! What do you think?

Summary

Obviously these are just the first steps to programmatically calling an AI model in C#, but the great thing about Microsoft.AI.Extensions is that it is designed to integrate with many different AI services, including using OpenAI's own API, or running a local model. At the moment there doesn't seem to be support for some of the other major AI models such as Anthropic Claude or Google Gemini but I expect that will come in due course.

I've been experimenting with a few more features of the library as well, so hoping to be able to share some other findings soon. I'd love to hear in the comments what you've built using APIs for AI services.