Containerized Activities in Durable Workflows

This is the fourth part in a series about how we can build a serverless workflow using Azure Durable Functions, but implement some of the activities in that workflow using containers with Azure Container Instances. Today we'll look at using the Azure .NET SDK to automate the creation of ACI Container Groups from C# using the Azure .NET SDK. We'll also see how to retrieve the status of an existing container group and delete one.

Current table of contents:

Reference the Azure .NET SDK

For this demo, I'm going to use the fluent Azure management APIs available in the Microsoft.Azure.Management.Fluent NuGet Package

We can add that to our .csproj file with:

<PackageReference Include="Microsoft.Azure.Management.Fluent" Version="1.20.0" />

Authenticate with Azure

Next, we need to authenticate with Azure using the managed identity we created in part 2. I'd also like us to be able to optionally use a regular service principal with a client id and secret, which we might want to use for testing locally, or if we ever needed this code from a place that couldn't use managed identities.

In the GetAzure function below we either use environment variables to create a ServicePrincipalLoginInformation object to log in directly with a tenant id, client id and secret, or (preferably) we use managed identities with MSILoginInformation (MSI was the old name for managed identities). Once we've generated the credentials object we want to use, we log in with the Authenticate fluent method, and then I optionally select a subscription to use if a subscription id has been provided as an environment variable.

private static readonly IAzure azure = GetAzure();

private static IAzure GetAzure()
{
    var tenantId = Environment.GetEnvironmentVariable("TenantId");
    var clientId = Environment.GetEnvironmentVariable("ClientId");
    var clientSecret = Environment.GetEnvironmentVariable("ClientSecret");
    AzureCredentials credentials;

    if (!string.IsNullOrEmpty(tenantId) && 
        !string.IsNullOrEmpty(clientId) &&
        !string.IsNullOrEmpty(clientSecret))
    {
        var sp = new ServicePrincipalLoginInformation
        {
            ClientId = clientId,
            ClientSecret = clientSecret
        };
        credentials = new AzureCredentials(sp, tenantId, AzureEnvironment.AzureGlobalCloud);
    }
    else
    {
        credentials = SdkContext
            .AzureCredentialsFactory
            .FromMSI(new MSILoginInformation(MSIResourceType.AppService), 
                     AzureEnvironment.AzureGlobalCloud);
    }
    var authenticatedAzure = Azure
        .Configure()
        .WithLogLevel(HttpLoggingDelegatingHandler.Level.Basic)
        .Authenticate(credentials);
    var subscriptionId = System.Environment.GetEnvironmentVariable("SubscriptionId");
    if (!string.IsNullOrEmpty(subscriptionId))
        return authenticatedAzure.WithSubscription(subscriptionId);
    return authenticatedAzure.WithDefaultSubscription();
}

Create an ACI Container Group

Next, let's create an ACI container group. Although there is a fluent API for managing container instances, it can be a bit of a pain to work with if you want run-time flexibility over the exact configuration of your container group. I think there are some lower-level APIs that I could have used instead, but for now I've stuck with the fluent API.

I started by creating my own simple class to define the parameters for my container group. There's a bunch of stuff missing (like operating system, CPU and memory choice, credentials if you're using an image with ACR), so I'll probably update this sample code in the future with more capabilities, but this was the minimum feature set I needed for my experiment.

public class ContainerGroupDefinition
{
    public string ResourceGroupName { get; set; }
    public string ContainerGroupName { get; set; }
    public string ContainerImage { get; set;}
    public string CommandLine { get; set; }
    public List<AzureFileShareVolumeDefinition> AzureFileShareVolumes { get; set; }
    public Dictionary<string, string> EnvironmentVariables { get; set; }
}

Next, I actually create the "container group" (in ACI, you always create a "container group" even if it will often only have a single container in it - it's like a "pod" in Kubernetes).

The first step is to get the region to put the container group in - I'm asking for the region of the resource group. Then I use the fluent API to specify all the characteristics of the container group I want, such as the operating system (Linux at the moment) and details of the container instances in the container group. We're only supporting a single container instance, and I've hard-coded it to expose port 80, have 1 CPU and 1GB RAM, and use a public (i.e. Docker hub) image. So there's plenty more work needed to make this completely generic.

However, I have added some of my own "optional" customizations. You can optionally mount a volume, and you can optionally override the container instance startup command line. I achieved that by creating some additional fluent extension methods of my own, because fluent APIs are very bad at supporting optional steps.

Once everything has been defined about the container group, we create it with CreateAsync.

public static async Task RunTaskBasedContainer(
            ContainerGroupDefinition cg)
{
    // Get the resource group's region
    IResourceGroup resGroup = await azure.ResourceGroups.GetByNameAsync(cg.ResourceGroupName);
    Region azureRegion = resGroup.Region;
    // Create the container group
    var vol = cg.AzureFileShareVolumes?.FirstOrDefault();
    var containerGroupDefinition = azure.ContainerGroups.Define(cg.ContainerGroupName)
        .WithRegion(azureRegion)
        .WithExistingResourceGroup(cg.ResourceGroupName)
        .WithLinux()
        .WithPublicImageRegistryOnly()
        .WithOptionalVolume(vol)
        .DefineContainerInstance(cg.ContainerGroupName + "-1")
            .WithImage(cg.ContainerImage)
            .WithExternalTcpPort(80)
            .WithCpuCoreCount(1.0)
            .WithMemorySizeInGB(1)
            .WithOptionalVolumeMount(vol)
            .WithOptionalCommandLine(log, cg.CommandLine)
            .WithEnvironmentVariables(cg.EnvironmentVariables)
            .Attach()
        .WithDnsPrefix(cg.ContainerGroupName)
        .WithRestartPolicy(ContainerGroupRestartPolicy.Never);

    var containerGroup = await containerGroupDefinition.CreateAsync();
}

Here's an example of one of my "optional" extension methods. You can find them in the AciHelpers class in my demo GitHub repo.

private static IWithContainerInstanceAttach<IWithNextContainerInstance> WithOptionalVolumeMount(this IWithContainerInstanceAttach<IWithNextContainerInstance> baseDefinition, AzureFileShareVolumeDefinition vol)
{
    if (vol != null)
        baseDefinition = baseDefinition.WithVolumeMountSetting(vol.VolumeName, vol.MountPath);
    return baseDefinition;
}

This extension uses another simple DTO I created to define the settings for a file share:

public class AzureFileShareVolumeDefinition
{
    public string StorageAccountName { get; set; }
    public string StorageAccountKey { get; set; }
    public string ShareName { get; set; }
    public string VolumeName { get; set; }
    public string MountPath { get; set; }
}

One other thing worth pointing out is that the CreateAsync method will succeed if the container group already exists. I haven't experimented too much yet with what happens if it already exists but with different configuration, so it's probably worth putting in additional checks for existence before you create the container group.

Retrieving Container Group Status

Another important requirement for my application is to be able to monitor a container group to see when the container instances have finished.

Again I defined my own simplified DTOs to store information about the current state of a container group. These are not strictly necessary as you may be happy to use the types in the Azure SDK as is.

public class ContainerGroupStatus
{
    public ContainerInstanceStatus[] Containers { get; set;}
    public string State { get; set; }
    public string Id { get; set; }
    public string Name { get; set; }
    public string ResourceGroupName { get; set; }
}

public class ContainerInstanceStatus
{
    public string Name { get; set; }
    public string Image { get; set; }
    public IList<string> Command { get; set; }
    public ContainerState CurrentState { get; set; }
    public int? RestartCount { get; set; }
}

And we can then use ContainerGroups.GetByResourceGroupAsync to get information about that container group and I just project the returned information into my own custom type.

public static async Task<ContainerGroupStatus> GetContainerGroupStatus(
                ContainerGroupDefinition cg)
{
    var containerGroup = await azure.ContainerGroups
        .GetByResourceGroupAsync(cg.ResourceGroupName, cg.ContainerGroupName);

    var status = new ContainerGroupStatus() {
        State = containerGroup.State,
        Id = containerGroup.Id,
        Name = containerGroup.Name,
        ResourceGroupName = containerGroup.ResourceGroupName,
        Containers = containerGroup.Containers.Values.Select(c => 
            new ContainerInstanceStatus() {
                Name = c.Name,
                Image = c.Image,
                Command = c.Command,
                CurrentState = c.InstanceView.CurrentState,
                RestartCount = c.InstanceView.RestartCount,
            }).ToArray()
    };
    return status;
}

I can then check if a specific container instance has exited by looking for the value Terminated in the current state:

if(containerGroupStatus.Containers[0]?.CurrentState?.State == "Terminated")

Deleting Container Groups

Finally, I want to be able to delete container groups to save cost or kill them if they have hung while performing their task. That's super easy with the Azure management SDKs - we just need to call DeleteByResourceGroupAsync.

public static async Task DeleteContainerGroup(
    ContainerGroupDefinition cg)
{
    await azure.ContainerGroups.DeleteByResourceGroupAsync(
            cg.ResourceGroupName, cg.ContainerGroupName);
}

Summary

The Azure .NET management SDK makes it easy to create and manage Azure Container Instances from C#. We can authenticate either with a service principal id and secret directly, or preferably with a managed service identity if its available. The fluent nature of the API makes things a little tricky if we want to support optional steps, but we were able to work around that with some custom extension methods.

With these capabilities in place, we're now finally ready to create a Durable Functions workflow that creates, monitors and then deletes the container instances on demand. We'll see how to do that in the next part.