0 Comments Posted in:

In this post I want to show how you can use the cross-platform Azurite Azure Storage emulator running as a Docker container to develop Durable Functions locally.

Background

You may know that for many years there has been an Azure Storage Emulator that can be used for local development of Azure Functions on Windows. This is great as it saves you from having to create a real Azure Storage account whenever you want to experiment with Azure Functions.

This is especially valuable if you are developing Durable Functions because they make use of a wide range of Azure Storage features including blobs, queues and table storage in order to implement task hubs.

More recently Azurite, which is cross platform, and open source has emerged as the successor to the old Windows-only storage emulator. The official documentation states:

Azurite is the future storage emulator platform. Azurite supersedes the Azure Storage Emulator. Azurite will continue to be updated to support the latest versions of Azure Storage APIs.

However, until recently Azurite has been lacking a few important features that make it suitable to fully replace the older storage emulator. In particular, Azurite did not support Table Storage until recently, making it unsuitable for use with Durable Functions.

The good news is that there is now preview support for Table storage in Azurite. You can check up in this issue to find out if it has gone GA yet. Fortunately it appears to be good enough already to support local development of Durable Functions (and apparently Logic Apps too).

Running Azurite as a Container

There are several ways you can install and run Azurite including a VS Code extension or just with a simple npm install -g azurite.

But I wanted to try it out as a container, so I started it with the following command. Note that I've included port 10002 which is the port used by table storage, which is now supported by Azurite, and necessary for Durable Functions.

docker run -d -p 10000:10000 -p 10001:10001 -p 10002:10002 mcr.microsoft.com/azure-storage/azurite

Note that in this example I'm not mounting a volume. I could do that by adding another parameter of -v c:/azurite:/data, which would allow my the contents of my emulated storage account to persist independently of the lifetime of the container. But I find that often I'm using storage emulator for a quick throwaway experiment, and so it's nice to be able to easily clean up once I'm done.

Testing it out with Durable Functions

You can try all this out very easily if you have the Azure Functions Core Tools installed.

  1. In an empty folder run func init and select dotnet as the runtime.
  2. Then run then func new and select the DurableFunctionsOrchestration template
  3. Make sure Azurite is running (you can use the docker run command shown above)
  4. Run the function app with func start
  5. Call the starter function URI to initiate a new workflow
  6. Call the statusQueryGetUri returned by the starter function to check on the workflow progress. You should see that the workflow has completed already.

Summary

If you're developing Azure Functions on a Mac or Linux, then it's definitely worth checking out Azurite. And even if you're developing on Windows and have been happy with the existing Storage Emulator, it's probably time to consider switching over to Azurite, as it is approaching feature parity, and going forwards is going to become the officially supported and recommended emulator.

Want to learn more about how easy it is to get up and running with Durable Functions? Be sure to check out my Pluralsight course Azure Durable Functions Fundamentals.

0 Comments Posted in:

Following on from my post about what Bicep is, I wanted to provide an example of a Bicep template, and an Azure Function app seems like a good choice, as it requires us to create several resources. There's the Function App itself, but also we need a Storage Account, an App Service Plan, and an Application Insights instance.

There's a helpful example here in the official docs, but I wanted to build my own to learn more about the syntax, and to try to produce a minimal template that was customized for my needs.

Parameters and Variables

A Bicep file can have parameters, and those parameters can be given default values. For my template I wanted the user to supply an "app name" which would be the name used for the Function App. The location is also a parameter, but it defaults to the location of the resource group.

But you can also create variables based on the values of the parameters. In this example, I'm using the appName variable to generate names for the Storage Account, Hosting Plan and App Insights instance. You can see that I've used the uniqueString method to get unique names for various resources and also the substring method to keep the Storage Account name within the valid limits.

param appName string
param location string = resourceGroup().location

// storage accounts must be between 3 and 24 characters in length and use numbers and lower-case letters only
var storageAccountName = '${substring(appName,0,10)}${uniqueString(resourceGroup().id)}' 
var hostingPlanName = '${appName}${uniqueString(resourceGroup().id)}'
var appInsightsName = '${appName}${uniqueString(resourceGroup().id)}'
var functionAppName = '${appName}'

Storage Account

Next, we specify the Storage Account resource. We can give it an identifier (storageAccount) to refer to it later in the template, and Visual Studio Code extension for Bicep will give us autocomplete to greatly simplify defining this resource. On top of name and location, which are set to variable values, the only other things to set up are the Storage Account kind, and pricing tier (known as "sku" in these templates), and again the VS Code autocomplete points us in the right direction for these.

resource storageAccount 'Microsoft.Storage/[email protected]' = {
  name: storageAccountName
  location: location
  kind: 'StorageV2'
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
}

App Insights

Application Insights has always been a bit of a pain to automate the creation of, and it seems that although Bicep offers some help, this was the bit I struggled with the most.

We need to say that the kind is web for Function Apps, and I discovered after some failed attempts that you do need to provide the properties section for this to work. I simply copied the contents of properties from another template I found, but wasn't sure what to put for WorkspaceResourceId so I just left that out. That's exactly the kind of guesswork I was talking about in my previous post that I'd like to see eliminated. It should be straightforward to discover which properties are and aren't needed for each Azure resource type, what their values mean, and what the defaults are if you leave them out.

Another annoyance is that the Azure Portal likes there to be a special tag on the App Insights resource which points to the Function App it is linked to. This is a bit of a pain to set up and hopefully the Portal can be improved in the future to deduce these connections without needing you to set them up explicitly.

resource appInsights 'Microsoft.Insights/[email protected]' = {
  name: appInsightsName
  location: location
  kind: 'web'
  properties: { 
    Application_Type: 'web'
    publicNetworkAccessForIngestion: 'Enabled'
    publicNetworkAccessForQuery: 'Enabled'
  }
  tags: {
    // circular dependency means we can't reference functionApp directly  /subscriptions/<subscriptionId>/resourceGroups/<rg-name>/providers/Microsoft.Web/sites/<appName>"
     'hidden-link:/subscriptions/${subscription().id}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Web/sites/${functionAppName}': 'Resource'
  }
}

App Service Plan

For the App Service Plan, we want to use the consumption plan, and this highlights the mismatch between the names that ARM gives things, and the names they are called in the official documentation. An App Service Plan is a "Server Farm" in ARM template speak, and the consumption plan is called "Dynamic" with the name "Y1".

resource hostingPlan 'Microsoft.Web/[email protected]' = {
  name: hostingPlanName
  location: location
  sku: {
    name: 'Y1' 
    tier: 'Dynamic'
  }
}

Function App

Finally we have the Function App itself, and this one is perhaps the most complex. Again I borrowed some of this from another sample I found, and this means that I have included a few settings (e.g. httpsOnly and clientAffinityEnabled) that probably aren't strictly necessary for a minimal template. The serverFarmId shows a good example of how easy it is to refer to another Bicep resource (the hostingPlan).

Another thing we need for a Function App is a few "app settings". We should provide the FUNCTIONS_EXTENSION_VERSION and the FUNCTIONS_WORKER_RUNTIME to match the code for our Function App. We also need to link the Function App to the App Insights instance with APPINSIGHTS_INSTRUMENTATIONKEY, which Bicep made really easy - I loved the fact that there was intellisense helping me to pick out this property.

There are two settings (AzureWebJobsStorage and WEBSITE_CONTENTAZUREFILECONNECTIONSTRING) that should contain the connection string of the Storage Account. The fact that we have to go to some lengths to construct this out of various pieces is a bit of an annoyance that I complained about in my previous post. Hopefully this will be simplified in future.

However, I did want to highlight a setting I am not including in the template. Function Apps usually have a WEBSITE_CONTENTSHARE app setting, but the documentation explicitly tells you not to include this in an ARM template, because it will get auto-generated. However, several other examples I saw for Bicep or ARM Azure Function App templates did include this setting and I presume that is because these templates were generated base on an existing Azure resource. This tends to end up with verbose templates that specify more than they need to (or should in this case). For that reason I think it would be nice if VS code offered autocomplete of minimal Bicep templates for each resource type, making it easier to see what you need to provide.

I also haven't set WEBSITE_RUN_FROM_PACKAGE to 1 which is what you would do if you want to use the "run from package" mode of publishing Function Apps (which is a good choice) and were planning to deploy the zips with az functionapp deployment source config-zip. However, if you are going to use func azure functionapp publish that will take care of this app setting for you.

The final thing I want to point out is the ability to configure dependencies with dependsOn. This is an ARM capability that allows us to ensure things get created in the right order (and also supports concurrent creation of resources that don't have dependencies on each other).

resource functionApp 'Microsoft.Web/[email protected]' = {
  name: functionAppName
  location: location
  kind: 'functionapp'
  properties: {
    httpsOnly: true
    serverFarmId: hostingPlan.id
    clientAffinityEnabled: true
    siteConfig: {
      appSettings: [
        {
          'name': 'APPINSIGHTS_INSTRUMENTATIONKEY'
          'value': appInsights.properties.InstrumentationKey
        }
        {
          name: 'AzureWebJobsStorage'
          value: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(storageAccount.id, storageAccount.apiVersion).keys[0].value}'
        }
        {
          'name': 'FUNCTIONS_EXTENSION_VERSION'
          'value': '~3'
        }
        {
          'name': 'FUNCTIONS_WORKER_RUNTIME'
          'value': 'dotnet'
        }
        {
          name: 'WEBSITE_CONTENTAZUREFILECONNECTIONSTRING'
          value: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(storageAccount.id, storageAccount.apiVersion).keys[0].value}'
        }
        // WEBSITE_CONTENTSHARE will also be auto-generated - https://docs.microsoft.com/en-us/azure/azure-functions/functions-app-settings#website_contentshare
        // WEBSITE_RUN_FROM_PACKAGE will be set to 1 by func azure functionapp publish
      ]
    }
  }

  dependsOn: [
    appInsights
    hostingPlan
    storageAccount
  ]
}

Deploying it with the Azure CLI

Deploying this Bicep template couldn't be easier if you already know how to deploy a regular ARM template with the Azure CLI. The latest version of the Azure CLI supports Bicep out of the box with the same command used for ARM templates (az deployment group create).

In this example I show first creating a resource group with az group create and then deploying our template to it with az deployment group create. I also show how we can provide a parameter to the deployment (appName in our example)

$RESOURCE_GROUP = "MyResourceGroup"
$APP_NAME = "myfunctionapp"
$BICEP_FILE = "deploy.bicep"
$LOCATION = "westeurope"

# create a resource group
az group create -n $RESOURCE_GROUP -l $LOCATION

# deploy the bicep file directly
az deployment group create `
  --name mybicepdeployment `
  --resource-group $RESOURCE_GROUP `
  --template-file $BICEP_FILE `
  --parameters "appName=$APP_NAME"

Modularity

A final thing to mention here is that the nice thing about Bicep is that you should only need to do this once. Now I've worked out how to create a Function App in Bicep, I can use this as a "module" in any other application that needs a Function App. This means that I can have even simpler top-level Bicep templates that express what I want in very generic terms (e.g. I need a Function App and a Cosmos DB database), and the lower-level templates which specify the details of how those are configured are not something I need to concern myself with if I'm happy to stick with the defaults I've used elsewhere.

Want to learn more about the Azure CLI? Be sure to check out my Pluralsight course Azure CLI: Getting Started.

0 Comments Posted in:

There is a huge variety of options in Azure for creating and configuring resources (e.g. creating a Function App or a Virtual Machine).

  1. Create resources manually in the Azure Portal (great for experimentation and learning)
  2. Create an ARM template (for declarative Infrastructure as Code, great for predictable deployment of production apps)
  3. Script creation of resources using the Azure CLI or Azure PowerShell (great for spinning up/tearing down dev/test resources)
  4. Use the Azure Resource Manager SDKs to create resources in your language of preference such as C# (great when your application itself needs to dynamically create Azure resources such as an Azure Container Instance)
  5. Use third party offerings such as Terraform or Pulumi (great when you need multi-cloud support or want to combine the best of Infrastructure as Code with custom logic)

So when Azure recently announced Bicep, you might be forgiven for wondering why we need yet another way to deploy resources in Azure.

What is Bicep?

Bicep is a "Domain Specific Language for deploying Azure resources declaratively". Probably the easiest way to think of it is that it's simply ARM templates with a much nicer syntax. Bicep code is transpiled to ARM templates. In other words if (like me) you avoid ARM templates in many situations because of their verbose and complex syntax, then Bicep offers a way to give you the strengths of ARM templates without their pain.

I'll do a followup post soon showing an example Bicep template for deploying an Azure Function app, but in this post I want to summarize what I like about Bicep, and also give some of my first impressions of how it can be improved going forwards.

Strengths

First of all, it certainly achieves the (not difficult) goal of being less verbose and easier for a human to read and write than an ARM template. If you've not seen a Bicep template before, here's the definition for a storage account:

param storageAccountName string
param location string = resourceGroup().location

resource storageAccount 'Microsoft.Storage/[email protected]' = {
  name: storageAccountName
  location: location
  kind: 'StorageV2'
  sku: {
    name: 'Standard_LRS'
    tier: 'Standard'
  }
}

Even better than just a nicer syntax, there is an excellent Visual Studio Code extension for Bicep that gives you syntax highlighting and auto-completion. This makes it very easy to discover the names of the required properties and assign the correct values.

I also really like the fact that Bicep is supported out of the box with the latest Azure CLI (and Azure PowerShell). This means you can directly deploy a Bicep template without the need to transpile it into an ARM template. Not only can it deploy, but it can also decompile an existing ARM template into a Bicep file (with az bicep decompile). So this gives you a great way to migrate existing templates into the newer syntax.

Another feature I like is that Bicep embraces modularity. You can create a Bicep file that defines your preferred way of configuring a storage account, or virtual machine. And then another Bicep template can include that definition, only needing to override parameters it wants to change. This means that over time, as you use Bicep more, you should find you can reuse pre-existing templates rather than needing to build everything up from scratch every time.

It's also nice that in the Bicep GitHub repo, you can find a generous collection of examples which should serve as a decent starting point for most of your needs.

Weaknesses

I was able to get an Azure Function App deployed with Bicep fairly easily (despite not knowing about the sample templates which would have saved me some time). However, the getting started experience did highlight some ways in which I think Bicep can still be improved.

First, I'd love to see the Azure Portal able to generate Bicep files instead of ARM templates. And for these to be as terse as possible, rather than the verbose overspecified monstrosities of ARM templates that the Portal currently generates. Even better, I'd love to see a mode for the Azure CLI where instead of actually creating a resource with (say az functionapp create), it emits a simple Bicep template to perform that operation, with the settings you specified parameterized. Since the Azure CLI calls the ARM APIs under the hood, I don't see why that would be too difficult.

Second, most Azure resource types have hundreds of configurable parameters, and if you look at auto-generated ARM templates you'll see values supplied for everything. This can be frustrating when you are working out which properties you do and don't need to include when defining a resource. My general preference is to only specify the properties only where I am deviating from the default or if I want to explicitly call out that a certain setting is important. I don't want my Bicep files cluttered with dozens of settings that I don't really understand but are in there because some other example template had them in. Really what is needed is a single place I can go to for excellent documentation of every property of an Azure resource type, including what its default value is if you don't specify it, and what the permitted values are. That doesn't seem to exist yet (or only in a very limited form).

Third, and this is a bugbear of mine with ARM templates in general, but the names things are called in the templates are often different to the names used in the official documentation. For example, the "consumption" "App Service Plan" is a "dynamic/Y1" "Server Farm" in ARM template-speak. This is unnecessarily confusing for people who don't know the historical names of Azure resources. I don't see why Bicep (or ARM) couldn't be extended to support aliases so that things can be referred to by their most current names.

Fourth, and this is also actually a limitation of ARM templates, but brings some unnecessary complexity into Bicep, and that is that some common operations seem to lack convenience functions. For example to get the connection string for a storage account, rather than there being a method or property I can directly reference, the whole thing has to be constructed with a bunch of string concatenation calling lower-level helpers. I don't see why Bicep (or ARM) couldn't make tasks like this much simpler.

value: 'DefaultEndpointsProtocol=https;AccountName=${storageAccount.name};EndpointSuffix=${environment().suffixes.storage};AccountKey=${listKeys(storageAccount.id, storageAccount.apiVersion).keys[0].value}'

Summary

Bicep is an excellent replacement for ARM templates. Anywhere you are using ARM templates, you should consider switching to Bicep. And although it is a huge improvement over ARM templates, I think there is still plenty of scope for Bicep to be made even easier to work with, so I look forward to seeing how the product evolves over the coming years.

Want to learn more about the Azure CLI? Be sure to check out my Pluralsight course Azure CLI: Getting Started.