Messaging with Azure Service Bus - Part 5

In this series:

In our last post, we saw some basic code examples of how to send and receive messages from an Azure Service Bus queue.

Today, I want to dig a bit deeper and explore some of the options we have for sending messages.

Message Metadata

In our demo to send a message, we simply created a Message object and set it's Body property to contain the payload of the message. But the Message object allows us to attach various additional metadata to our message. There are some standard properties, as well as the ability to specify arbitrary metadata as a series of key value pairs.

There's a helpful guide available here to many of the properties you can set on the Message object, so I won't try to exhaustively cover it, but I'll just pick out a few interesting properties.

1. ContentType

Every message has a ContentType, indicating what format the message payload is in. A typical value might be application/json as the format for this property should follow RFC2045.

2. Label

A message also has a Label property which you can set to whatever you like to indicate the "subject" of a message. It's completely up to you what you use this property for. You could use this to indicate what type of object was serialized into the message body, but I tend to use the user properties for that...

3. UserProperties

Each message also has a UserProperties dictionary of key value metadata, allowing you to attach any information to a message. Some examples of what you might store here are an identifier of what the message type is, allowing you to deserialize it into a strongly typed object, or even to set up filtered subscriptions that only listen for a specific message type (another nice feature of Service Bus). Or if you're implementing custom message compression or encryption, you could use the UserProperties metadata to indicate what algorithms or keys you are using to do this.

4. MessageId

Another interesting property is the MessageId - by default it will be set to a guid, but it is used for (optional) message de-duplication, so if you have a scenario where it's really important that the same message doesn't get sent twice by accident, then you should set the MessageId to something that would be the same for both "duplicate" messages. This isn't a feature I've made use of myself yet - generally I've tried to promote "idempotent" message handlers, as an alternative way of coping with the same message being delivered twice.

5. CorrelationId

Finally, I want to mention the CorrelationId - this can be used to identify that this message relates to some other message or long-running operation.

A common use of correlation ids is that when an end user initiates an action, a "correlation id" is generated - just a GUID, but from then on, all messages that are sent, or HTTP requests that are made as a result of that single action will use that same correlation id. That way, if all parts of the code ensure they write the correlation id to the logs, it's possible to piece together all the log messages relating to that single operation, even if they're spread out across multiple microservices.

It's a really useful and powerful technique, and I recommend you use it. Of course, you don't necessarily need to use this CorrelationId property - you might use a user property, or even put your correlation id in the message payload.

If you are using correlation ids, I recommend having a base message handler that automatically ensures the correlation id gets logged every time you log everything in that message handler, and also automatically attaches that correlation id to any outgoing messages. Some frameworks might be able to do this for you.

Scheduled Messages

One really powerful feature is the ability to schedule a message to be delivered in the future. This allows you to implement all kinds of interesting patterns like reminders, timeouts, or delayed retries. You can access this by setting the ScheduledEnqueueTimeUtc property on the Message object.

If you set the enqueue time to be say 1 hour in the future, that means that the message will not be visible to the receiver for an hour. Of course, you can't guarantee that the receiver will be listening in an hour - if the receiver happens to be offline then, the message will just be picked up when it starts polling again.

Demo - sending a scheduled message with metadata

I've updated my sender application (available here on GitHub) to set up a few message metadata properties and schedule the message to become visible five minutes into the future

var message = new Message(body);
message.CorrelationId = Guid.NewGuid().ToString();
message.Label = "My message";
message.UserProperties["Sender"] = "Mark";
message.ScheduledEnqueueTimeUtc = DateTime.UtcNow.AddMinutes(5);
await queueClient.SendAsync(message);

And send it in the usual way:

dotnet run $connectionString "scheduled message"

I also updated my receiver application to output a few of the metadata properties

Console.WriteLine("Got a message:");
Console.WriteLine(messageText);

Console.WriteLine($"Enqueued at {m.SystemProperties.EnqueuedTimeUtc}");
Console.WriteLine($"CorrelationId: {m.CorrelationId}");
Console.WriteLine($"ContentType: {m.ContentType}");
Console.WriteLine($"Label: {m.Label}");
Console.WriteLine($"MessageId: {m.MessageId}");
foreach(var prop in m.UserProperties)
{
    Console.WriteLine($"{prop.Key}={prop.Value}");
}

And now when I run the receiver application and wait five minutes for the scheduled message to appear.

dotnet run $connectionString

We can see the correlation id, label and user metadata I supplied, as well as the fact that a MessageId was generated automatically for me:

Got a message:
scheduled message
Enqueued at 28/04/2020 06:54:33
CorrelationId: 872ecaeb-82ee-467e-b8ba-9a016f3a3668
ContentType:
Label: My message
MessageId: 22865392df49461f86fe39e9ff9fdf2c
Sender=Mark

Summary

As we've seen in this post, Azure Service Bus offers us several ways to attach useful metadata to the messages we send, which allow the message handler to know how to correctly interpret the message and provide us useful logging and diagnostic information. We also saw how the scheduled message capability can be really useful to request that an action is performed in the future. In the next installment, we'll look at some of the options for how we handle messages.