If you was at my recent training about extending Dynamics 365 Business Central with Azure services in Microsof Italy or if you attend some of my sessions about Dataverse extensions, I think you already know that I’m a huge fan of using Azure Service Bus for handling messages in the cloud.
Azure Service Bus is a fully managed enterprise message broker with message queues and publish-subscribe topics (in a namespace). Service Bus is used to decouple applications and services from each other, providing the following benefits:
- Load-balancing work across competing workers
- Safely routing and transferring data and control across service and application boundaries
- Coordinating transactional work that requires a high-degree of reliability
One of the Azure Service Bus features that I often use when architecturing serverless applications are topics. While a queue is often used for point-to-point communication, topics are useful in publish/subscribe scenarios:
In simple words, messages coming to an Azure Service Bus can be decored with a topic and a subscriber to a topic can receive a copy of each message sent to that topic.
Azure Service Bus supports two tiers: Standard and Premium.
The Standard tier offers:
- variable throughput
- variable latency
- pay as you go pricing
- message size up to 256 kb
The Premium tier offers:
- high throughput
- high and predictable performances
- fixed pricing
- dynamic scaling (up and down)
- message size up to 100 MB
- resource isolation at CPU and memory level (each workload runs in full isolation)
The Premium tier is recommended for production workloads when you have an high volume of messages to exchange or when you need to exchange big messages (price is high compared to standard tier).
As standard behaviour, all the messages sent to azure Service Bus are stored in a single message store and all the requests are handled by a single message broker.
For performances and reliability of message exchangements, Azure Service Bus also supports partitioning. Service Bus partitions enable messaging entities to be partitioned across multiple message brokers. This means that the overall throughput of a partitioned entity is no longer limited by the performance of a single message broker. Additionally, a temporary outage of a message broker, for example during an upgrade, does not render a partitioned queue or topic unavailable, as messages will be retried on a different partition.
Uisng partitioning is recommended if you need a reliable service broker in the cloud. But Azure Service Bus partitions was supported only in the Standard tier, until now.
The big news is that now also Azure Service Bus in the Premium tier suppors partitioning. This feature is currently (at the time of writing this post) available only in the East US and South Central US regions, with other regions being added during the public preview phase.
How can you enable partitioning?
You can enable partitioning in your Azure Service Bus namespace during creation. Partitioning is only available at entity creation for namespaces in the Premium SKU. Any previously existing partitioned entities in Premium namespaces continue to work as expected and it’s not possible to change the partitioning option on any existing namespace.
First, you need to create an Azure Service Bus instance in the Premium tier:
Then you need to go to the Advanced tab and here set Partitioning to enable and specify the number of partitions to create:
Why partitioning can help?
Imagine the following scenario: we have N Dynamics 365 Business Central instances (N different tenants but this is valid also in a scenario where you have a single tenant with N companies) and every tenant need to send sales orders to a service bus queue. Then N external applications (1 for BC tenant/company) pulls orders from that service bus queue accordingly to a topic (each applications pulls the sales orders for the relative tenant/company) in order to manage them.
Scenario is as follows:
In this scenario, you could have two problems:
- If the volume of messages is high, having a single service bus (single message broker) can degrade performances.
- If there’s a failure on a message broker, you will have a stop in message exchangement.
By using partitioning, you can create N partitions in your Azure Service Bus namespace and this can guarantee high performances and reliability. In addition, a message sent from Business Central to the Azure Service Bus can be decored with a partition key, so messages with the same partiton key will be stored in the same partiton in Azure Service Bus.
When a message is enqueued into a partitioned queue or topic, Service Bus checks for the presence of a partition key. If it finds one, it selects the partition based on that key. If it doesn’t find a partition key, it selects the partition based on an internal algorithm.
The new scenario is as follows:
Each external application can now pull orders from their partitions.
Just for a reference, in C# a message with a partiton key can be sent as follows:
ServiceBusMessage msg = new ServiceBusMessage("This is a message");
msg.PartitionKey = "myPartitionKey";
await sender.SendMessageAsync(msg);
Without using a partition key, Azure Service Bus distributes messages in a round-robin fashion to all the partitions of the partitioned queue or topic.
Stefano Demiliani 