When working with Azure Functions, in real-world projects could happen that you need to connect to resources in an Azure virtual network (VNET) from your function app. The easy way to do that is hosting an Azure function on a Premium plan.
Premium plan hosting provides the following benefits to your functions:
- Avoid cold starts with warm instances.
- Virtual network connectivity.
- Supports longer runtime durations.
- Choice of Premium instance sizes.
- More predictable pricing, compared with the Consumption plan.
- High-density app allocation for plans with multiple function apps.
In the Premium plan, you can have your app always ready on a specified number of instances. Your app runs continuously on those instances, regardless of load. If load exceeds what your always ready instances can handle, additional instances are added as necessary, up to your specified maximum.
You can configure the number of always ready instances in the Azure portal by selected your Function App, going to the Settings tab and selecting the Scale out (App service plan) option:
In this post I want to report a problem discovered in these days in a big customer project. They have an Azure Function app running in an Elastic Premium Plan (EP1) and connected to a VNET. The App Service plan was set to scale out from a minimum of 4 to a maximum of 20 instances:
The function app was used by lots of serverless applications (mainly for receiving and manipulating tons of external data before inserting them into the ERP system). When the serverless process was activated in production and the Azure Function app was under an heavy load, we start monitoring the performances of the process and we saw something like the following:
The Azure Function app seems not scaling correctly, we have never more than 4 to 6 instances running also when the load is high.
Monitoring the App Service CPU (via the High CPU Analysis in the Diagnose and solve problems section) it shows something like this:
and this is a clear signal that the App Service plan is not scaling out accordingly when the load increases.
Why the scaling is not working accordingly to the settings we’ve done in the Premium plan?
The VNET integration was the key here!
When you create an Azure Function in a Premium plan connected to a VNET, if you go to Configuration in your function app and you select Function runtime settings, you will have the Runtime Scale Monitoring setting set to Off:
When using Azure Functions with a VNET integration, it’s important to enable the Runtime Scale Monitoring feature if you want scaling:
This was the crucial point.
Azure Functions uses a component called Scale Controller to monitor the rate of events and determine whether to scale out or scale in:
When the Azure Function is paired with a VNET, the Scale Controller hangs and it’s not able to handle scaling accordingly. By enabling this feature the scaling decisions are managed by the Azure Functions host (that is inside the VNET). In this way the Scale Controller is able to understand what type of scaling to apply. This feature must be enabled to allow the function app to be triggered by resources in a VNET or by custom triggers.
Just by turning on this flag the scaling problem was solved.
Remember that to enable this feature you need to have the Contributor access at your App Service plan for which you have created the function app.
Stefano Demiliani 