In these hours Dynamics 365 Business Central 2023 Wave 1 will join the public preview state and together with this upcoming new release also telemetries will receive interesting updates.
As usual, Kennie has released a summary of the monthly updates here and you can also start downloading the March update of the Power BI telemetry app.
With this post I want to put the evidence on something interesting.
When calling Business Central web services, you can soon inject information about the caller into telemetry by setting the HTTP User-Agent header and this will be logged with the httpHeaders dimension (for OData and API calls).
In your client, you can do like in the following code:
using var client = new HttpClient();
client.DefaultRequestHeaders.Add("User-Agent", "SDApp");
or better:
var httpClient = new HttpClient();
var request = new HttpRequestMessage(HttpMethod.Get, "https://businesscenyral.dynamics.com/api/...");
request.Headers.UserAgent.Add(new ProductInfoHeaderValue("SDApp", "1.0"));
Also setting the HTTP header client-request-id, will log requests with the httpHeaders dimension (for OData and API calls) and it will also set the “OperationId”/ClientActivity in Application Insights:
HttpClient client = new HttpClient();
var requestId = Guid.NewGuid();
client.DefaultRequestHeaders.Add("X-Request-Id", requestId.ToString());
The updated Power BI app adds an interesting analysis for Monthly/Weekly/Daily users (MAU/WAU/DAU) based on the user_Id column in Application Insights telemetries. This gives you the unique amount of users who are “active” within a given amount of time and thism permits you to have a measure for your user engagement.
The above analysis is created starting from a KQL query like the following:
let dau_mau = traces | extend aadTenantId = tostring( customDimensions.aadTenantId ) , environmentName = tostring( customDimensions.environmentName ) | where timestamp > ago(90d) | evaluate activity_engagement(user_Id, timestamp, ago(40d), now(), 1d, 28d, aadTenantId, environmentName) | project-rename dau=dcount_activities_inner, mau=dcount_activities_outer | project-away activity_ratio | extend timestamp_truncated = startofday(timestamp) ; let dau_wau = traces | extend aadTenantId = tostring( customDimensions.aadTenantId ) , environmentName = tostring( customDimensions.environmentName ) | where timestamp > ago(90d) | evaluate activity_engagement(user_Id, timestamp, ago(40d), now(), 1d, 7d, aadTenantId, environmentName) | project-rename dau=dcount_activities_inner, wau=dcount_activities_outer | project-away activity_ratio | extend timestamp_truncated = startofday(timestamp) ; dau_mau | join kind=inner dau_wau on $left.aadTenantId == $right.aadTenantId and $left.environmentName == $right.environmentName and $left.timestamp_truncated == $right.timestamp_truncated | project-away aadTenantId1, environmentName1, timestamp_truncated1, timestamp, timestamp1, dau1 | project-rename timestamp=timestamp_truncated | order by timestamp desc
If you prefer to use the “rough way” of analyzing telemetries, the above query can help you and I think that this analysis is useful to check your user’s engagement and if you’re using the right number of licenses.
In the 2023 Wave 1 release, when an administrator deletes an environment from the tenant admin center, the environment will be marked for deletion (soft delete) and will stay in this state for a period of time (currently 7 days). During this period, the administrator can recover (un-delete) the environment and only after the grace period has ended, the Business Central control plane service will permanently delete the environment. All operations around this new soft/hard deletes functionality are now logged into telemetry with new event ids LC0148-LC0151 (hard delete), LC0180-LC0182 (soft delete), and LC0183-LC0185 (recovery).
As an example, to check if an environment is soft-deleted, you can use the following KQL query:
// Environment soft deleted successfully traces | where timestamp > ago(30d) // adjust as needed | where customDimensions.eventId == 'LC0181' | project timestamp , message , aadTenantId = customDimensions.aadTenantId , applicationFamily = customDimensions.applicationFamily , countryCode = customDimensions.countryCode , deletionReason = customDimensions.deletionReason , environmentName = customDimensions.environmentName , environmentType = customDimensions.environmentType , totalTime = customDimensions.totalTime
To check if environment recovery is failed (un-deleted), you can use the following KQL query (and this could be useful to be used in an alert in order to be proactive):
// Environment recovery (un-deleted) operation failed traces | where timestamp > ago(30d) // adjust as needed | where customDimensions.eventId == 'LC0185' | project timestamp , message , aadTenantId = customDimensions.aadTenantId , applicationFamily = customDimensions.applicationFamily , countryCode = customDimensions.countryCode , deletionReason = customDimensions.deletionReason , environmentName = customDimensions.environmentName , environmentType = customDimensions.environmentType , failureReason = customDimensions.failureReason , totalTime = customDimensions.totalTime
Other smaller improvements that will come in version 22.0:
- Error message telemetry (event RT0030) will include the error text in English in the custom dimension alEnglishLanguageDiagnosticsMessage
- Incoming web service telemetry (event RT0008) will include the time spent waiting in the throttling queue in the custom dimension requestQueueTime
- Error codes in failed OData calls (event RT0008) to help troubleshoot 400 return code signal. I received requests from some partners to log also incoming and outgoing JSON bodies of the OData calls. Please remember that this is not an option because logging that data means saving customer-specific data on telemetry and this can break GDPR rules.
- Long running AL method telemetry (eventId RT0018) will include details on SQL operations in the custom dimensions sqlRowsRead and sqlStatements
- Outgoing web service telemetry (eventId RT0019) will include details on client type and AL stack trace in the custom dimensions clientType and alStackTrace
I want to finish this post by signaling an interesting KQL query (always provided by the telemetry master Kennie) that I think it’s incredibly useful to monitor changes on environments.
This query shows the following changes to environments (detailed summary):
- environment changes:
- updated
- started
- stopped
- copied
- point-in-time restored
- moved to different AAD tenant
- database exported
- configuration changed
- marked for deletion
- deleted permanently
- renamed
- app hotfix applied
- recovered (un-deleted)
- feature changes:
- enabled
- disabled
- extension lifecycle events:
- compilation succeeded
- compilation failed
- synch succeeded
- synch failed
- publish succeeded
- publish failed
- install succeeded
- install failed
- un-install succeeded
- un-install failed
- un-publish succeeded
- un-publish failed
- update failed (upgrade code)
- update succeeded
- update failed (other)
- company changes:
- created
- copied
- deleted
- index changes:
- added
- removed
- data deletes (retention policy)
- data changes (sensitive field monitoring)
let _lookback = ago(7d) // change as needed
;
let environment_lifecycle_events =
traces
| where timestamp > _lookback
| where customDimensions has 'Dynamics 365 Business Central Control Plane'
| where customDimensions.eventId in ( 'LC0106', 'LC0114', 'LC0117', 'LC0120', 'LC0126', 'LC0134', 'LC0141', 'LC0142', 'LC0146', 'LC0150', 'LC0153', 'LC0159', 'LC0181' )
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, whatChanged = 'Environment'
, operation = case(
customDimensions.eventId == 'LC0106', 'Updated'
, customDimensions.eventId == 'LC0114', 'Started'
, customDimensions.eventId == 'LC0117', 'Stopped'
, customDimensions.eventId == 'LC0120', 'Copied'
, customDimensions.eventId == 'LC0126', 'Point-in-time restored'
, customDimensions.eventId == 'LC0134', 'Moved to different AAD tenant'
, customDimensions.eventId == 'LC0141', 'Database exported'
, customDimensions.eventId == 'LC0142', 'Configuration key updated'
, customDimensions.eventId == 'LC0146', 'Update window modified'
, customDimensions.eventId == 'LC0150', 'Deleted (permanently)'
, customDimensions.eventId == 'LC0153', 'Renamed'
, customDimensions.eventId == 'LC0159', 'App hotfix applied'
, customDimensions.eventId == 'LC0181', 'Deleted (marked for deletion)'
, customDimensions.eventId == 'LC0184', 'Recovered (un-deleted)'
, 'Unknown event'
)
, onWhat = tostring( customDimensions.environmentName ) // which environment
;
let extension_lifecycle_events =
traces
| where timestamp > _lookback
| where customDimensions has 'RT0010'
or customDimensions has 'LC0010'
or customDimensions has 'LC0011'
or customDimensions has 'LC0012'
or customDimensions has 'LC0013'
or customDimensions has 'LC0014'
or customDimensions has 'LC0015'
or customDimensions has 'LC0016'
or customDimensions has 'LC0017'
or customDimensions has 'LC0018'
or customDimensions has 'LC0019'
or customDimensions has 'LC0020'
or customDimensions has 'LC0021'
or customDimensions has 'LC0022'
or customDimensions has 'LC0023'
| where customDimensions.eventId in ('RT0010', 'LC0010', 'LC0011', 'LC0012', 'LC0013', 'LC0014', 'LC0015', 'LC0016', 'LC0017', 'LC0018', 'LC0019', 'LC0020', 'LC0021', 'LC0022', 'LC0023')
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, whatChanged = 'Extension'
, operation = case(
customDimensions.eventId=='RT0010', 'Update failed (upgrade code)'
, customDimensions.eventId=='LC0010', 'Install succeeded'
, customDimensions.eventId=='LC0011', 'Install failed'
, customDimensions.eventId=='LC0010', 'Install succeeded'
, customDimensions.eventId=='LC0012', 'Synch succeeded'
, customDimensions.eventId=='LC0013', 'Synch failed'
, customDimensions.eventId=='LC0014', 'Publish succeeded'
, customDimensions.eventId=='LC0015', 'Publish failed'
, customDimensions.eventId=='LC0016', 'Un-install succeeded'
, customDimensions.eventId=='LC0017', 'Un-install failed'
, customDimensions.eventId=='LC0018', 'Un-publish succeeded'
, customDimensions.eventId=='LC0019', 'Un-publish failed'
, customDimensions.eventId=='LC0020', 'Compilation succeeded'
, customDimensions.eventId=='LC0021', 'Compilation failed'
, customDimensions.eventId=='LC0022', 'Update succeeded'
, customDimensions.eventId=='LC0023', 'Update failed (other)'
, 'Unknown message'
)
, onWhat = tostring( customDimensions.extensionName ) // which extension
;
let index_lifecycle_events =
traces
| where timestamp > _lookback
| where customDimensions has 'LC0024'
or customDimensions has 'LC0025'
| where customDimensions.eventId in ('LC0024', 'LC0025')
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, whatChanged = 'Index'
, operation = case(
customDimensions.eventId == 'LC0024', 'Added'
, customDimensions.eventId == 'LC0025', 'Removed'
, 'Unknown message'
)
, onWhat = tostring( customDimensions.alObjectName ) // which table
;
let company_lifecycle_events =
traces
| where timestamp > _lookback
| where customDimensions has 'LC0001'
or customDimensions has 'LC0004'
or customDimensions has 'LC0007'
| where customDimensions.eventId in ('LC0001', 'LC0004', 'LC0007')
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, whatChanged = 'Company'
, operation = case(
customDimensions.eventId == 'LC0001', 'Company created'
, customDimensions.eventId == 'LC0004', 'Company copied'
, customDimensions.eventId == 'LC0007', 'Company deleted'
, 'Unknown message'
)
, onWhat = case(
customDimensions.eventId == 'LC0001', tostring( customDimensions.companyName )
, customDimensions.eventId == 'LC0004', tostring( customDimensions.companyNameSource )
, customDimensions.eventId == 'LC0007', tostring( customDimensions.companyName )
, 'Unknown message'
) // which company
, usertelemetryId = case(
// user telemetry id was introduced in the platform in version 20.0
toint( substring(customDimensions.componentVersion,0,2)) >= 20, user_Id
, 'N/A'
) // who did it
;
let feature_management_state_changes =
// Feature management state changes
// Available from 22.0
traces
| where timestamp > _lookback
| where customDimensions has 'AL0000JT3'
| where customDimensions.eventId == 'AL0000JT3'
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, whatChanged = 'Feature'
, operation = tostring( customDimensions.alStatus) // enabled/disabled
, onWhat = tostring( customDimensions.alFeatureDescription ) // which feature
, usertelemetryId = user_Id // who did it
;
let retention_policy_deletes =
traces
| where timestamp > _lookback
| where customDimensions has 'AL0000D6H'
| where customDimensions.eventId == 'AL0000D6H'
| extend RecordsDeleted = toint(customDimensions.alRecordsDeleted)
, TableNumber = customDimensions.alTableNo
, TableName = tostring( customDimensions.alTableName )
| where RecordsDeleted > 0
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, whatChanged = 'Data'
, operation = 'Data deleted'
, onWhat = TableName // which table
, usertelemetryId = case(
// user telemetry id was introduced in the platform in version 20.0
toint( substring(customDimensions.componentVersion,0,2)) >= 20, user_Id
, 'N/A'
) // who did it
;
let field_changes =
traces
| where timestamp > _lookback
| where customDimensions has 'AL0000CTE'
| where customDimensions.eventId == 'AL0000CTE'
| extend TableName = iff( isempty(customDimensions.alTableCaption), customDimensions.altableCaption, customDimensions.alTableCaption )
, FieldName = iff( isempty(customDimensions.alFieldCaption), customDimensions.alfieldCaption, customDimensions.alFieldCaption )
| project timestamp
, aadTenantId = customDimensions.aadTenantId
, environmentName = customDimensions.environmentName
, environmentType = customDimensions.environmentType
, whatChanged = 'Data'
, operation = 'Data changed'
, onWhat = strcat( TableName, '.', FieldName ) // which table.field
, usertelemetryId = case(
// user telemetry id was introduced in the platform in version 20.0
toint( substring(customDimensions.componentVersion,0,2)) >= 20, user_Id
, 'N/A'
) // who did it
;
environment_lifecycle_events
| union extension_lifecycle_events
| union index_lifecycle_events
| union company_lifecycle_events
| union feature_management_state_changes
| union retention_policy_deletes
| union field_changes
Stefano Demiliani 