Power Automate license enforcement - looking at it from a Dynamics 365 CE perspective

Mid May 2023, a warning popped up in Microsoft Message Center regarding "Non-Compliant Power Automate Flows" -  soon after the message however disappeared so this got probably missed by the majority of Dataverse and Dynamics 365 admins.

This however raised some concerns in the broader Microsoft community ( see Will Power Automate enforcement licensing kill your flows?  and Upcoming licensing enforcement in Power Automate explained ). 

To be honest I did not pay a lot of attention since the message apparently vanished in thin air and after consultation with Microsoft support they said that this message was sent prematurely. But then, end of October another warning popped up.

So it seems that Microsoft is finally cracking down on Power Automate flows which are not associated with a properly licensed user for premium connectors or Power Automate flows not directly linked to a Power App. When you built your own model-driven app on top of Dynamics 365 Customer Engagement (CE) which uses Power Automate flows, you will need to associate the Power Automate Flow with with your new app.

There is a PowerShell script to identify the flows which at risk to be turned off  across your tenant - see I have many environments - how can I get the flows that need my attention across tenants in the Power Automate Licensing FAQ - which uses the Get-AdminFlowAtRiskOfSuspension cmdlet

The Get-AdminFlowAtRiskOfSuspension cmdlet is part of a separate PowerShell module which you can install using Install-Module -Name Microsoft.PowerApps.Administration.PowerShell. It will run a scan of your environments and outline

Check out Associate flows with apps - Power Automate | Microsoft Learn on how you need to link up a flow with an app (see below screenshot on where to do this in the Power Automate flow detail screen). If you make this change on a flow which is a part of a solution, then the associations will be part of the solution file and can be transported cross environments.

Related articles/blog posts:

• Will Power Automate enforcement licensing kill your flows?
• Upcoming licensing enforcement in Power Automate explained
• Flows in app context - is there a clear definition of what it means?
• How do you license Power Automate flows when using them with Power Apps

The ABC of AI: Retrieval-Augmented-Generation (RAG) and grounding

This is the first in a series of blog posts about more advanced generative AI and Large Language Model (LLM) concepts which I use as notes to myself (check out Why I blog and you might want to consider it as well)

Retrieval-augmented generation (RAG) is an AI framework that enhances the quality of responses generated by large language models (LLMs). LLMs are trained on a massive amount of data and understand statistical relationships between words but lack true comprehension of their meanings. So when faced with specific questions in a dynamic context so that is where RAG comes in.

RAG integrates information retrieval into LLM answers by using these steps:

1. User inputs prompt: when you ask a question, RAG uses your input prompt
2. RAG retrieves relevant information from an external knowledge base based on the user prompt
3. RAG combines this external content with your original promt creating a richer input for the LLM

RAG and grounding are related concepts in the context of enhancing LLMs. Grounding is the process of providing LLMs with information about a specific use-case. RAG is one of the techniques which is used for grounding (another technique is dense retrieval - see Dense X Retrieval: What retrieval granularity should we use  for more details). 

Microsoft Copilot offers a good example of grounding and RAG in use. The reason why Copilot is able to give more targetted responses, is because it uses grounding to improve the specificity of the prompt. 

Copilot uses Microsoft Graph, which can retrieve information about relationships between users, activities and organizational data (like info in Power Platform/Dataverse and/or Dynamics 365, info from e-mails, chats, documents and meetings) as part of the prompt grounding process. Microsoft Copilot will use the user prompt and additional info retrieved through Microsoft and then sends it to the LLM. For more details see Microsoft Copilot for Microsoft 365 overview

The most common systems to provide external data for RAG LLMs are vector databases and feature stores.

References:

• Implement Retrieval Augmented Generation (RAG) with Azure OpenAI Service (Microsoft Learn)
• RAGAs- How to evaluate RAG pipelines chatbot
• Intro to Retrieval-Augmented Generation (RAG) with Generative AI and OpenAI on Azure
• Azure Cosmos DB - Vector Database
• How vector search and semantic ranking improve your  GPT prompts (Microsoft Mechanics)
• The 5 types of LLM apps (YouTube)
• RAG application with Azure open AI & Azure Cognitive Search (French legal use case - Python notebook)
• Microsoft Copilot for Microsoft 365 overview
• Grounding language model with chunking-free in-context retrieval
• Grounding LLMs
• Langchain - use cases - Q&A with RAG
• Retrieval Augmented Generation (RAGs) for LLMs

Dynamics 365 and Power Platform monthly reading list April 2024

Copilots, AI and machine learning

• Artificial Intelligence 2023 Playlist by Stanford Online (YouTube) and Stanford CS229: Machine Learning | Spring 2022 Stanford Online (YouTube)
• Use Semantic Kernel with Lakehouse in Microsoft Fabric
• Fabric end-to-end use case: Analytics engineering part 1 - dbt with the Lakehouse by @s_debruyn
• From Microsoft to global brands, Dynamics 365 Copilot is helping transform customer experiences across service, sales and marketing
• Accelerate support ramp-ups, handoffs and wrap-ups with Copilot summarization in Dynamics 365 Customer Service
• Introducing new Copilot experiences to boost productivity and elevate customer experiences across the organization
• Transform your business with Microsoft AI (Microsoft Learning Path)
• Accelerating revenue with AI Powered Solutions: a CRO Guide (e-book)
• Microsoft Azure AI Fundamentals: Generative AI (Learning path)
• 10 AI terms everyone should know
• Revolutionizing marketing workflows with Copilot in Dynamics 365 Customer Insights
• Creating Custom Copilots: Azure AI Studio and Copilot Studio. When to use what?
• Build your Copilot testing strategy in Dynamics 365 Customer Service
• Learn Live: Microsoft Learn AI Skills Challenge
• The Art and Science of working with AI
• Microsoft Copilot in Azure extends capabilities to Azure SQL Database (Private preview)
• How to connect Copilot Studio to Dynamics 365 Customer Service (YouTube)
• Introducing Copilot pane in Power BI Desktop (preview)
• Enable Copilot case and conversation summaries 
• Trust Micrsoft Copilot with your Power Platform and Dynamics 365 data - Power CAT live (YouTube)
• Building Modern Enterprise Solutions: Microsoft Dataverse and Microsoft Copilot
• Microsoft Copilot scenario library - real use cases for HR,Finance, IT, Marketing and Sales
• Microsoft Copilot for Sales FAQ
• Day 19: Microsoft Copilot vs Microsoft Copilot for Microsoft 365 vs Copilot aka PVA
• Announcing Microsoft Copilot for Service
• Enable Copilot in Customer Service and custom apps (Dynamics 365)
• Microsoft Copilot for Service: Introducing new Copilot experiences to boost productivity and elevate customer experiences across the organization  and Copilot in Microsoft Dynamics 365 (YouTube)
• Generative AI for beginners (Microsoft)
• Large language models, explained with a minimum of math and jargon
• Using copilot with SharePoint knowledge to serve your customers
• Dynamics 365 Customer Insights - Deliver higher-impact experiences with better data quality and configuration with Copilot
• Information Assistant Industry accelerator on GitHub
• Announcing Microsoft Copilot for Service and Microsoft Copilot for Service product page
• Build your own copilots with Azure AI Studio (Microsoft Mechanics recording)
• Build GPT-automated customer support with Azure Communication Services

 

Technical topics (Configuration, customization and extensibility)

• Lakehouse vs Warehouse vs Datamart - the difference between the three Fabric objects
• Power Platform Well-Architected framework
• Creating service principals really easily using PAC CLI by @carldesouza
• The power of Prefer Headers in the  Dynamics CRM web API
• Analyze Dataverse tables from Microsoft Fabric
• Enterprise security with Power Platform (Microsoft white paper)
• Microsoft Fabric, explained for existing Synapse users
• Announcing new cloud governance guidance in the Microsoft Cloud Adoption Framework (CAF) for Azure
• PCF Learning Roadmap
• Mask sensitive data and prevent unauthorized access and Dataverse: about masking rules 
• Protect enterprise solutions with new Microsoft Power Platform security features
• Power Platform Wave 1 2024: Top 10 features you need to know! (YouTube)
• Add intelligence into your apps with Dataverse AI functions (YouTube)
• Leverage the Power Platform API to create a capacity report
• Associate activities with multiple related records (2023 Wave 2 Public preview)
• Power Platform tenant settings  (YouTube)
• Working with solution dependencies for objects made easier
• Improve solution import time
• https://aka.ms/azai - develop apps that use Azure AI services
• How to create a new Dynamics 365 SCM trial environment from the Power Platform Admin Center (PPAC) preview
• What's new in Playwright 1.32 (YouTube)
• Using Application Insights for Dataverse monitoring
• Modernize customer support with Copilot in Dynamics 365 Customer Service
• Announcing the Power Platform solutions with AI and Copilot learning path
• Sample library for Digital Contact Center Platform

Topics for Dynamics 365 Business Applications Platform consultants, project managers and power users

• Modernize Service Guided Tour
• Forrester TEI study shows 315% ROI when modernizing customer service with Microsoft Dynamics 365 Customer Service
• End of support for Project Service Automation (PSA 3.x)
• Model driven Apps (Power Platform)  visual glossary
• My annual letter: leading in a new era - annual letter from Satya Nadella, CEO at Microsoft
• Finding adequate metrics for  outer,inner and process quality in software development
• Customer story: Avanade puts people first, accelerates sales productivity with Microsoft Copilot for Sales
• Keeping up to date with changes to the Release Wave Plans
• A consultant's guide: overcoming the challenges of joining a Microsoft Dynamics 365 project mid-way through
• Accelerate Copilot extensibility and Power Apps creation with Microsoft Dataverse
• How Microsoft and generative AI are transforming financial services contact centers

SQL Server Integration Services Project template available for Visual Studio 2022

 Since end of 2022, there is also a SQL Server Integration Services  Project template available for Visual Studio 2022 which you can install from the Visual Studio Marketplace. You can install it from the direct download link here or you can search for it in the Visual Studio 2022 extension manager and install it from there.

Classic Azure Application Insights deprecated on February 29th 2024 - 7 days to go

 If you missed it - classic Azure Application Insights will be deprecated on February 29th 2024. If you missed the different notification e-mails, you can quite easily see the warning if you navigate to an Azure Application Insights resource in Azure Portal.

Migration is actually quite easy - you just click on the link provided and this will open up the menu depicted below which allows you to associate your Azure Application Insights resource to a Log Analytics Workspace. The good news is that there are no pricing changes when moving to the workspace-based model. 

As indicated in the migration window, this is  a one way operation so plan for it in advance - the points below might impact on how you will do the migration:

• You can link different Application Insight resources to a single Log Analytics workspace or you can make the split - in most case you want to consolidate it.
• Instrumentation keys do not change during the migration so you don't need to worry about this
• The export feature is not available on the Application Insights workspace-based resources - you need to look at diagnostic settings for exporting telemetry
• There might be some schema changes - important to consider when doing KQL queries - check out query data across Log Analytics workspaces, applications and resources in Azure Monitor
• Existing log data will not immediately move to the Log Analytics workspace - only new logs generated after the migration will be stored in the new log location.

Procreate 5 handleiding video overzicht

Lijst van ondersteunende video's bij het boek Procreate 5 Handleiding - tekenen op de ipad (Stefan de Groot, 2020)

• Hoofdstuk 2 - Gallery
• Hoofdstuk 3 - Canvas & brushes
• Hoofdstuk 4 - Teken, veeg, gum en vormen
• Hoofdstuk 5 - Layers
• Hoofdstuk 6 - Kleuren
• Hoofdstuk 7 - Gebaren
• Hoofdstuk 8 - Actions
• Hoofdstuk 9 - Adjustments
• Hoofdstuk 10 - Selecteren en transformeren
• Hoofdstuk 11 - Snelkoppelingen & Apple Pencil

Running SSIS packages in Azure Data Factory - scaling and monitoring

Lifting and shifting SSIS packages to Azure Data Factory (ADF) can provide several benefits. By moving your on-premises SSIS workloads to Azure, you can reduce operational costs and the burden of managing infrastructure that you have when you run SSIS on-premises or on Azure virtual machines. 

You can also increase high availability with the ability to specify multiple nodes per cluster, as well as the high availability features of Azure and of Azure SQL Database. You can also increase scalability with the ability to specify multiple cores per node (scale up) and multiple nodes per cluster (scale out) - see Lift and shift SQL Server Integration Services workloads to the cloud

To lift and shift SSIS packages to ADF, you can use the SSIS Integration Runtime (IR) in ADF. The Azure SSIS-IR is a cluster of virtual machines for executing SSIS packages. You can define the number of cores and compute capacity during the initial configuration (Lift and shift SSIS packages using Azure Data Factory on SQLHack)

Even though there is Microsoft article which explains how to Configure the Azure-SSIS integration runtime for high performance, there is not a lot of guidance of how to run it at the lowest possible cost but still being able to complete the jobs. So would you recommend a higher sizing running on a single node or running a lower sizing on multiple nodes? Based on experience, it seems perfectly possible to run most jobs on a single node and up until now we have been running all of them on a D4_v3, 4 cores, 16GB Standard. If you decide to run it on a lower configuration, it would recommend monitoring failures, capacity usage and throughput. (See Monitor integration runtime in Azure Data Factory for more details)

Reference:

• Configure the Azure-SSIS integration runtime for high performance 
• Comparing SSIS and Azure Data Factory

Dynamics 365 and Power Platform monthly reading list November 2023

 2023 Release Wave 2

• Dynamics 365 Customer Service: new dialer experience (preview) by @D365Goddess
• New sales capabilities in 2023 release wave 2 - helping sellers work smarter
• Dynamics 365 CRM Wave 2 Release: enhance categorization of knowledge search articles
• Configure sales Copilot for Dynamics 365 Sales (Preview) by @DianaBirkelbach

Technical topics (Configuration, customization and extensibility)

• Dataverse - use bulk operation messages (MS Learn) -  CreateMultiple and UpdateMultiple are now GA!  Read the small print to see when to use it
• Dataverse let's try elastic tables (preview) - by @temmy_raharjo
• Ways to deal with missing Power Platform environments  by @inogic
• Edit subgrids side by side with Power Apps Grid or editable grid by @DianaBirkelbach
• Announcing monthly channel for model-driven apps
• Announcing general availability of custom connectors in solutions as well as environment variable secrets
• Announcing SharePoint Embedded Public Preview at ESPC23 - will be interesting to explore whether it is possible to combine this with Dynamics 365 data as an alternative for Power Pages. Head over to http://aka.ms/start-spe/ to start building your first SharePoint Embedded app. 
• Dataverse + Azure Service Bus queue + Azure function for processing long operations
• Workflow automation in Dynamics 365 CRM: triggering actions on email send and receive events by @inogic
• August 2023 updates for modernization and theming in Power Apps
• Connecting to Dataverse from Function App using Managed Identity - using azd
• Power BI - Mastering sales calculations: a comprehensive guide to departmental analysis 
• How to enable the enhanced email template editor in model-driven apps (Dynamics 365/Dataverse)

Copilots, AI and machine learning

• Write an email with Copilot in Dynamics 365 Customer Service by @nishantranacrm
• Getting your enterprise ready for Microsoft 365 Copilot (Ignite 2023 recording)
• Learn live: prepare, implement and secure Microsoft 365 Copilot (Ignite 2023 recording)
• New study validates the business value and opportunity of AI
• Architecture and deployment diagrams for Microsoft 365 Copilot
• Analyse the impact of AI-enhanced customer service with Copilot analytics
• What is prompt engineering? (McKinsey)
• Anti-hype LLM reading list
• Microsoft Chat Copilot vs Azure ChatGPT - which generative AI capability to choose for the enterprise
• Measuring the productivity impact of Generative AI (NBER digest)
• The power of prompting (Microsoft Research)
• Get started with Microsoft Copilot Studio: how to create your first Copilot by @lisacrosbie
• Use Copilot to summarize cases - Dynamics 365 Customer Service

Topics for Dynamics 365 Business Applications Platform consultants, project managers and power users

• Microsoft to make Dynamics 365 Marketing a part of D365 Customer Insights
• Customer Service in the age of AI
• Refresh the sales experience with Dynamics 365 Sales modern update