Microsoft Fabric cuts Spark compute costs and streamlines enterprise governance
Source: Microsoft
Microsoft Fabric has introduced significant platform capabilities in its July 2026 update, targeting large-scale data engineering workloads, security, and developer efficiency. Key highlights include the release of Fabric Runtime 2.0 powered by Apache Spark 4.1, cost-optimising Efficient Scaledown technology, customer-managed key encryption for disk shuffle data, and frictionless Git branch switching for Contributor roles. Together, these enhancements give enterprise data leaders greater control over infrastructure costs while removing operational bottlenecks across engineering workflows.
Decoupling shuffle storage lowers cluster costs and improves resiliency
A major architectural advancement this month is Efficient Scaledown for Spark workloads. By decoupling shuffle data from executor lifetimes and offloading large shuffles directly to Azure Blob Storage, Fabric allows Spark clusters to release idle compute nodes significantly faster. This eliminates the risk of pinned executors causing unnecessary compute billings after heavy processing stages finish. Coupled with native execution engine acceleration for Python UDFs and complex data types, data engineering teams can run expressive transformation pipelines faster and at lower operational cost without modifying existing notebook code.
Granular access controls and actionable catalog governance accelerate delivery
To support enterprise DevOps governance, Fabric now allows workspace Contributors to switch Git branches without needing workspace Admin privileges. This removes developer dependency on platform administrators during feature development, aligning with the principle of least privilege. Simultaneously, the OneLake catalog Govern tab has been upgraded from passive recommendations to direct operational workflows. Data owners can now inspect exact affected items, evaluate security risks, and remediate issues like unlabelled sensitive assets or stale data directly from a unified interface.
Corporate enterprise organisations evaluating or scaling Microsoft Fabric stand to gain immediate financial and operational advantages from these features. The combination of automated compute cost reductions, enterprise-grade encryption options like customer-managed keys for intermediate data, and predictable runtime release channels reduces platform risk significantly. For CDOs and data platform heads, these updates make Fabric a far more resilient, cost-conscious, and enterprise-ready choice for mission-critical data workloads.
Event-driven refresh and AI-driven monitoring transform Real-Time Intelligence
Microsoft has introduced event-driven refresh capabilities and automated lakehouse monitoring for materialized lake views in its latest Fabric update. Rather than relying on fixed time-based schedules, data engineering teams can now trigger view updates immediately when upstream files land in OneLake or when ingestion jobs complete.
This shift eliminates unnecessary compute spending on empty runs while preventing stale data from propagating to downstream executive reports. Complementing this, new Analytics and Insights dashboards provide automated telemetry tracking, identifying runtime drift, failure patterns, and cost inefficiencies across the materialized view estate before pipelines break.
Secure private streaming and enterprise identity reduce Real-Time ingestion risks
Fabric Real-Time Intelligence has received critical enterprise security upgrades, headed by private network support for Eventstream streaming connectors and Microsoft Entra ID Workspace Identity authentication for Azure Event Hubs. By utilizing VNet injection and streaming virtual network data gateways, organisations can now stream sensitive operational data directly from on-premises or private cloud sources without exposing endpoints to the public internet. Furthermore, replacing static shared access keys with managed workspace identities removes the overhead and security risks associated with manual secret rotation.
These enhancements offer immediate operational value for enterprise architecture and security leaders. By replacing rigid cron-like schedules with event-driven execution and automating table health checks via native SQL procedures, organisations can significantly lower compute overhead while guaranteeing data freshness. Simultaneously, network-level isolation and managed identity integration allow highly regulated industries – such as banking, healthcare, and telecommunications – to safely adopt Fabric Real-Time Intelligence for mission-critical ingestion pipelines without violating compliance standards.
Learn more on Fabric Updates Blog
Power BI disconnects Fabric data agents as OpenAI API deprecation forces architectural shift
Source: Microsoft
The direct integration between Copilot in Power BI and Fabric data agents will end on August 26, 2026. Driven by the retirement of the underlying OpenAI Assistant API, Microsoft is sunsetting the capability to query Fabric data agents directly within Power BI Copilot. Enterprise platform teams must now redirect conversational data access through Microsoft 365 Copilot, Copilot Studio, or custom Model Context Protocol (MCP) endpoints.
OpenAI API phase-out prompts power BI copilot adjustments
The retirement stems directly from external platform dependencies: the feature was built on the legacy OpenAI Assistant API, which is being phased out. Rather than rebuilding a custom point-to-point bridge specifically for the Power BI Copilot interface, Microsoft is redirecting investments toward its broader ecosystem integrations.
Crucially, the underlying Fabric data agents, along with their configured security boundaries, permissions, and data source connections, remain intact. The change only impacts the Power BI Copilot side panel as an entry point, leaving the core data agent architecture unharmed.
Decoupling conversational interfaces from reporting layers
For enterprise data architects, this update highlights a necessary shift away from relying on monolithic UI wrappers for AI interactions. While losing a native interface within Power BI adds short-term friction, it reinforces a cleaner, decoupled architecture where data agents serve as unified backend services across multiple frontends.
Organizations can route these agents through Microsoft 365 Copilot for broader workplace adoption, Microsoft Copilot Studio for tailored low-code assistant workflows, or Microsoft Foundry. Furthermore, support for Model Context Protocol (MCP) server endpoints allows data teams to expose Fabric data agents directly to bespoke internal applications and developer tools.
Strategic recommendations for enterprise data leaders
This transition is an opportunity to centralize conversational data governance outside of isolated reporting tools. Organizations currently relying on Power BI Copilot for Fabric data agent interaction should audit their active user workflows and select a alternative channel based on target audience needs:
- For broad business users: Migrate interaction paths to Microsoft 365 Copilot to keep data queries embedded in daily productivity flows.
- For custom enterprise workflows: Leverage Microsoft Copilot Studio or MCP endpoints to build tailored, scalable AI interfaces with fine-grained control.
Existing security implementations, including Row-Level Security (RLS) and Column-Level Security (CLS), will carry over seamlessly to any chosen alternative experience without requiring security model refactoring.
Learn more on Fabric Updates Blog
Microsoft Fabric cuts streaming latency with real-time mode in Spark 4.1
Source: Microsoft/Apache
Micro-batch streaming has long forced enterprise data engineering teams into a compromise between throughput and end-to-end latency. Microsoft Fabric removes this bottleneck by introducing real-time mode for Structured Streaming on Fabric Runtime 2.0. By shifting from discrete micro-batches to continuous, long-running task execution, Spark can now process streaming records the instant they arrive. For organization routing high-frequency telemetry, fraud signals, or operational alerts, this capability eliminates per-batch scheduling overhead and unlocks true ultra-low-latency event processing directly within your data platform.
Continuous task execution replaces the micro-batch loop
In traditional Structured Streaming, Spark plans, schedules, and commits work across discrete interval boundaries. Real-time mode fundamentally alters this execution pattern. Built on Apache Spark 4.1’s Trigger.RealTime API, it launches long-running tasks across all query stages simultaneously and links them with a streaming shuffle. Events flow continuously from streaming sources through lightweight transformations to downstream sinks without pausing at stage boundaries. Spark bypasses per-batch planning costs entirely, reserving trigger intervals solely for background state store updates, offset commits, and progress metrics.
Message-to-message routing takes priority over lakehouse sinks
This ultra-low-latency pipeline is explicitly optimized for event-driven streaming architectures rather than broad data lake ingestion. Real-time mode target scenarios center on Kafka-compatible endpoints, including Apache Kafka and Azure Event Hubs, as well as custom external outputs via ForeachWriter. File-based targets, Delta tables, and standard Lakehouse sinks are intentionally excluded from this continuous execution model. Organizations should continue leveraging standard micro-batch triggers for lakehouse ingestion and heavy table maintenance, while dedicating real-time mode to latency-sensitive hops between messaging systems.
Strict compute constraints demand dedicated pool planning
Adopting real-time mode requires careful infrastructure and query planning. Because Spark schedules every execution stage concurrently, the underlying capacity pool must provide enough task slots to cover the entire pipeline at once. In practice, enterprise architects must allocate a dedicated Fabric compute pool per real-time query to avoid slot starvation. Additionally, stateful operations like joins or custom processors deliver events row-by-row rather than in per-key batches, requiring updates to stateful logic. Monitoring strategies must also adapt, shifting from internal Spark progress metrics to external source-lag tracking.
Unlocking event-driven operations without operational overhead
For enterprise data leaders, real-time mode provides a dedicated fast path within Microsoft Fabric. It allows organizations to build responsive event-routing infrastructure, operational alerting, and real-time data enrichment directly inside their existing Spark environment, eliminating the need to maintain separate, complex stream-processing engines. By separating latency-critical message routing from bulk lakehouse ingestion, platform teams can deliver sub-second event processing where it counts while keeping storage costs optimized.
Learn more on Microsoft Learn
Microsoft Fabric cuts Spark compute costs with Native Execution Engine acceleration
Source: Microsoft
Microsoft Fabric has introduced four key query optimizations to its Spark Native Execution Engine, targeting broadcast joins, window ranking functions, repeated table scans, and common table expressions (CTEs). By vectorizing end-to-end execution paths and eliminating redundant compute steps, these default improvements deliver faster pipeline runs while directly cutting capacity unit (CU) consumption for data engineering workloads.
Eliminating repeated work across joins and common subqueries
The updated engine addresses structural inefficiencies in typical analytical queries. Broadcast hash tables are now cached per executor and reused across queries within a session, avoiding CPU-intensive rebuilds when joining large fact tables to dimension tables. Additionally, reused CTEs and repeated Delta table scans are materialized automatically: the engine computes the subquery once, stores the columnar result in memory, and reuses it for subsequent references without requiring manual code refactoring.
Vectorized ranking keeps high-volume analytics natively accelerated
Window operations like rank and dense rank no longer trigger execution fallbacks to slower runtime paths. Keeping ranking logic entirely inside the native, vectorized pipeline yields up to 16 percent faster execution on ranking-heavy jobs such as top-N reporting and customer segmentation. Because these enhancements execute transparently during query planning, existing Spark job definitions, notebooks, and automated data pipelines benefit immediately upon enabling the runtime.
Lower Fabric capacity consumption without code changes
For enterprise data leaders managing Fabric environments, these optimizations translate directly into improved cost governance and platform efficiency. Data engineering pipelines consume fewer capacity units to process identical volumes of data, freeing up shared capacity for concurrent workloads and reducing the risk of throttling. Organizations operating mid-to-large-scale data platforms can achieve better price-performance instantly by enabling spark.native.enabled in their session or environment settings, capturing meaningful operational savings without modifying legacy codebase.
Learn more on Fabric Updates Blog
Microsoft Fabric bridges AI clients and enterprise data with MCP servers
Source: Microsoft
Microsoft has introduced public preview support for model context protocol (mcp) servers in microsoft fabric data agents, allowing external AI systems to query enterprise data sources directly. This update eliminates the need for custom point-to-point integrations, enabling organizations to plug their lakehouses, warehouses, and semantic models safely into any mcp-compliant client like visual studio code or custom python workflows.
Standardizing AI access to OneLake data
A published fabric data agent acts as a standardized mcp server, exposing a single tool that encapsulates its underlying data sources. When an mcp client initiates a connection, it performs an initialization handshake, requests the tool list, and automatically parses the input schema. This architecture shifts the integration burden away from developers, as clients dynamically discover agent capabilities without hard-coded endpoints or custom api wrappers. Because the agent’s published description drives tool selection by orchestrators, data engineering teams gain precise governance over how enterprise knowledge is exposed to large language models.
Streamlined deployment across developer toolchains
Integrating the new capability requires a paid f2 or higher fabric capacity and proper cross-geo configuration. Organizations can consume the mcp server from any client that supports streamable http and token-based authentication. Developers can configure environments like visual studio code via an mcp.json file or build automated pipelines using python scripts backed by azure identity credentials. Every request relies on standard bearer tokens acquired for the fabric scope, ensuring that existing enterprise security postures and workspace permissions remain strictly enforced during ai interactions.
For enterprise data architects, this preview marks a shift toward modular, protocol-driven ai infrastructure. By abstracting data access behind a universally accepted standard, organizations can scale generative ai initiatives across different client applications without reinventing security or connectivity layers, accelerating time-to-value for analytics investments.
Learn more on Microsoft Learn
Power BI mobile app unlocks conversational Copilot chat to accelerate executive decision-making
Source: Microsoft
Microsoft Power BI now enables a full conversational Copilot chat experience directly within its mobile app, replacing static predefined prompts with dynamic data exploration. Available in public preview, this update allows decision-makers and operational leaders to query reports, request contextual summaries, and receive interactive visual answers on phones and tablets. By bringing natural language analysis into mobile workflows, enterprise teams can evaluate KPIs and extract insights on the go without relying on desktop tools or manual analyst requests.
Moving from static summaries to interactive data exploration on mobile
Previously, mobile report interactions with Copilot were limited to basic summaries and fixed prompt templates. While useful for high-level overviews, this restricted deep-dive analysis when managers needed to investigate specific operational shifts. The updated chat interface mirrors the capabilities of the Power BI service, allowing users to ask iterative follow-up questions, refine queries, and use voice dictation and read-aloud features on iOS devices. Crucially, when users inquire about specific metrics or KPIs, Copilot generates inline visualization cards alongside text responses, complete with clickable citations that open visuals in focus mode for instant interaction.
Driving self-service analytics and reducing ad-hoc BI requests
For enterprise analytics leaders, extending conversational AI to mobile endpoints addresses a common adoption hurdle: friction in mobile BI consumption for busy stakeholders. Because Copilot grounds its responses strictly within the underlying report data, it maintains governance and data context without requiring BI teams to design custom mobile dashboards for every executive request. Business leaders can now explore metrics independently during field visits or meetings, copy response summaries, or tap Share directly from visual panes to distribute actionable findings across team channels immediately.
Empowering mobile leaders with grounded, actionable insights
While mobile BI has traditionally served as a passive reporting view, embedding a chat-based Copilot experience transforms mobile devices into active analytical tools. For enterprise organizations, this update maximizes the value of existing Power BI assets by making enterprise data accessible and conversational for mobile workers. Organizations should consider enabling this public preview to reduce reporting bottlenecks and empower leadership with context-aware data exploration wherever decisions happen.
Learn more on Power BI Updates Blog
