In today’s dynamic digital world, real-time data analytics is no longer just a competitive advantage – it has become a necessity. At Onex Group, we are proud to share how our innovative approach using Microsoft Fabric, Azure Service Bus and Azure Cosmos DB has transformed Nsure.com’s data capabilities. This new AI-ready data platform not only replaces Dynamics 365 CRM analytics, but also enables Nsure.com’s flexible, data-driven decision-making.
Business challenges
Before the transformation, Nsure.com relied on data analysis from CRM Dynamics 365 where data was stored in Dataverse. While this system was effective in its time, it became a bottleneck for the growing online insurance shopping platform.
Key challenges included:
- Delays in obtaining analysis: Data updates and analysis were not real-time.
- Scalability limitations: Older systems struggled to handle growing data volumes.
- Integration problems: Lack of seamless integration of transactional and analytical processing.
Nsure.com also struggled with the difficulty of aggregating data for complex reports, the need for a database based on facts and dimensions, the high cost of maintaining data in Dataverse, and the need to replace the existing Dynamics 365 CRM system.
Our innovative solution
Our team at Onex Group has implemented a modern data platform centered around Microsoft Fabric. Here’s how the solution is redefining real-time analytics:
Data acquisition and integration
- Data sources:
- Event data: Using Azure Service Bus, system events such as user creation and renewal are sent to our architecture. We use Microsoft Fabric EventStream to pull these events into Delta tables stored in lakehouse etl_configuration.
- Cosmos DB data: Are replicated in Microsoft Fabric databases, acting as a brown layer for further processing.
- Processing:
- Events are processed using a combination of Python, SQL and KQL in notebooks, querying GraphQL with dynamic parameters.
- A batch deletion process every 15 minutes keeps the lakehouse up to date by deleting stale messages.
Medallion architecture for data reliability
- Brown layer: raw data from Azure Service Bus and Cosmos DB.
- Silver and gold layers: materialized views, ensuring that data is up-to-date and complete.
- Historical vs. real-time data:
- Current data: Stored for 24 hours.
- Historical data: Archived in Delta tables for efficient analysis.
Consumption and visualization
- Power BI Reports:
- Direct Query Model: Real-time analysis with refresh every 15 minutes, synchronized with the batch process.
- Import mode: Historical data analysis, refreshed daily.
This approach not only provides immediate analysis, but also supports in-depth analysis for strategic planning.
Support for Data Science
The platform supports advanced data analysis:
- Advanced analytics: Data Science teams use Python and R to perform strategic, complex analysis on demand.
- Integrated visualizations: Results are stored in OneLake or visualized in Power BI.
Implementation process
The implementation included key milestones:
- Data streaming: Used Azure Service Bus for event streaming, Python notebooks for GraphQL queries and Eventhouse for real-time data.
- GraphQL configuration: Defined data messages and configured the necessary infrastructure to support GraphQL queries.
- Data and message definition: Data was extracted from various sources, transformed and loaded into relevant databases.
- Data insertion: Configured the system to handle data insertion and ensured that data is correctly stored in the database.
- Materialized Views: Created to optimize performance and provide a single source of truth for Power BI and data science.
- Azure Cosmos DB Replication: Implemented to provide near-real-time data replication.
- Historical and current data: Designed to handle both historical and current data.
- Power BI integration: Real-time and historical data visualization provided.
- Data Science environment: configured to support advanced analytics and machine learning.
Implementation and cooperation
The project took four months, including three weeks of planning and research. We worked closely with Nsure.com, managing the backlog in Azure DevOps and engaging architects, data engineers, ETL/ELT and Power BI developers.
Business value and benefits
The implementation has significantly increased Nsure.com’s operational efficiency and competitiveness, bringing key business benefits:
- Faster decision-making
- Better customer service
- Greater operational efficiency
- Readiness for future AI innovations
Future plans
Nsure.com plans to leverage Copilot and AI Skills, integrate with other business systems, and develop and scale the platform. Examples of AI Skills that Nsure.com plans to use in the short term include predictive analytics, natural language processing (NLP), machine learning models and automated risk assessments.
Summary
By moving to a Microsoft Fabric-based data platform, Nsure.com not only solved its challenges with its previous infrastructure, but also prepared for future growth in a competitive market. Our solution demonstrates how modern cloud technologies can unlock real-time value, enable data science and ultimately lead to better business outcomes. At Onex Group, we are excited to continue innovating and helping our customers thrive in the era of real-time analytics.
Author: Maksymilian Zabrzycki, Data & AI Co-Owner, Onex Group
