The Viking clan Asgaard possesses a highly robust analytical platform. It allows them to thoroughly record all the loot gathered during their various raids, to track real-time exchanges with other clans, and to meticulously plan the next pillages, optimizing their resources and maximizing their gains.
The platform is comprehensive and functional, but it also presents numerous challenges as it is spread over a patchwork of tools.
Indeed, the village uses different solutions to collect, store, process, enrich, and represent the data. Although this may seem trivial, it is the source of numerous problems.
Steep Learning Curve
Each tool in the Asgaard clan’s analytical platform has its own interface and mode of operation, making it challenging for members to master them all. It’s like navigating a vast sea with many different maps and compasses, each with its own symbols and legends.
As a result, members often specialize in one or two tools, while neglecting the others. The Asgaard clan faces the challenge of balancing the need for specialization with the need for versatility in their use of the analytical platform.
Maintenance complexity
What is true for users of these tools is also true for maintenance and administration. Maintaining the Asgaard clan’s analytical platform can be compared to keeping a fleet of different boats seaworthy. Each boat has its own unique needs and requirements.
Skilled administrators must stay up to date with the latest advances and innovations in their field to provide effective and reliable support to the clan members. In addition, managing multiple products and solutions can result in additional costs, both in terms of training and software licenses. The Asgaard clan’s administrators must find ways to streamline their operations, consolidate their tools, and simplify their infrastructure as much as possible, to keep their fleet sailing smoothly.
Poor data quality
This segmentation of analytical tools inevitably creates silos where each platform is isolated from the others. This does not mean that they cannot be integrated, but this integration is not native and natural.
The Asgaard clan’s analytical platform is a vast archipelago of islands, each with its own unique resources and treasures. But navigating between these islands can be challenging, as the waters are treacherous and the maps are incomplete. Each new analytical project sets out on a voyage of discovery, with many questions to be answered before the journey can even begin.
If the answer to any of these questions is “no,” it could mean that we are charting a course through uncharted waters, which can be both exciting and perilous. We must be mindful of the risks of creating shadow data, copies of information scattered across the archipelago, no longer subject to the best practices and governance of the clan.
We strive for a single source of truth, a secure and reliable data catalog that can be trusted by all members of the clan. But working in silos, with each island isolated from the others, makes it difficult to achieve this unified repository.
Additionally, the presence of multiple silos can result in a lot of duplicate data, not all of which is maintained or curated. This can lead to poor data quality for end users, as it becomes difficult to determine which data is accurate and up-to-date.
Longer delivery time for end users
These different silos inevitably lead to an increase in delivery times for the final solution on large-scale projects because it requires prior synchronization of all the different villagers and manual integration between the different stages of data processing.
In the face of all these different challenges, our Asgaard clan believes that it has become essential to find a complete end-to-end analytical platform for all new business intelligence projects.
Introducing Microsoft Fabric
Microsoft was the ideal provider to offer this unified platform, as it already had the complete range of tools to process data end-to-end, but in separate solutions. It now offers a platform that unifies everything under the same umbrella, Fabric.
Fabric brings together a range of experiences to meet all the needs of the Asgaard clan.
Data Factory
Data Factory provides the ability to ingest, transform, and load data into a data warehouse or lakehouse. This is commonly referred to as an ETL (Extract, Transform, Load) tool.
Data Engineering
Data Engineering also enables you to collect, process, and save data, but using Spark. This experience allows you to run notebooks developed with PySpark (Python), Spark (Scala), Spark SQL (SQL), and SparkR (R). It also allows you to set up a lakehouse to store unstructured and structured data.
Data Science
The Data Science experience is designed for training machine learning models that enrich data and provide predictive insights.
Data Warehouse
The data warehouse experience is intended for storing structured data and querying it with SQL or using the warehouse as a data source for reporting.
Real Time Analytics
The real-time analytics experience allows you to meet the most pressing needs in terms of interactive and live data. This experience offers an eventstream to capture and route live events, a KQL database for storage, and KQL querysets to execute queries. This solution is optimized for streaming and time-series data.
Power BI
Power BI is a data visualization tool that connects to various sources and render consistent, visually immersive, and interactive information. It is one of the most important experiences in Fabric since Power BI has expanded to accommodate other new experiences in its tenant. If your village already uses Power BI, your village already uses Fabric, although you need to activate a parameter to activate the other experiences.
Data Activator
Data Activator offer the ability to monitor and detect real-time events and automatically trigger predefined actions.
All of these solutions existed before, except for Data Activator. The real novelty is that all of these products, which were disseminated in a segmented siloed network, are now unified under a single banner.
OneLake, the foundation layer
OneLake is like OneDrive for data, providing a centralized location for all experiences to store their data in a single, open, and reusable format: Delta. Fabric offers both a unified interface and a unified storage, simplifying data management by facilitating integration between applications and avoiding duplication.
In practice, instead of each application storing its data in decentralized and locked locations under a proprietary data format, everything is centralized and open. This facilitates governance, makes data much more accessible and visible, and helps to limit the risks of shadow data.
To test the capabilities of Fabric, our Asgaard clan has created a Raids table that records all the raids carried out by the village.
This table is accessible through all experiences, whether through PySpark, SQL, KQL or a semantic model (previously known as a dataset) without creating a local copy, it is the same table that is accessible regardless of the compute tool.
After learning about the benefits of Fabric, the Asgaard clan has chosen to adopt it for the following reasons:
- It provides a unified interface for all tools, making it easier for members to learn and use them, reducing the steep learning curve.
- It simplifies maintenance and administration by consolidating tools and streamlining operations, reducing complexity and costs.
- It improves data quality by breaking down silos and providing a single source of truth, reducing the risks of shadow data and duplicate data.
- It reduces delivery times for the villagers by enhancing collaboration among the craftsmen and facilitating the integration between various stages of data processing.
As the Asgaard clan sets sail on their journey with Fabric, they know that this is only the beginning. With each passing day, they will delve deeper into the platform, exploring its many experiences and discovering the best practices that will help them thrive. The horizon is vast, and the possibilities are endless.
And so it was that, thanks to Fabric, the Asgaard clan could sail confidently into the future, with all their data needs met by a single, unified platform
Asgaard Analytics 
Leave a comment