Whoever monopolizes the most important data wins: this expert argues that the world must decentralize such control for universal parity.
Across every industry in the digital age, the data being created is building up exponentially. This massive volume of data can be transformed into a wide range of value add, in the form of new sources of income and new business models; better decisions and customer experiences.
However, when it comes to data value creation, the average organization has a low level of maturity.
For example, a global study by IDC and Seagate found that respondent organizations used only 32% of the available data productively. This lack of maturity can be one significant barrier: another is the requirement to share data to build a bigger picture. This is often referred to as the ‘network effect’: the more people participating, the greater the value for all involved.
Historically, the main way to create value via the network effect was through a centralized approach: aggregating data into a central place (often a cloud) and sending insights back to the edge. However, this can quickly become economically untenable, and very often it is too slow for real-time use cases.
Centralization also raises issues of digital sovereignty because it can lead to a monopolistic situation where the digital fate of organizations depends on just a few intermediaries that organize it.
Cloud and data decentralization
A radically new approach to meet the centralization challenges described is via decentralized clouds and data infrastructures.
The most prominent example is Gaia-X, a Europe-led project to build a federated decentralized data infrastructure. Its goal is to drive digital innovation, economic growth, and social progress by enabling the sharing of data, insights, and services at scale to capitalize on exponential data growth.
Gaia-X does that based on a completely decentralized architecture; common standards; and rules to ensure data privacy and digital sovereignty. This includes, for example, the standards of the International Data Spaces Association for secure and sovereign data exchange in so-called “data spaces”: that is, the unification of distributed data sources into a virtual, common data pool.
Since its launch in 2019, Gaia-X has already achieved significant momentum, driven by 300 members, including private companies and public institutions, of which HPE is a founding participant. A growing number of participants are joining from outside Europe for the simple reason that organizations that operate globally must interact with European data and organizations. Getting into Gaia-X in the early stages makes sense for many non-European organizations seeking to unlock the value of data distributed across edges and clouds.
Decentralizing through Swarm Learning
Another concept that pursues the same goal in different ways is “swarm learning”. It is a decentralized machine-learning approach that unites edge computing, blockchain-based peer-to-peer networking, and coordination while maintaining confidentiality without the need for a central coordinator.
Swarm learning has promising applicability in scenarios where multiple parties can benefit from sharing their data, but cannot do so due to privacy or latency considerations. The solution is to only share the data ‘intelligence’ outcomes rather than sharing the data itself—thereby sharing insights while preserving the privacy of the actual data.
Research published in the journal Nature on swarm learning technology developed by HPE in a medical research setting had found that using this solution to facilitate the sharing of intelligence data between individual AI algorithms yielded significantly better results than when the algorithms learned separately.
The applicability of such solutions spans across a broad range of use cases where one needs to share insights and intelligence, but cannot share raw data, such as in autonomous driving, robotics, or smart cities.
There is no doubt that initiatives like Gaia-X or swarm learning are complex endeavors. Yet the exponential growth of data and the race to gain data insights demand that we push ahead to solve the key problem of the next wave of digital transformation: how to create network effects without centralizing all the data.
We must take on these challenges now to ready ourselves for the data economy ahead.
