How organizations can accelerate the pace of recovery from COVID-19 by leveraging the capabilities of cloud data platforms.
In an on-demand culture, and especially in the ‘new normal’ brought on by the COVID-19 pandemic, consumers and businesses are exposed to lots of online communication, collaboration and transactions – which also means so much more data is being created, exchanged and stored every second.
Having control and a single source of truth for data is especially crucial for organizations seeking to leverage and profit from the exponential data growth in this era of cloud and mobile apps. Being able to perform real-time data exchange has also become business-critical.
Data sprawl in these uncertain times can get an organization down. How should organizations turn the tables to better manage this data explosion and leverage it for business advantage?
DigiconAsia gleaned some insights from Geoff Soon, Managing Director, South Asia, Snowflake.
What are the biggest concerns for businesses when it comes to exponential data growth?
Geoff Soon: One of the biggest concerns for businesses is the capability to derive real-time insights from the wealth of data that they store.
Complex data exchange methods combined with costly and inflexible computing platforms make it difficult for organizations to collaborate and leverage their enterprise data. Using legacy platforms, many organizations are still using multiple spreadsheets, conducting batch processes with file transfer protocol (FTP), extract, transfer, load (ETL) software and using application programming interfaces (APIs).
Also, the value of shared data is diminished due to the time required to extract data from traditional platforms. As the shared data is in its static version, it becomes immediately stale once downloaded. Thus, every time data needs to be changed, data extraction and the trader processes must be repeated, wasting away valuable time, money, and resources.
Companies must have the right technology and tools in place that will enable real-time data exchange.
How does cloud data sharing platforms help businesses maximize resources and derive benefits from their data?
Geoff: As an example, Snowflake’s secure data sharing capabilities allow data exchange through Snowflake’s unique services layer and metadata store. Without actually copying or transferring files between accounts, shared data does not take up any storage in a consumer’s account and therefore, does not contribute to the consumer’s monthly data storage charges.
Consumers are charged only for the compute resources used to query the stored data. They can immediately use and query the shared data at the highest performance profile.
One use case that I can share is Rakuten Rewards. The company established a highly scalable enterprise data hub for internal use with Snowflake. As a result, wait time for data is shortened, the data sets are available earlier in the day and IT teams can identify and detect problems quickly. While query performance can vary depending on the size of the cluster and the exact type of job, Rakuten Rewards has definitely seen a 95% faster run time.
Here are some key benefits of leveraging data sharing capabilities:
A. Drive simplicity
Traditional data sharing platforms create unnecessary complexity. To maximize time and resources, data sharing across the organization should be simple and straightforward. With streamlined data sharing processes, businesses can save on time, infrastructure costs and other resources.
A cloud data platform such as Snowflake’s makes sharing of data simple and easy. The architecture is designed to separate compute, storage, and services to allow unlimited scaling, concurrency and data sharing. With unlimited scalability and compatibility, organizations can better allocate time and resources to strategic data sharing projects.
B. Derive valuable insights from data
Data from traditional data storage needs to be downloaded first before it can be shared with data consumers. Since the downloaded data is not updated, data consumers are presented with stale and inconsistent information. Valuable data and information are lost in the process, which impacts the integrity of insights drawn from the organization’s data.
We can address this problem by sharing ‘live’ data in real-time. Data meant for scaling and sharing is stored in physically separate storage pool which ensures data consistency and integrity. From the cloud data platform, data consumers can access data as soon as an update or transaction is committed and successfully executed.
C. Support any scale with ease
As the world becomes more connected, organizations need to anticipate data-sharing with tens, hundreds, or thousands of data consumers. Each of these data consumers will have unique data-sharing requirements and businesses must prepare to support these varying needs without manually building more clusters, managing external tables or metadata stores.
How can organizations get the most out of data exchange and sharing while minimizing the risks?
Geoff: Secure data sharing empowers organizations by helping them understand their customers better. Equipped with a better understanding of their customers’ needs through data-driven insights, organizations can create better products and provide better customer experience.
Privacy and security concerns in data exchange can be addressed with the help of modern, governed, and secure data monetization practices. By establishing secure data monetization practices, organizations can stay competitive in the data-driven economy.
When it comes to data regulations and compliance, how can organizations be sure they are not breaking any rules?
Geoff: With the recent high-profile data breach incidents such as the Singapore Ministry of Defence malware attack, there has been a strong call for the regulation of digital technology. There is also an increasing public demand for greater transparency on how customer data is being handled, which brought data management and storage practices in the spotlight recently.
Last year, the Singapore government implemented measures to improve data security in the public sector. The government is aiming to implement majority of these data security practices by the end of 2021.
Here are some of the recommendations:
- Implement data security measures like protecting data directly when stored to ensure it is unusable even if extracted
- Account for data protection at every level through measures like amending the Personal Data Protection Act to cover third-party vendors handling government data
- Continuously improve data security through measures like improving the government’s expertise in data security technology
Organizations need to understand and adhere to local regulations and international regulations to protect and secure customer data. As a global business hub, businesses in Singapore must also adhere to international regulations such as the General Data Protection Regulation (GDPR) and California Consumer Privacy Act (CCPA).
How can Snowflake help businesses in their data management journey?
Geoff: The cloud offers unlimited capacity for data storage and processing, which opens the door to an unprecedented number of concurrent high-performance workloads within a centralized platform. In a data-driven environment, organizations need a cost-effective and efficient platform to load, transform, integrate and analyze unlimited amounts of structured and semi-structured data, in their native formats.
There is also a requirement to simplify access to data to automate routine data management activities and support a broad range of data and analytics workloads. To achieve this, organizations must be able to collect, store, analyze and draw insights from their data in one place.
A cloud data platform can bring this all together with a common set of services that streamline the use of data. By creating a single place for all data types and workloads, a cloud data platform will dramatically simplify the IT infrastructure, without incurring the costs inherent in traditional architectures.
For example, by centralizing data, organizations can reduce the number of stages the data needs to go through before it becomes actionable, which eliminates the need for complex data pipeline tools. With reduced wait time for data, users can obtain the data and insights they need, when they need them, so they can immediately spot business opportunities and address pressing issues.
Snowflake’s cloud data platform allows organizations to consolidate their data from data warehouses, data marts, and data lakes into a single source of truth that powers multiple types of analytics and data science applications. Teams can easily collaborate and share governed data, internally and externally without having to copy or move files in different places. Highly extensible, multi-region and multi-cloud, Snowflake’s cloud data platform can also support all types of data workloads.
Raw, structured, and semi-structured data is easily discoverable and immediately accessible for data science workflows, with native support for data file formats such as JSON, AVRO, XML, ORC, and Parquet. Snowflake offers the capability to use one set of tools to manage both structured and semi-structured data which shortens the data discovery and preparation cycle.
What we believe is the most exciting trend moving forward is that companies such as Snowflake can provide a single unified platform that combines the cost efficiencies of storing massive amounts of structured and semi-structured data with a highly-performant processing to give organizations unlimited virtual warehouses to drastically simplify the way our customers perform analytics.
How would a cloud data platform help businesses resume operations in the wake of the COVID-19 pandemic?
Geoff: Organizations need to come up with robust business continuity strategies to thrive in the ever-changing market and part of this strategy is access to business data. Cloud technology provides a platform that enables access to data, anytime, anywhere, and from any device. It provides an infrastructure that can support the business during times of crisis such as the COVID-19 pandemic.
Depending on the needs of the organization, cloud data platforms offer scalability and flexibility with unlimited storage capability. During tough times, when the use of resources needs to be optimized, businesses benefit from cloud data platforms by paying only for the compute and storage they actually use.
As COVID-19 continues to disrupt industries, businesses need to make quick decisions to resume operations. Organizations can accelerate the pace by leveraging the capabilities offered by cloud data platforms.
