A year into the pandemic, businesses have realised the importance of gathering useful and actionable insights from their data. However, according to a report by MIT Technology Review, a majority of businesses have yet to excel at delivering on this data strategy to deliver measurable results across the enterprise.
iTNews Asia speaks to Andrew Martin, Head of Databricks ASEAN and India to find out why companies fall short of maximising their data strategies, how they can do so, and the benefits from fostering a data driven culture.
iTNews Asia: Data analytics is considered a useful tool for organisations to gain insights into their business, yet the report by MIT Technology Review Insights revealed that only 13% of organisations excel at delivering on their data strategy. Why is it that most companies fall short at gaining results from their data strategies?
Data is increasingly critical to the success of businesses across all industries, yet few organisations have laid the proper groundwork for a truly data-driven business model. Enterprises are investing in core data competencies across business intelligence, data engineering, data science, machine learning (ML) and more. Yet, delivering properly on these priority areas requires a range of technologies to be seamlessly integrated which is often not the case.
This leads many organisations to build and maintain different technology stacks to handle all of their data workloads. This lack of proper integration can translate into multiple unnecessary copies of data and inconsistencies in data security, which is further exacerbated by siloed systems and the absence of a unified governance model.
Over time, organisations will adopt more solutions or add onto existing ones, creating a messy and expensive network of solutions that don’t always work well together. Networks like these are difficult to maintain and prevent separate data teams from collaborating with a single source of truth, limiting an organisation's ability to get the most value from data.
iTNews Asia: How can organisations go about maximising their data that they’ve acquired to support their business?
Organisations across ASEAN and India are well-primed to embark on wider data-driven transformation to emerge stronger from the pandemic. Our research has shown that 56% of Asia Pacific respondents are currently evaluating or implementing new data platforms to better enable the use of data across the enterprise.
In order to maximise the business value of their data, organisations should focus on three critical technology pillars to create the building blocks of a successful data strategy:
- Modernise the data architecture
A modern data architecture is the foundation of a transformative enterprise data strategy. In the past, different data use cases demanded different data structures.
For example, business intelligence typically demanded data warehouses — a collection of data structured and filtered for very specific purposes — whereas data science and ML use cases required separate data lakes for large collections of unstructured data.
Rather than incorporate a number of disconnected solutions for different use cases, enterprises should look to establish an open and modern data architecture that can grow and adapt with their current needs and future vision.
Architectures that allow all data teams, from marketing analysts and data engineers to data scientists, to work with data and collaborate on a central platform — no matter the use case — will accelerate innovation at a faster rate for the entire organization.
- Build in the cloud(s)
Once considered a nice-to-have, cloud is now central to modernising and successfully scaling data management. Cloud adoption has steadily gained momentum since its introduction in the early 2000s, exploding in recent years as the de facto approach to building modern platforms.
The shift to all things virtual throughout 2020 may have best highlighted this approach, with cloud-based technologies empowering teams to work together seamlessly while on-prem solutions faltered.
Cloud-native platforms provide greater storage and computing power at declining cost and, according to our research, 43% of data and technology executives across the globe cited increasing the adoption of cloud platforms to be one of the most important enterprise-wide data strategy initiatives over the next two years. Among the companies surveyed, 63% use cloud services widely in their current data architecture, with over one-third (34%) of those operating in multiple clouds.
As adoption of cloud-based technology grows, modern data teams are looking for solutions that can move across major clouds. A multi-cloud approach offers organisations a number of benefits: the flexibility to run workloads anywhere, easy integrations when bringing on new solutions or businesses that use other cloud providers, and the assurance that they can comply with regulations down the road.
Many APAC businesses embraced the transformative year as an opportunity to double down on cloud, with investment in public cloud services growing over 38% in the region last year.
- Embrace open source and open standards
As data architectures evolve, the value of open source technology and open standards will only increase, as the approach discourages teams from building overly-complex solutions in-house, which eats up unnecessary resources.
Open source technology usually comes at little to no cost and popular solutions have been thoroughly adopted and vetted by experts and data professionals within the ecosystem. This means fewer operational risks and hiccups for IT teams down the road, especially when looking to scale the business, as they’re building on reliable, proven solutions.
Additionally, open source technology ensures full transparency and visibility into source code and offers data teams a connection to the wider open source community - a limitless resource for inspiration, troubleshooting and tech talent.
iTNews Asia: How does the growing use of AI and ML in the cloud help to support the development of a strong data culture? Is this the trend moving forward for businesses looking to build on their data strategy?
A data-driven culture embraces the use of data in everyday decision making, across all facets of the business. Establishing a data-driven culture requires democratic access to data for every team, with centralised governance that ensures high-quality data is always used for decision-making.
The growing use of AI and ML allows data teams to augment many of the tedious, repeatable data tasks and instead focus on solving more complex problems for their business - like uncovering key contributors to prevent customer churn.
With an open, collaborative platform that unifies all the data, analytics, and AI workloads, data and analytics leaders can foster a data-driven culture that focuses on adding value by relieving the daily grind of siloed systems and unreliable data. A modern architecture ensures teams are working collaboratively from a single source of truth for their data ecosystem and brings the power of AI and ML-driven insights to more people throughout an organisation.
To successfully build a strong data culture, organisations need to develop and execute a comprehensive strategy that enables them to more easily deploy a modern cloud-based data architecture, unlock the full potential of all their data, and future proof their investments to provide the greatest ROI.
They now have the option to move away from closed, proprietary systems offered by a variety of cloud vendors and adopt a strategy that emphasises open, nonproprietary solutions built using industry standards.
iTNews Asia: The CDOs interviewed for the study ascribe the importance to democratise analytics and machine learning capabilities to help end-users to make more informed decisions. How do we ensure the right parties have the necessary access to the data, and how can we ensure that the data is protected despite increased access being given?
While the ability to easily access, analyse and share data is crucial for fostering innovation and building truly data-driven organisations, data governance is perhaps the most challenging aspect of data transformation initiatives.
Every stakeholder recognises the importance of ensuring that data is readily available, is of high quality, and relevant to help drive business value. Likewise, organisations understand the risks of failing to get data governance right, with a heightened potential for undetected data breaches, negative impact on brand and the potential for significant fines in regulated environments.
- Andrew Martin, Head of Databricks ASEAN and India
Maintaining multiple, competing data architectures is often the culprit. Maintaining multiple, competing systems that offer varying levels of security and a mixed approach to data governance, often leads to out-of-date, unreliable data for some and limited access for others.
Organisations are looking for a new, unified approach that can offer stronger security and governance - from data lineage, auditing, and fine-grained access controls - within a single, scalable platform that offers a complete view of all the available data.
Minimising the number of copies of data and moving to a single data processing layer allows enterprises to run all of their data governance controls together, improving data flow and ensuring that data quality and compliance are maintained throughout the organisation.
Building and maintaining a single source of truth is critical and allows data teams from across the business to feel confident they are working towards a shared outcome.
.jpeg&h=271&w=480&c=1&s=1)