April 24, 2020

The explosion of streaming data has presented CMOs with many challenges – high costs, large scale ingestion, data complexity. However the Matching Engine uses Azure and Databricks to optimize the entire matching process.

A leading challenge facing music rights organizations is the proliferation of transactional streaming data. Due to the popularity of music streaming and the accompanying globalization of music, CMOs are facing the task of matching an unprecedented volume of data. Processing and matching this data places consistent pressures on internal teams, budgets and onsite servers and applications. With a predicted 1.15 billion paid streaming users globally by 2030, the volume of data will continue to increase in scale (Goldman Sachs, 2019).

The scale of music data to be processed has increased so rapidly due to music shifting from consumer ownership to access. Rather than purchasing songs and albums, consumers are streaming individual tracks, with each stream valued at a fraction of a cent. This has resulted in the requirement for CMOs to have the capability to process billions of transactions while maintaining cost-effectiveness.

From large scale data ingestion to processing complex data, no longer can onsite systems manage effectively. CMOs are turning to high performing and scalable cloud technologies to overcome these issues. Spanish Point Technologies developed the Matching Engine to address these data volume challenges presented by music streaming.

The foundation of the Matching Engine is modern cloud technologies, with high volumes of data being processed using distributed cluster computing based on Apache Spark. The Matching Engine can plug into existing systems and support CMOs overcome key challenges by leveraging these modern cloud technologies.

Large scale data ingestion times radically reduced

Through the integration of apache spark, The Matching Engine supports the rapid input and processing of high volumes of data. The time it takes to input, process and report data is radically reduced. In a recent webinar, Spanish Point Technologies CTO John Corley shared how this process is optimized, highlighting that processing that previously took 10 hours can now be processed in 5 minutes. Not only does this mean that data is processed faster, but it also frees up team resources.

Supports the transformation of varied data structures

As music becomes more globalized and new streaming platforms emerge, the types of transactional data to be processed becomes complex. Different global music organisations use files with different structures and names, despite this, they must live in one data repository . This would have been a nightmare for CMOs, however, the Matching Engine  can efficiently read, identify and transform different data types and make them available for further processing in structured relational data bases.

The Matching Engine can auto-scale to meet demand and keep costs low

In contrast to structured databases or expensive on-premise applications, the Matching Engines cloud technologies have the ability to auto-scale. This means that The Matching Engine can scale up and down to meet the demands and requirements of the data being processed by CMOs. As has been widely reported, streaming levels are highly unpredictable, so this capability is important and cost effective for CMOs.

Creates a connected environment

One of the benefits of the Matching Engine  is the ability to connect various data sources. Changes can be made quickly and conveniently at scale. In the changing landscape of music streaming, this can be a vital asset for CMOs.

The Matching Engine is built by design to address the transactional data challenges faced by CMOs. Modern cloud technologies ensure CMOs can cost effectively manage and process complex large volumes of streaming data. Join our upcoming webinar to learn more about the Matching Engine or set up a meeting with our team to discuss your matching challenges.