January 14, 2021
Spanish Point Technologies wanted to help musicians earn fairer incomes for their properties by creating a solution that helps collective management organizations accurately invoice for music royalties when their works are used in a public space. The company worked with Microsoft and Databricks to build a big data and AI platform, based on Azure resources, which matches music performances to copyright holders at scale, processing hundreds of millions of transactions every month.
”Using our Matching Engine built on Azure resources, the organizations we work with are able to reduce their processing times from 48 hours to just five minutes. It’s created a big time-saving benefit.”John Corley: Chief Technology Officer – Spanish Point Technologies
Technology has always been a big disruptor in the music industry, but it hasn’t always benefited music creators. With the fast rise of streaming services like Spotify, YouTube, and Apple Music, the value of individual plays has seen a rapid decline. As a result, music creators and publishers sometimes receive less for their works, and invoicing for royalties is a greater challenge for collective management organizations (CMOs) than ever before.
But Spanish Point Technologies, a Gold competency member of the Microsoft Partner Network, is working to change the industry. The software company builds solutions for CMOs in the music industry to help them accurately invoice for royalties and distribute payments fairly to artists and publishers. The company’s core solution, known as its Matching Engine, was built using Azure Databricks. It offers a faster way to match public performances of songs, song lyrics, scores, and other musical copyrights to the license holders, making it easier to identify who needs to be paid and who should pay them.
But building a solution that could keep up with the fast pace of streaming services was a major challenge, and it involved a completely new approach from the company’s original technology.
When the music industry relied on radio, CDs, and vinyl, invoicing for royalties was simpler. Every time a song was played or an album was bought, it generated a transaction in the CMO’s system—usually worth a few dollars. These transactions all happened in regional databases, which could competently handle tens of thousands of transactions each month, but the method quickly became outdated when streaming arrived. With streaming, millions of transactions occur every second, each worth fractions of a cent, putting significantly more pressure on CMOs’ processing capabilities.
“Some of our customers would process 500 million transactions in a given month—that’s more than a CMO would have previously processed in its entire history,” says John Corley, Chief Technology Officer at Spanish Point Technologies.
Using a traditional processing solution, end-of-month processing for CMOs would take up to 48 hours. “As streaming services and transaction volumes continued to grow, Spanish Point Technologies understood that its database approach and traditional integration technologies would no longer meet its customers’ needs.
Spanish Point Technologies re-engineered its core CMO solution from the ground up using new technologies to adapt to a more demanding market. “Our industry uses a lot of older, traditional IT solutions. We knew that if we used a pure cloud-native application, it would give us a significant advantage over our competitors,” says Corley.
The company worked with Microsoft and Databricks to build a new Matching Engine solution using Azure Databricks and Azure Data Factory, with Azure Data Lake acting as the company’s core data repository location. Matching Engine uses Azure Databricks to process transactions at a rapid pace in a CMO’s Apache Spark environment. It relies on Data Factory to ingest data and make the Matching Engine solution interoperable with other applications, such as invoicing and finance.
Whenever a transaction isn’t automatically matched to a known publisher or music creator, Azure Cognitive Search offers a fast, easy way to search the database and figure out to whom the transaction belongs. “Using our Matching Engine built on Azure resources, the organizations we work with are able to reduce their processing times from 48 hours to just five minutes. It’s created a big time-saving benefit,” says Corley.
Spanish Point Technologies also uses Azure Data Lake Storage to reduce its operating costs. “We take full advantage of the tiered storage in Data Lake Storage. After we’ve processed a data batch, it goes into low-cost storage on cool tiers,” says Corley. “We still have access to it, and we can scale usage quickly, but we’re not paying for what we don’t use.”
Since Spanish Point Technologies built the new Matching Engine, a lot has changed for the company. In helping CMOs handle growing transaction volumes with less manual effort and greater invoicing accuracy, the company has gained a worldwide reputation. “We recently started a project with the umbrella organization of all music creators in the world, CISAC, and the Azure technology behind our Matching Engine won us the contract,” says Corley. “Our Matching Engine is now used to categorize most of the musical works available in the world, assigning each one an international standard work code.”
And most importantly, the company is achieving its key objective: helping music creators and publishers—from international superstars to local legends—get paid fairly for their works. “For some of our customers, the Matching Engine has helped identify historical backlogs of music usage that haven’t been invoiced,” says Corley. “That means the CMOs can invoice for more transactions, and their members will gain more income from it.”
We take full advantage of the tiered storage in Data Lake Storage. After we’ve processed a data batch, it goes into low-cost storage on cool tiers. We still have access to it, and we can scale usage quickly, but we’re not paying for what we don’t use.John Corley: Chief Technology Officer – Spanish Point Technologies