Has Reference Data outgrown the humble Data Model?
John Randles CEO PolarLake
What would be the outcome if you commissioned a group of traditional Reference Data Management experts to build the World Wide Web back in the early 90s? As a renowned expert in Data Modeling there is nothing for it but to build a Data Model, identifying all attributes and relationships to be used for the sum of experiences demanded by the user. You have gifts as a Data Modeler your peers can only dream of. And who cares if you end up with 25,000 attributes in your Data Model – you understand them far better than any mere mortal could and are determined to show your mastery of modeling tools, CERN will be impressed. You would tell CERN as long as they put in place the right “governance” process all will be well.
Thankfully this wasn’t the approach taken in CERN when building the original World Wide Web. The complexity of non-stop change, massive data volume, conflicting demands and classifications of Data is what the World Wide Web is known for. The future of Reference Data Management needs to take the same approach.
We have gotten to a stage where traditional Reference Data Management approaches have outlived their useful life. The demands of Reference Data Management in 2010 are far beyond what was envisaged 15-20 years ago, where the roots of today’s Data Model centric solutions lie. The world then was one of known knowns, a couple of feeds, simple asset classes and a handful of Data consumers. The world has changed dramatically and the key word we hear constantly from the Reference Data Management community today is scale – and they mean scale in every direction. The next generation of Data Management solutions will be built to address World Wide Web style scale – massive volume with non-stop change.
Scale can be defined in the Reference Data Management world in 2010 as addressing:
- Can I onboard my data fast enough and not endanger my batch window?
- Can I scale the number of feeds I onboard as quickly as the business needs them?
- Can I move from a world of end of day pricing to intra pricing with my current environment?
- How can I scale development efforts over the next 24 months to meet the 200+ regulations coming down the track?
- Can I control my hardware spend to meet these requirements?
- Do I add to the Data explosion in my firm by endlessly duplicating “golden copies” in order to meet individual consumer’s Data requirements or management of conflicting classifications schemes?
- Can I find what I am looking for in this sea of data?
- Do I believe my Reference Data architecture is fit for purpose for the next 10-15 years given the current trends in demands made on it? (it’s only getting more complex not less)
RDMBS technology was first documented back in 1970 and has served us well. However the next generation Reference Data Management platforms will be based on Semantic Web Technology, truly applying the World Wide Web technology to the Reference Data problem. If we keep doing the same things (monolithic Data Models) we invariably get the same results (frustration and stagnation).
Talk to us on our latest product release to understand how applying Semantic Web technology moves us to a post relational world for Reference Data Management. All the benefits of the RDBMS Data Model but built for 21stCentury speed and scale.
2012