Master Data System

A Master Data System provides a centralised source of common business data that is used across multiple systems and processes (see Wikipedia – master data management).


The Problem

Lonsec is an Australian company providing financial research and tools to financial planners. They gather and process investment data from many different market sources, and they have multiple teams and business units that use different tools and data repositories for their business processes.

Lonsec create complex sets of business data that is overlaid across the investment data, including research ratings, asset class/sector hierarchy and many other product attributes. Previously this data was held (and duplicated) in many systems, causing considerable maintenance and data quality issues to ensure that data changes were applied and reconciled between systems.

Furthermore, the rapid evolution of new investment products could not be handled by their legacy systems, and a major system refactoring was required. For example (and this is a major simplification), the legacy system had one set of tables for Managed Funds (with unit prices and fund manager attributes) and another set of tables for Stocks (with market prices and share market attributes). However, the growth of Exchange Traded Funds (ETFs), Listed Investment Companies (LICs) and other Hybrid products meant that Investment Products started to share attributes of both Funds and Stocks, which broke hardcoded assumptions made through large numbers of legacy tools.

Typically, the discovery that an investment product needed additional product attributes to adjust or remove some functionality or behaviour from the system would be made during an investment rating process, when the research team were working to a rating deadline and the IT systems could not be changed in time.

The Solution

A major system refactoring was required to solve existing problems and high maintenance costs, and to allow new investment products or new product attributes to be efficiently configured with minimum risk or changes to dependent systems.

Firstly, investigation identified master data attributes that were maintained and duplicated across five different systems. A strategy to centralise these was formed, and a review and re-structuring of these attributes was made to address complex product attributes and to overcome historic legacy limitations.

The Master Data System was based around a SQL Database with a highly data-configured dynamic schema, so that new products and attributes could be added without needing to make any schema changes to the system. A powerful evaluation language was implemented to add configurable business rules and validations to the system. Together these enabled “live” system changes to attributes and rules, without needing software changes or releases to support new products.

Many other features were added to the Master Data System to save maintenance time and improve data quality. For instance, editing some product attributes could be automatically propagated to related products, or warnings could be raised for data inconsistencies. All of these rules and warnings were data configured, so could be easily reviewed or changed by the business.

The initial deployment was complex as current master data values had to be imported from five (5) different systems, and loaded into the Master Data System. All five other systems were concurrently released to disable local editing of Master Data, and to implement real-time data feeds that would synchronise master data changes into these systems. This was highly successful and went without a hitch.

The Benefits

There are many operational benefits from the Master Data System including improved data quality, reduced maintenance costs, and ensuring data changes are synchronised across all related systems and tools.

The main strategic benefit is that Lonsec can easily cope with new Investment products or configuration of new product attributes in just a few hours, enabling them to be more responsive to their customers, and to quickly create new research products or services.

This gave Lonsec a very strong architectural platform to support their research business for many years to come.