Should We Keep All Our Data Eggs In One Basket?

Should we keep our data eggs all in one basket? The idiom “Don’t put your eggs in one basket” has been around for centuries. Our peers tell us that eggs are precious and delicate objects and should be handled with care, so if we carry them in a single container and we drop it, we risk losing them all.

In the world of business these words often cause much sweating and heated discussions. Should a company put all its efforts into one plan or one course of action, should they risk everything and put their money and reputation on the line in support of one idea?

Important decisions to be made I’m sure you will agree, but does this also apply to business data?

Over the last 20 or so years most data warehouses were implemented according to the definition that your data (eggs) should be stored in one physical location (one basket). Companies were primarily forced to go down this route because of the hardware and software solutions that were available to them at that time. It was necessary to store all your data in one place so that you could develop a fast and efficient data warehouse solution.

Being a global RPO company BPS World uses data from many sources, and as we continue to expand our global footprint the volume of this data is ever growing.  As the manager of our Business Intelligence solution I am now pondering over the question “Should we keep all our data eggs in one basket?”

We are now in the era of BIG Data and more and more of this is being stored offsite in the cloud. Companies are finding a growing need to analyse unstructured data, and hardware and software has now progressed to a level where there is no longer the need to store everything in one single database. Mature data virtualisation servers now offer advanced data federation and on demand integration capabilities that can make multiple data sources look like one big data warehouse. This is regardless of whether they're SQL databases, NoSQL systems, or cloud-based applications and Web services. So the user is still presented with an integrated view but unaware of the multiple data warehouses that are driving it.

I am a great believer that a Business Intelligence data warehouse solution should supply our global users with the right data at the right time, and more importantly at the right level of quality to support effective business decision making and reporting. But is it now becoming of secondary importance where all the data is stored? If it is easily accessible and still presented in an integrated view, does a data warehouse become a logical concept rather than a physical one?

So, this is the dilemma I will considering. Can a global RPO company have a more effective BI solution using a multiple data basket rather than a single one? What do you think?

To disucss further please contact me on 01628 857319 or tim.hampton@bps-world.com