Gadgets and Technology

The processes required to populate the warehouse focus on extracting the data, cleaning it up, and making it available for analysis. This is typically done on a daily basis after the close of the business day. A common misconception is that data warehouses are read-only systems. In fact, one of the key challenges for a data warehouse is the daily load and management of new data. It is, however, true that the factual data, once loaded, is usually not updated, but reference information will change on an ongoing basis as new requirements are identified to analyze the factual data in different ways.

The day-to-day management of the data warehouse is different from the management of an operational system, because the volumes can be much larger, and require more active management, such as creating and deleting summaries, or rolling data on and off the archive. In essence, a data warehouse is a database that is continually changing to satisfy new business requirements. Requirements evolution tends to be the most complex aspect of a data warehouse. This requires the architecture to be structured in such a way as to cope with future changes in query profiles. This evolution will also encompass the addition of completely new subject areas. In practice, a critical issue to address up front is how big the data warehouse will eventually be. The answer to this question indicates the magnitude of the total solution, and how much headroom is required from the underlying hardware and software.