An Overview of Business Intelligence Technology

Chaudhuri, S., Dayal, U. & Narasayya, V., 2011. An overview of business intelligence technology. Communications of the ACM, 54(8), pp.88—98.

This article discusses - based on an overview of a typical business intelligence architecture - selected business intelligence components, their use, research issues, and optimization potentials.

Introduction

Business Intelligence (BI) software draws upon a company's data assets for providing fact-based decision support that enables knowledge workers to make better and faster decisions.

Typical application areas of BI are:

1. manufacturing (order shipment and customer support)
2. retail (user profiling for customizing coupons, and product offers)
3. financial services (claim analysis and fraud detection)
4. transportation (fleet management, customer care)
5. telecommunications (identification of customers likely to switch to another provider)
6. utilities (power usage analysis)
7. health care (outcomes analysis)

Business Intelligence Architecture

A typical BI Architecture consists of

1. data sources such as a company's databases and external sources
2. data movement and streaming engines that extract, transform and load (ETL) data from external sources into the data warehouse servers. They address the issues of data quality and provide data profiling tools for discovering problems such as missing entities and attribute values, violations of constraints, etc.
3. data warehouse servers that store the normalized data
4. mid-tier servers, providing functionalities for different BI scenarios and applications based on components such as
• Online Analytic Processing (OLAP) servers
• enterprise search engines
• data mining and extraction engines that currently primarily extract the following structured data: (i) named entities, (ii) concepts and topics, and (iii) text sentiment
• reporting servers
5. Front-end applications, such as search and query frontends, visual dashboards, and export functions.

Technologies Used

This section contains selected samples of the technology used in business intelligence systems.

1. Column-oriented storage for databases. Advantages: optimized for "read-mostly" workloads, better data compression, limit read operations to the columns required
2. Data compression - reduces IO and storage requirements; certain query operations such as de-duplication and equality operations may take place directly on the compressed content. Dictionary compression identifies frequently used values and transforms them into a more compact representation (e.g. {active, deleted} --> {0, 1})
3. Real Time BI - reduces the latency between data generation and analysis. Complex Event Processing (CEP) engines that operate directly on data streams rather than on data that needs to be loaded into the data warehouse are often used for such tasks.