Requirements elicitation, writing, analysis, and modeling by IIBA Endorsed Education Provider.

e-Data: Turning Data into Information with Data Warehousing (Addison-Wesley Information Technology Series)

Summary

TOC

Back Cover

Preface

Author

Look Inside

Comments

Reviews

Jill Dyche, Jill Dyché

February 2000, Addison-Wesley Pub Co, Textbook Binding, 384 pages, ISBN 0201657805

Try our new Virtual Workshopsand e-Coaching
for today's Business Analysts (BA's) and Subject Matter Experts (SME's)

Over the last ten years the strategic use of detailed data has changed the face of business. This change was made possible through the use of data warehouses, which are now widely accepted for their role in the delivery of decision-support and business-intelligence applications. Today's data warehouses are the critical hubs of such burgeoning strategic initiatives as e-commerce, knowledge management, database marketing, and customer relationship management. Given this, a working knowledge of the fundamentals of data warehousing is essential for today's executives, managers, and other professionals who must maximize the power of data warehousing in both existing business contexts and future strategic initiatives.

Written especially for these business professionals, e-Data: Turning Data into Information with Data Warehousing covers data warehousing and its surrounding technologies in a straightforward and engaging way, illustrating how companies are leveraging their data warehouses to serve a wide range of business needs. This book clearly lays out what business people should know about data warehouse implementation and the best techniques for evaluating and justifying new data warehouses and data marts. This book provides:

Definitions of key data warehousing terms
Descriptions of emerging database marketing applications that mandate detailed data
A primer on data warehouse technologies, as well as a clear taxonomy of different analysis types
Staffing and hiring tips for data warehouse development teams
A review of the diverse uses of business intelligence across various industries
Key questions to ask your vendors and consultants
A fresh perspective on the politics involved with data warehouses
Checklists and success metrics for evaluating data warehouse effectiveness
Coming trends in the use of e-data in business

Inspirational real-world case studies and staff profiles appear throughout, showcasing data warehousing's "vanguards"--companies that have succeeded in achieving long-term financial and strategic benefits. Included are Bank of America, Charles Schwab & Co., Qantas Airways, GTE, Royal Bank of Canada, Sears, and Twentieth Century Fox.

e-Data provides invaluable information about data warehousing as a whole, its development and strategic value, the technologies that support it, and its effect on corporate decision making--information that will enable you to turn a gold mine of raw data into valuable information, position your company for market leadership, and enhance customer satisfaction.

Topics covered: Data warehouses, decision support systems, data mining, target marketing, cross-selling, sales analysis, industry applications, database tools, vendor selection, project planning, and pitfalls.

Foreword xvii
Acknowledgments xxi
About the Author xxiii

Introduction xxv: The Book and Its Purpose xxvi; You the Reader xxvii; Content Overview xxviii; Part I: Getting the Value xxix; Part II: Getting the Technology xxix; Part III: Getting Ready xxix; A Case Study Sneak Preview xxx; Requisite Caveats xxxi

PART I Getting the Value 1

CHAPTER 1 WHAT IS A DATA WAREHOUSE ANYWAY? 3

The Data Warehouse Defined 4
Data Warehousing, Decision Support, and Business Intelligence 6
The Data-Warehousing Bandwagon and Why Everyone Jumped on It 10: Data-Warehousing Objectives 14
Some Trite Data-Warehousing Aphorisms 15
Venus and Mars:How IT and Businesspeople Communicate 16
Some Other Buzzwords and What They Mean 19
Some Lingering Questions 21

CHAPTER 2 DECISION SUPPORT FROM THE BOTTOM UP 23

The Evolution of Decision Support 25: Standard Query: The Workhorse of DSS 25; Multidimensional Analysis: The Power of Slice 'n' Dice 27; Modeling and Segmentation: Analysis for Knowledge Workers 28; Knowledge Discovery: The Power of the Unknown 29
Some Real-Life Examples 30: Standard Queries 31; Multidimensional Analysis 32; Modeling and Segmentation 35; Knowledge Discovery 36
Wherefore Data Mining? 38
Data Warehousing in the Real World 40
What It Takes to Get to the Top 41

CHAPTER 3 DATA WAREHOUSES AND DATABASE MARKETING 43

Customer Relationship Management 44: Customer Segmentation 46; Individual Customer Analysis 48; Case Study: Bank of America 50; A Word About CRM Technology 52
Popular Database-Marketing Initiatives and What They Mean 54: Target Marketing 54; Cross-Selling 56; Sales Analysis and Forecasting 58; Market Basket Analysis 60; Promotions Analysis 61; Customer Retention and Churn Analysis 62; Profitability Analysis 64; Customer Value Measurement 65; Product Packaging 66; Call Centers 68; Sales Contract Analysis 69
Database Marketing Lessons Learned 71
Some Lingering Questions 73

CHAPTER 4 DATA WAREHOUSING BY INDUSTRY 75

Retail 76: Uses of Data Warehousing in Retail 77; Market Basket Analysis 78; In-Store Product Placement 79; Product Pricing 80; Product Movement and the Supply Chain 80; The Good News and Bad News in Retailing 82; Case Study: Hallmark 83
Financial Services 84: Uses of Data Warehousing in Financial Services 86; The Good News and Bad News in Financial Services 91; Case Study: Royal Bank of Canada 92
Telecommunications 94: U.S. Local Service Carriers 95; U.S. Long-Distance Carriers 96; International Long-Distance Carriers 96; Wireless Carriers 97; Uses of Data Warehousing in Telecommunications 97; The Good News and Bad News in Telecommunications 102; Case Study: GTE 103
Transportation 105: Yield Management 106; Frequent-Passenger Programs 107; Travel Packaging and Pricing 108; Fuel Management 108; Customer Retention 109; The Good News and Bad News in Transportation 110; Case Study: Qantas 111
Government 113: The Good News and Bad News in Government 114; Case Study: State of Michigan 115
Health Care 117: Uses of Data Warehousing in Health Care 117; The Good News and Bad News in Health Care 121; Case Study: Aetna U.S. Healthcare, U.S. Quality Algorithms 122
Insurance 123: Uses of Data Warehousing in Insurance 124; The Good News and Bad News in the Insurance Industry 126; Case Study: California State Automobile Association 127
Entertainment 128: Case Study: Twentieth Century Fox 129
Some Lingering Questions 131

PART II Getting the Technology 133

CHAPTER 5 THE UNDERLYING TECHNOLOGIES: A PRIMER 135

Data Warehouse Architecture 135: The Operational Data Store 139; Two-Tier Versus n-Tier 139; Middleware 142
Databases and What They're Good For 143
Multidimensional Databases 146
Metadata 148
Disseminating the Information: Application Software 150: Graphical User Interfaces 150; A Word About the Web 152
Development Definitions and Differentiators 153: OLAP Subcategories 153; Data Modeling and Design Tools 154; Data Extraction and Loading Tools 156; Management and Administration 158
Putting It All Together 160
Some Lingering Questions 162

CHAPTER 6 WHAT MANAGERS SHOULD KNOW ABOUT IMPLEMENTATION 165

What You Should Know About Data Warehouse Methodologies 166: Evaluating a Methodology 168
The Data Warehouse Implementation Process 169: The Steps in Data Structure and Management 171; The Steps in Application Development 172; Who Should Be Doing What? 173; Development Job Roles and Responsibilities 173; Consultants Versus Full-Time Staff 181; The Lost Fine Art of Skill Delineation 184
Good and Evil Square Off:A Tale of Two Project Plans 190
Executive Involvement on the Project 192: Profile: Hank Steermann of Sears, Roebuck and Co. 196
Some Lingering Questions 198

CHAPTER 7 VALUE OR VAPOR? FINDING THE RIGHT VENDORS 201

The Hardware Vendors 202: Five Questions to Ask Your Hardware Vendor 204
The Database Vendors 205: Five Questions to Ask Your Database Vendor 206; TPC Benchmarks 207
The Application Vendors 208: Five Questions to Ask Your Application Tool Vendor 211; Data-Mining Tools: A Breed Apart 212; Ten Questions to Ask Your Data-Mining Vendor 213
The Consultants 213: The Big Guys 214; The Little Guys 217; A Word About the Analysts 218; A Word About the Vendors 219; Five Questions Your Consultant Should Ask You 219
The RFP Process 222: The Components of a Good RFP 222; A Sample Table of Contents 226
Some Lingering Questions 227

PART III Getting Ready 229

CHAPTER 8 DATA WAREHOUSING'S BUSINESS VALUE PROPOSITION 231

Return on Investment 231: Hard ROI: The Tangible Benefits 234; Soft ROI: The Intangible Benefits 239
Budgeting for the Data Warehouse 241: Technology Costing 242; Resource Costing 243; Obtaining Funding--But Not Too Much! 245
Data Warehouse Operations Planning 245: Developing an Operating Plan 246
Are You Ready for a Data Warehouse? A Quiz 251: Data Warehouse Readiness Score 254
Some Lingering Questions 254

CHAPTER 9 THE PERILS AND PITFALLS 257

The New Top 10 Data-Warehousing Pitfalls 258: Pitfall #1: The Data Warehouse as Panacea Syndrome 259; Pitfall #2: They Talked to End-Users--But the Wrong Ones! 260; Pitfall #3: Too Much Time Spent on Research, Alienating Constituents 261; Pitfall #4: Bogging a Good Project Down by Creating Metadata 262; Pitfall #5: Being Sidetracked by "Neat to Know" Analysis 262; Pitfall #6: Adopting Decision Support Without Supporting Decisions 263; Pitfall #7: Greediness on the Part of Development Organizations 263; Pitfall #8: Lack of "Internal PR" 264; Pitfall #9: Failing to Acknowledge That DSS Applications Are Finite 264; Pitfall #10: Overemphasizing Development and Ignoring Deployment 266
Thinking of Outsourcing? 267
Data Warehousing's Dirty Little Secrets 269
The Politics of Data Warehousing 276: The Top 10 Signs of Data Warehouse Sabotage 278
The Vanguards of Data Warehousing 279: Case Study: Charles Schwab & Co., Inc. 282

CHAPTER 10 WHAT TO DO NOW 285

If You Need a Data Warehouse 285: Establish Up-Front Success Metrics 286; Consider Benchmarking 287; Research External Staff 288; Prepare Your Environment 289; Classify Your Stakeholders 289; Ramp Up Support Capabilities 293; Profile: Philippe Klee, Qantas Airways 293; Look Outside Your Box 294; Solicit a Request for Information 295
If You Already Have a Data Warehouse 296: Establish a Formal Postmortem Process 296; Inventory Existing Applications 297; Spring for an Audit 299; Improve Customer-Facing Business Processes 300; Establish a Closed-Loop Process 302; Go Web, Young Man! 302; Case Study: Allsport 303; Consider Branching Out Vertically 305; Consider Branching Out Horizontally 306
If You Have a Data Mart or Marketing Analysis System 308: Share Your Toys 308; Migrate to Enterprisewide 309
An Insider's Crystal Ball 311: Clickstream Storage 311; Enterprise Resource Planning 312; Extending the Data Warehouse to External Vendors 313; Customized Web Portals 314; Real-Time E-Marketing 316; Privacy 317
The Whole Truth 318

Appendix: Haven't Had Enough? Suggested Reading 321: Business Books 321; Technology Books 323; Web Sites 324

Index 327

Data warehouse adoption is increasing 30-40% per year, and data warehouse technology is evolving rapidly to serve new requirements. e-Data: Turning Data Into Information with Data Warehousing is the complete manager's briefing on what this technology can do today, and how to achieve optimal results.

Start by understanding why data warehousing has generated so much excitement throughout the business community;
review four types of decision support analyses commonly performed with data warehouses;
understand how data warehouses support powerful new database marketing applications; and
see data warehousing at work in a wide range of industries.

Next, master data warehouse technologies, understand today's best practices for implementation and staffing, and learn the tough questions to ask vendors before you make product commitments.

e-Data includes

detailed metrics for evaluating the success of a data warehouse;
techniques for identifying both "hard" and "soft" benefits; and
"from the trenches" advice on what to do next, once your data warehouse is running successfully.

For IT managers, business professionals, consultants, and others who need to understand the benefits of the latest data warehousing technology.

Foreword

E-commerce. Knowledge management. CRM. ERP. Smart cards. Data mining. It's true that in the vast realm of technology, the term data warehousing has recently ceded ground to some whiz-bang buzzwords. Even with data warehouse adoption rates steadily increasing by 30 percent a year, the Web and its patois have drowned out discussions of even more advanced technological developments, rendering state-of-the-art technical breakthroughs a second-page story.

At first, I was a bit depressed by all the Web hype. Inasmuch as data delivery was critical to the enterprise, you still needed to store that data someplace, Internet or no. With all the hullaballoo about Y2K and Web portals, had data warehousing simply faded away?

Recent customer experiences quickly shook me awake. Not only have data warehouses not faded away, they've assumed center stage again. While certain terminology might ebb and flow--data warehouses are now synonymous with "decision support" and "business intelligence" and naturally symbiotic with all things Web--the data warehouse is in fact the hub in the wheel when it comes to many companies' most important strategic initiatives.

Attend any conference these days, whether focused on industry, marketing, or technology, and there's bound to be a presentation on customer loyalty programs, retention, or customer relationship management (CRM). Sometimes supply chains and even business process reengineering still rear their heads. The point here is that regardless of what the business initiative is, the data warehouse will likely play a central role in its execution by making key data available to a cross-section of the business.

In this book, the term e-data refers to data that has been intelligently modeled, cleansed, and consolidated into a data warehouse so that it's meaningful and useable by business people. The fact is, e-data is more important than ever. Whether you have a data warehouse, a data mart, or a decision support application (chapter 1 defines the differences), or are considering stepping up your CRM, e-commerce, or target marketing programs, having clean, consolidated information is no longer a nice add-on; it's a necessity. This book explains how e-data and a data warehouse can solve a wide range of business problems and provides real-world examples from a diverse set of industries, countries, and companies.

The Book and Its Purpose
A lot has been written about data warehouses. Development methodologies, database design conventions, and system architectures have been surveyed in a myriad of technology books, most of them discerning and clear. These books have pinpointed a market eager for information on data warehousing's technology components and how to integrate them. They are important for practitioners, offering tips on eclectic subspecialties such as data replication, star schema design, concurrency planning, and horizontal database partitioning. They deconstruct the development lifecycle and guide readers through critical processes that are fundamental to data warehouse development.

This is not one of those books.

Rather, it's a book for those of us who aren't interested in lofty technical dissertations but whose work nevertheless touches corporate data in some way. Those who are keen on getting the information the data warehouse can deliver, are hiring staff who will use it, and are interfacing with their technical colleagues in making it all work.

We need to understand what data warehouse technology really does, in common terms, and why it's right for our companies. While we're not interested in implementing it, we'd like to differentiate the well-worn buzzwords. We'd appreciate some implementation scenarios as they pertain to data warehouses and why they're used, and checklists of success criteria. We want to know how e-data can aid in marketing, assist our companies in winning customers--sometimes for the second time--and help us eat our competitors' collective lunch. We want trenchant examples and are hungry for tips from those who've realized the vision. We want to understand what data warehouses will do for us, as well as what they will not.

In short, this is a book for the rest of us.

You the Reader
Readers of this book are most likely business professionals with limited technical expertise or people who have learned a bit about technology in spite of themselves. However, technicians and practitioners might find this book a refreshing review, especially in light of the real-world case studies it presents. The audience for this book thus encompasses a wide range of readers, including those listed below.

Executives and managers will glean a lot of practical information from this book, both in terms of how to tell whether a data warehouse is the right solution for the business problem at hand and how to determine whether an existing data warehouse is living up to its value proposition.

Businesspeople whose thirst for new information alone is often justification for a data warehouse will be interested in how data warehouses are being used in various industry and marketing capacities. The book introduces concepts and terms that managers and end-users alike can learn in order to speak the same language as their information technology (IT) colleagues, ensuring that their business requirements are understood and addressed, and offers several checklists against which to gauge data warehouse readiness.

Marketing experts, including product managers, merchandisers, and strategic planners, can read about key corporate initiatives that directly leverage data warehouses.

IT managers will find this book a practical tool in confirming the requirements for successful data warehouse delivery. The book includes a variety of metrics and success factors with which technology management can measure its efforts or bolster its preparatory activities.

Consultants, too, will find this book useful; they can employ the various checklists and matrices in order to evaluate staff and review delivery success metrics, as well as to prepare their practices for what's on the horizon. Project managers, both administrative and technical, can translate the information for their own implementation strategies, supplementing both their project plans and the methodologies that drive them.

Finally, technical practitioners and implementation team members can use this book for review; in the process they may discover a thing or two about how other companies are implementing their data warehouses and as a result refine existing development activities.

Content Overview
This book provides an evolving look at data warehousing, from its various definitions to its place in the overall corporate infrastructure to its variety of uses. You can either read the chapters linearly or go directly to the areas that interest you most.

Certain readers might surmise that the book focuses on the technology platform, the data warehouse hardware itself. This approach would be like writing a book about television and discussing the electronic circuitry of the television set rather than the actual shows. While the book does explain the underlying technology involved in data warehousing by way of framing the picture, it nevertheless focuses more on "what's inside" the data warehouse, not to mention the prevalent audience. In short, the book is about what data warehouses do for a business.

The book is divided into three parts, which categorize the chapters into high-level areas. Below is a thumbnail sketch of the book's organization and contents.

Part I: Getting the Value
Chapter 1, "What Is a Data Warehouse Anyway?," discusses why data warehouses have seized hold of the corporate Zeitgeist, introducing some key concepts and exposing some of the trite aphorisms currently touted by the so-called experts.

Chapter 2, "Decision Support from the Bottom Up," presents an e-data analysis taxonomy for the data warehouse. It describes the four main types of business intelligence that call for data and offers some examples on their usage.

Chapter 3, "Data Warehouses and Database Marketing," outlines both the popular and the emerging database marketing applications that focus on customers while leveraging data, explaining their origins and business benefits.

Chapter 4, "Data Warehousing by Industry," covers the gamut of industry sectors and what they're doing with e-data and data warehouses, using case studies to illustrate various usage scenarios from real-world companies.

Part II: Getting the Technology
Chapter 5, "The Underlying Technologies: A Primer," not only presents some of the baseline technologies and technical concepts involved in data warehousing but also covers some of the technical activities involved in development.

Chapter 6, "What Managers Should Know About Implementation," exposes the often arcane world of data warehouse development, the methodologies it employs, and some of the well-worn staffing mistakes that get development managers into trouble. In addition, it offers some tactical hiring guidelines for you to use when conducting your next round of interviews.

Chapter 7, "Value or Vapor? Finding the Right Vendors," presents metrics for assessing the data warehouse solutions that fit best with your organization and its unique needs, including hardware, database, application, and consulting evaluation criteria.

Part III: Getting Ready
Chapter 8, "Data Warehousing's Business Value Proposition," explains how to justify your data warehouse in terms of both "hard" and "soft" benefits and offers ways to continue justifying the warehouse over time.

Chapter 9, "The Perils and Pitfalls," presents several sets of metrics in order to outline why some customer data warehouses succeed while others fail. Not content with offering the negatives, this chapter concludes with a list of what the "vanguards of data warehousing"--those companies attributing improvements of several orders of magnitude to their data warehouses--have in common when it comes to successful decision support delivery.

Chapter 10, "What to Do Now," provides some advice from the trenches on how to continue your data warehousing journey, whether you're a seasoned traveler or are just breaking in your boots.

The appendix of supplementary reading material provides a guide to other recent works for those readers who want to learn more about either the business or the technology side of e-data.

A Case Study Sneak Preview
This book is replete with both real-world case studies of companies that use data warehouses and profiles of staff members and their roles in data warehouse development teams. For example, you'll see the following processes in action.

By using customer segmentation, Bank of America is getting to know its customers even better.
Charles Schwab & Co., Inc. is applying the same customer satisfaction principles on which it has built its leading brokerage business to its data warehouse end-users.
Qantas Airways was able to predict the Asian economic crisis with the data warehouse, and is gearing up for an encore.
GTE is socializing e-data across the enterprise and across the country.
The California State Automobile Association is doing more than delivering new marketing programs with its data warehouse, it's motivating cultural change.
Canada's largest bank, Royal Bank of Canada, doesn't let different vendors get in the way of delivering best-of-breed e-data across its business.
The State of Michigan's Family Independence Agency uses its data warehouse to behave more like a cutting-edge commercial business than a government bureau.
Twentieth Century Fox may well change the face of the entertainment industry with its data mart.

These case studies and others should at a minimum serve as examples by which you can measure your own progress with e-data, and at best provide you with some great role models.

Requisite Caveats
This book is replete with examples of both successes and failures. It takes on some of data warehousing's sacred cows, including exalted methodologies, big consulting companies, venerated data models, and empire-building managers. Of course, there are exceptions to these and other evils portrayed in the book.

Most technology books abstain from discussions of specific vendors for valid and practical reasons. However, because of this book's heavy emphasis on real-world examples, specific vendors pop up here and there, particularly in chapter 4, where most of the case studies mention the company's chosen data warehouse platform.

New companies and technologies are emerging every day, and I apologize to those vendors that may have slipped through the cracks. The technologies discussed in the book are those of particular interest to the primary audience, that is, businesspeople, and thus a mere nod of the head to the many worthy data warehouse software companies that target--and are of greater interest to--the IT side of the house.

Jill Dyché is a partner with Baseline Consulting Group, a specialty consulting firm focusing on the delivery of business intelligence solutions across industries. Since 1985 she has been working with Fortune 1000 companies worldwide to help align strategic technology initiatives with corporate business objectives. Jill is a frequent speaker at technology and marketing conferences, and her articles have been featured in a variety of publications: Information Week, Oracle magazine, Teradata Review, Telephony Magazine, The Washington Times, and The Chicago Tribune.

Amazon.com
The concept of data warehousing can be very hard to grasp at first--especially if you're not a database person. In e-Data: Turning Data into Information with Data Warehousing, author and data warehousing expert Jill Dyché offers the big picture of data warehousing in an informative and comfortable read.

Although the title may spark notions of an Internet-specific topic, the author's term "e-data" isn't a Net-related notion at all. By e-data, she means any data that has been refined and stored in a data warehouse, whether Internet-related or not. She explains the basic concepts behind data warehousing and decision support systems in refreshingly plain English and provides real-world case study summaries of well-known corporations such as Hallmark and Bank of America, as well as the applications of data warehousing by industry segment.

Some of the technologies that make data warehousing possible are discussed, but the book is primarily targeted at managers and executives who are responsible for implementing successful marketing and data management strategies. The discussion is lifted above the technical details of how data warehousing takes place to examine why your organization should consider the approach.

There is plenty of focus on the daunting task of implementing a data warehouse, and the author provides many tips for selecting the proper consultants, technologies, and staff to do the job. This text is a great real-world introduction to the sphere of data warehousing. --Stephen W. Plain