Software Testing In The Real World: Improving The Process

Description

Software testing.
It is exhilarating. It is exasperating.
It can be the best of jobs––or the worst.
It is fundamental to delivering quality software on time within budget.

This book is for anyone interested in improving their test process, including test specialists, product developers, managers, quality assurance specialists, maintainers, project managers, auditors, systems analysts, tool developers, and others whose jobs involve understanding software testing. It offers developers and other people whose work is not primarily in the testing area an opportunity to improve their own specialty through a better understanding of testing.

To understand testing and to improve testing practice, it is essential to see the software testing process in its broadest terms––as the means by which people, methodology, tools, measurement, and leadership are integrated to test a software product. More than 20 years in testing practice, teaching, and consulting have shown me the following key questions raised again and again by people wanting to improve their testing process: Methodology questionsWhat are the key testing methods, when do we use them, and how do we implement them?How do we prevent defect migration and improve defect finding capabilities?How can we understand and leverage the relevant test standards and terminology?How do we create meaningful test plans and other critical test deliverables?How do we identify and prioritize potential software testing process improvements?Leadership questionsHow do we organize the software testing effort so it is measurable and controllable?Where can we find additional help regarding software testing?How can we ensure that improvements are sustainable in the medium and long term?How can we organize testing for the most effective relations between people?How can we generally maximize exhilaration and minimizeexasperation?Tools and technology questionsWhat are the most significant opportunities for automating the testing process?How can we use measurement to understand and control the software test process?How can we select test tools and vendors and implement an effective tools program?How can we use risk analysis techniques as a basis for decision making in testing?

My experience has shown that a balanced strategy addressing methodology, leadership, and technology is most effective for improving testing.

Most software development organizations suffer from immature software processes. Given this starting point, practitioners and managers naturally ask: "Yes, but what do I do now?". This book offers a tool–box for effectively improving the software testing process. A tool–box is not a single methodology to be followed from A to Z. It is a menu of techniques that can be taken individually or in combination to provide the means to formulate and reach realistic improvement objectives.

Part I is for orientation. It describes the six essentials of software testing, the history of testing, and a simple, practical approach to getting started with improving the testing process. It identifies software engineering in general and testing in particular as new disciplines grappling with the escalating demands of an environment expecting miracles.

Part II establishes practical goals from a "now we are here" position, and explains how these relate to formal and less formal definitions of testing and testing objectives, and to the current standards that are fundamental (and useful) to practitioners in their everyday work.

The basic forms of the testing process are explained, and the critical real–world issues of cost and risk are fully clarified and integrated in the decision making process of what, when and how to test. In this part we address the items that need to be in place before we dig more deeply into the testing activities of verification and validation. The issues of planning, software engineering maturity goals, configuration management, standards, and tools are explained and positioned.

Part III explains the basic verification and validation testing tasks, including planning and controlling testing costs, and making the best use of resources. The use of tools as a way of gaining leverage in the various testing activities is explained, together with clarification of tool categories, and practical guidance on the critical questions for tool acquisition. The appropriate and constructive use of measurement is explained, and guidelines given for the proper use of measurement results.

Part IV explains the organizational and human management issues that surround the test effort and which are key determinants of its success. It surveys some current best practice sources, and then refocuses on how to get sustainable improvements in place in the short and medium term.

Definitions, references, and examples are integrated within the main text of the book as far as possible. The appendices contain a variety of material that is too detailed for inclusion in the main text of the book, and which is essentially complementary. This includes sample checklists, exercises, planning documents, and sample working documents. There is further detail on official standards and specifications as well as useful help sources such as journals, tools, conferences, newsletters, user groups, and an annotated bibliography.Acknowledgments

Life provides many teachers. Some say all of life is our teacher. Space prohibits acknowledging all who have positively influenced this book, but does allow acknowledgment of the following people due special appreciation.

Great appreciation is due my wife, Cathy Baldwin, who not only provided continuous moral support and endless editing of the book, but also provided encouragement during the early start–up years of Software Development Technologies. Special thanks to our families for their support over the years.

I am particularly grateful to David Gelperin and Bill Hetzel: to David for providing the opportunity to teach my first course on software testing nearly a decade ago, and for providing the ongoing friendship and support that kept me going; to Bill for encouraging me to write a book and forwriting the foreword for the book; and to both of them for providing the opportunity to be associated with Software Quality Engineering for many years.

Appreciation is due Chris Larson, who for 15 years has generously shared his friendship and wealth of software engineering experiences gained from over 25 years with Hewlett Packard, IBM, Tandem, and CDC, and who contributed enormously to the course on which this book is based.

Appreciation is also due Susannah Finzi for being an outstanding friend and editor, and for keeping the book appealing from a global perspective while patiently enduring seemingly endless rewrites without a single complaint.

Appreciation is due the reviewers who gave their time to providevaluable comments, especially Bill Hetzel, David Gelperin, Dot Graham, Marty Gipson, Vijay Sarathy, Ray O‘Neal, Ellen Brodbine, Trevor Reeve, Cathy Baldwin, Chris Larson, Denise Leigh, and Paul Claydon.

Finally, appreciation is due to the students who attended the courses upon which this book is based and who in the process became my teachers and provided suggestions for improving the course, especially the participants from Hewlett Packard, Motorola, Microsoft, Apple Computer, US Army, IBM, NCR, Johnson & Johnson, Grumman Data Systems, Australia and New Zealand Bank, ADP, EDS, PeopleSoft, SCO, General Electric, Oracle, NEC, Lloyds Bank, USDA, NetFrame, Cadence, Sears, AT&T, Informix, British Aerospace, Sybase, Tandem, and Octel Communications.

Ed Kit
Cupertino, September 1995 0201877562P04062001

Backcover Copy

"I really enjoyed the book. If I had written a book on testing, it would have resembled Ed Kit‘s. His focus on the testing process is excellent."
––Greg Daich, Senior Software Engineer, Science Applications International Corporation and member of the Software Technology Support Center (STSC) Test Group

"The book is easy to read and suitable for anyone interested in how to achieve better testing...Software Testing In The Real World should go a long way towards helping many of us make practical and lasting improvements... I encourage you to ‘test‘ it out."
––Bill Hetzel, President, Software Quality Engineering (from the Foreword)

"The Ed Kit book will be a good one. It has a nice practical approach, and brings testing up to date with recent developments."
––Barry Boehm, Director USC Center for Software Engineering

Software Testing In The Real World provides the reader with a tool–box for effectively improving the software testing process. The book gives the practicing software engineer a menu of techniques with guidance on how to create a strategy for continuous, sustainable improvement within their organization––whatever its size or level of process maturity.

Ed Kit addresses the most frequently asked questions about methodologies, tools, technology and organizational issues being posed in the testing community today. Pragmatic in its approach, the book confronts the problem of the relative immaturity of the software engineering discipline in most organizations with practical guidance on cost and risk, standards, planning testing tasks and testing tools.

Test and Quality Assurance Specialists, Developers and Project Managers alike will benefit from the practical, proven techniques for improving testing as well as the specific "best of breed" software testing tools information.

Author Bios

Ed Kit is an international software consultant with over 20 years of experience in software engineering. He is currently the president and founder of Software Development Technologies, a corporation providing consulting and educational services dedicated to improving the processes used to create and maintain software. Included among his many clients are Microsoft, Hewlett–Packard, Europay International, Sun Microsystems, Apple Computers, PeopleSoft Inc., Tektronix Inc., and Tandem Computers. He has delivered consulting and educational services in the United States, Japan, India, Australia and throughout Europe. Graduating with a BSEE and MSEE from Purdue University, Ed has previously worked for Bell Laboratories and Tandem Computers.

0201877562AB04062001