Table of Contents

Preface xi

Glossary xv

Chapter 1 Fundamentals of Testing 1

1.1 Why is testing necessary? (FL 1.1) 1

1.1.1 Software systems context 1

1.1.2 Causes of software defects 3

1.1.3 Role of testing in software development, maintenance and operations 5

1.1.4 Test and quality 5

1.1.5 Terminology 6

1.2 What is testing? (FL 1.2) 7

1.2.1 Origin of defects 8

1.2.2 Common goals of testing 9

1.2.3 Examples of objectives for testing 9

1.2.4 Test and debugging 10

1.3 Paradoxes and main principles (FL 1.3) 11

1.3.1 Testing identifies the presence of defects 11

1.3.2 Exhaustive testing is impossible 11

1.3.3 Early testing 12

1.3.4 Defect clustering 12

1.3.5 Pesticide paradox 13

1.3.6 Testing is context dependent 13

1.3.7 Absence of errors fallacy 14

1.4 Fundamental test process (FL 1.4) 14

1.4.1 Planning 15

1.4.2 Control 15

1.4.3 Test analysis and design 16

1.4.4 Test implementation 18

1.4.5 Test execution 19

1.4.6 Analysis of exit criteria 20

1.4.7 Reporting 20

1.4.8 Test closure activities 21

1.5 Psychology of testing (FL 1.5) 22

1.5.1 Levels of independence 23

1.5.2 Adaptation to goals 24

1.5.3 Destructive or constructive? 25

1.5.4 Relational skills 25

1.5.5 Change of perspective 26

1.6 Testers and code of ethics (FL 1.6) 27

1.6.1 Public 27

1.6.2 Customer and employer 28

1.6.3 Product 29

1.6.4 Judgment 30

1.6.5 Management 31

1.6.6Profession 32

1.6.7 Colleagues 33

1.6.8 Self 33

1.7 Synopsis of this chapter 34

1.8 Sample exam questions 35

Chapter 2 Testing Throughout the Software Life Cycle 43

2.1 Software development models (FL 2.1) 43

2.1.1 Sequential models 44

2.1.2 Iterative models (FL 2.1.2) 47

2.1.3Incremental model 49

2.1.4 RAD 50

2.1.5 Agile models 52

2.1.6 Selection of a development model 56

2.1.7 Positioning tests 57

2.2 Test levels (FL 2.2) 58

2.2.1 Component level testing or component tests 59

2.2.2 Integration level testing or Integration tests 60

2.2.3 System tests 62

2.2.4 Acceptance tests 64

2.2.5 Other levels 66

2.3 Types of tests (FL 2.3) 66

2.3.1 Functional tests 67

2.3.2 Non-functional tests 68

2.3.3 Tests based on the structure or architecture of the software 71

2.3.4 Tests associated with changes 72

2.3.5 Comparisons and examples 73

2.4 Test and maintenance (FL 2.4) 74

2.4.1 Maintenance context 75

2.4.2 Evolutive maintenance 76

2.4.3 Corrective maintenance 76

2.4.4 Retirement and replacement 77

2.4.5 Regression test policies 78

2.4.6 SLA validation and acceptance 81

2.5 Oracles 81

2.5.1 Problems with oracles 81

2.5.2 Sources of oracles 82

2.5.3 Oracle usage 82

2.6 Specific cases 83

2.6.1 Performance tests 83

2.6.2 Maintainability tests 84

2.7 Synopsis of this chapter 84

2.8 Sample exam questions 85

Chapter 3 Static Techniques (FL 3.0) 91

3.1 Static techniques and the test process (FL 3.1) 91

3.2 Review process (FL 3.2) 94

3.2.1 Types of reviews 95

3.2.2 Roles and responsibilities during reviews 100

3.2.3 Phases of reviews 103

3.2.4 Success factors for reviews 116

3.2.5 Comparison of the types of reviews 117

3.3 Static analysis by tools (FL 3.3) 119

3.3.1 Types of static analysis 120

3.3.2 Types of defects that can be identified 125

3.3.3 Data flow analysis 126

3.4 Added value of static activities 130

3.5 Synopsis of this chapter 131

3.6 Sample exam questions 131

Chapter 4 Test Design Techniques (FL 4.0) 137

4.1 The test development process (FL 4.1) 139

4.1.1 Terminology 139

4.1.2 Traceability 140

4.2 Categories of test design techniques (FL 4.2) 143

4.2.1 Black box, white box, or gray box 143

4.2.2 Experience-based techniques 145

4.2.3 Test characteristics 145

4.2.4 Limitations and assumptions 146

4.3 Black-box techniques (FL 4.3) 147

4.3.1 Equivalence partitioning (FL 4.3.1) 148

4.3.2 Boundary value analysis (FL 4.3.2) 155

4.3.3 Decision tables (FL 4.3.3) 158

4.3.4 Other combinational techniques 162

4.3.5 State transition testing (FL 4.3.4) 163

4.3.6 Use case testing (FL 4.3.5) 170

4.3.7 Limitations and assumptions 172

4.4 Structure-based techniques (FL 4.4) 172

4.4.1 Statement testing and coverage (FL 4.4.1) 174

4.4.2 Decision testing and coverage (FL 4.4.2) 180

4.4.3 Other structure-based techniques (FL 4.4.3) 183

4.4.4 MC/DC coverage (FL 4.4.3) 185

4.4.5 Limitations and assumptions of structure-based testing 186

4.4.6 Coverage level and exit criteria 187

4.5 Experience-based techniques (FL 4.5) 187

4.5.1 Attacks 188

4.5.2 Defect taxonomies 189

4.5.3 Error guessing and ad hoc testing 190

4.5.4 Exploratory testing 190

4.5.5 Limitations and assumptions 192

4.6 Choosing test techniques (FL 4.6) 193

4.7 Synopsis of this chapter 194

4.8 Sample exam questions 195

Chapter 5 Test Management (FL 5.0) 209

5.1 Test organization (FL 5.1) 209

5.1.1 Independence levels 209

5.1.2 Roles and responsibilities 212

5.1.3 Human and contract aspects 214

5.2 Test planning and estimation (FL 5.2) 215

5.2.1 Planning and evaluation activities 218

5.2.2 Test planning activities 228

5.2.3 Test documentation 229

5.2.4 Entry and exit criteria for test activities 235

5.3 Test progress monitoring and control (FL 5.3) 239

5.4 Reporting 242

5.4.1 What to report, to whom, and how? 242

5.4.2 Statistics and graphs 244

5.5 Transverse processes and activities 246

5.5.1 Test data definition 247

5.5.2 Configuration management (FL 5.4) 247

5.5.3 Change management 248

5.6 Risks management (FL 5.5) 248

5.6.1Principles of risk management 249

5.6.2 Project risks and product risks 253

5.6.3 Introduction to risk management 254

5.7 Defect management (FL 5.6) 257

5.7.1 Introduction to defect management 257

5.7.2 Defect identification 258

5.7.3 Actions applied to defects 263

5.7.4 Defect disposition 264

5.8 Synopsis of this chapter 264

5.9 Sample exam questions 265

Chapter 6 Tools support for Testing (FL 6.0) 277

6.1 Types of test tools (FL 6.1) 278

6.1.1 Test tool classification 278

6.1.2 Tools supporting test management 278

6.1.3 Tools supporting requirement management 279

6.1.4 Tools supporting static tests 279

6.1.5 Modeling tools 280

6.1.6 Tools supporting test design and test data creation 280

6.1.7 Tools supporting test execution 281

6.1.8 Tools supporting test environment management 281

6.1.9 Tools supporting test data comparison 282

6.1.10 Tools supporting test coverage measurement 282

6.1.11 Other test supporting tools 283

6.2 Assumptions and limitations of test tools (FL 6.2) 283

6.2.1 Advantages and risks of the tools 284

6.2.2 Specific considerations for some tools 285

6.3 Selecting and introducing tools in an organization (FL 6.3) 289

6.3.1 Main principles 290

6.3.2 Tools selection process 290

6.3.3 Test tool implementation 293

6.3.4 To build or to buy test tools? 295

6.4 Synopsis of this chapter 295

6.5 Sample exam questions 296

Chapter 7 Mock Exam 301

Chapter 8 Templates and Models 315

8.1 Master test plan 315

8.2 Testplan 317

8.2.1 Testplan as per IEEE 829-1998 317

8.2.2 Test plan as per IEEE 829-2008 317

8.3 Test design document 319

8.3.1 Test design specifications as per IEEE 829-1998 319

8.3.2 Test design document as per IEEE 829-2008 319

8.4 Test case 320

8.4.1 Test case document as per IEEE 829-1998 320

8.4.2 Test case document as per IEEE 829-2008 320

8.5 Test procedure 321

8.5.1 Test procedure document as per IEEE 829-1998 321

8.5.2 Test procedure document as per IEEE 829-2008 321

8.6 Test log 322

8.6.1 Test log as per IEEE 829-1998 322

8.6.2 Test log as per IEEE 829-2008 322

8.7 Defect report 322

8.7.1 Defect report as per IEEE 829-1998 322

8.7.2 Defect report as per IEEE 829-2008 323

8.8 Testreport 324

8.8.1 Test report as per IEEE 829-1998 324

8.8.2 Interim test report as per IEEE 829-2008 324

8.8.3 Level test report as per IEEE 829-2008 325

8.8.4 Master test report as per IEEE 829-2008 325

Chapter 9 Answers to the Questions 327

9.1 Answers to the end of chapter questions 327

9.2 Correct answers to the sample paper questions 329

Bibliography 331

Index 335