Certified Tester Advanced Level Syllabus Test Analyst Version 2012 International Software Testing Qualifications Board ISTQB be penrty densourced
Certified Tester Advanced Level Syllabus Test Analyst Version 2012 International Software Testing Qualifications Board Copyright Notice This document may be copied in its entirety, or extracts made, if the source is acknowledged
Certified Tester Advanced Level Syllabus-Test Analyst Qualifications Board Copyright International Software Testing Qualifications Board (hereinafter called ISTQB). Version 2012 Page 2 of 64 19 October 2012
Certified Tester Advanced Level Syllabus - Test Analyst International Software Testing Qualifications Board Version 2012 Page 2 of 64 19 October 2012 © International Software Testing Qualifications Board Copyright © International Software Testing Qualifications Board (hereinafter called ISTQB®). Advanced Level Test Analyst Sub Working Group: Judy McKay (Chair), Mike Smith, Erik Van Veenendaal; 2010-2012
Certified Tester Advanced Level Syllabus-Test Analyst Qualifications Board Revision History Version Date rks ivllabus from EoQ-SG V2007 120cT07 Certified Tester Advanced Level svillabus version 2007 D100626 26JUN10 Incorporaion of changes as accepted in 209,separaion of c pters fo the ser mo D101227 27DEC10 A9cp3&8n2hgne990mgl8netecsetonsthathave D2011 230CT11 Change to split syllabus,re-worked LOs and text changes o match tion of BOs Alpha 2012 09MAR12 October rele from NBs received fron Beta 2012 07APR12 Incorporation of all comments from NBs received from the Beta 2012 Apha release submitted to GA 1 eta201227 UN12 G and Glossary con orated RC201 15AUG12 Release candidate version-final NB edits included GA2012 19CT12 Final edits and cleanup for GA release Version 2012 Page 3 of 64 19 October 2012
Certified Tester Advanced Level Syllabus - Test Analyst International Software Testing Qualifications Board Version 2012 Page 3 of 64 19 October 2012 © International Software Testing Qualifications Board Revision History Version Date Remarks ISEB v1.1 04SEP01 ISEB Practitioner Syllabus ISTQB 1.2E SEP03 ISTQB Advanced Level Syllabus from EOQ-SG V2007 12OCT07 Certified Tester Advanced Level syllabus version 2007 D100626 26JUN10 Incorporation of changes as accepted in 2009, separation of chapters for the separate modules D101227 27DEC10 Acceptance of changes to format and corrections that have no impact on the meaning of the sentences. D2011 23OCT11 Change to split syllabus, re-worked LOs and text changes to match LOs. Addition of BOs. Alpha 2012 09MAR12 Incorporation of all comments from NBs received from October release. Beta 2012 07APR12 Incorporation of all comments from NBs received from the Alpha release. Beta 2012 07APR12 Beta Version submitted to GA Beta 2012 08JUN12 Copy edited version released to NBs Beta 2012 27JUN12 EWG and Glossary comments incorporated RC 2012 15AUG12 Release candidate version - final NB edits included GA 2012 19OCT12 Final edits and cleanup for GA release
Certified Tester ISTQB Advanced Level Syllabus-Test Analyst Qualifications Board Table of Contents Ack ocument 0.20E 0.3 Examinable Learning Objectives 1sting Process-300 mins. 1.2 Testing in the Software Development Lifecycle 1.3 Test Planning.Monitoring and Control.. 1.4 Test Analysis. 20 1.51 esign ical Test Cases. 1.5.2 Creation of Test cases 13 1.6 Test ntauion..... 18 Evaluating Exit Criteria and Reporting 1.9 Test Closure Activities 2 19 2.2 Test Progre ess Monitoring and Control 21 ng 2410 50 2.4.2 Risk Identification. ent 24 Test Techniques-825 mins. 26 Introduction 2 3.2.2 Boundary Value Analysis 29 State Transion T 30 Combinatorial Testing Techniques 33 3.28 User Story Testing 33 32.Doma niques 25 tR. 3.3.1 Using Defect-Based Techniques. 35 .3.21 3 3.4.1 Error Guessing 37 38 Version 2012 Page 4 of 64 19 October 2012
Certified Tester Advanced Level Syllabus - Test Analyst International Software Testing Qualifications Board Version 2012 Page 4 of 64 19 October 2012 © International Software Testing Qualifications Board Table of Contents Revision History....................................................................................................................................... 3 Table of Contents .................................................................................................................................... 4 Acknowledgements ................................................................................................................................. 6 0. Introduction to this Syllabus................................................................................................................. 7 0.1 Purpose of this Document............................................................................................................. 7 0.2 Overview ....................................................................................................................................... 7 0.3 Examinable Learning Objectives .................................................................................................. 7 1. Testing Process - 300 mins. ........................................................................................................... 8 1.1 Introduction ................................................................................................................................... 9 1.2 Testing in the Software Development Lifecycle ............................................................................ 9 1.3 Test Planning, Monitoring and Control........................................................................................ 11 1.3.1 Test Planning....................................................................................................................... 11 1.3.2 Test Monitoring and Control ................................................................................................ 11 1.4 Test Analysis............................................................................................................................... 12 1.5 Test Design ................................................................................................................................. 12 1.5.1 Concrete and Logical Test Cases........................................................................................ 13 1.5.2 Creation of Test Cases ........................................................................................................ 13 1.6 Test Implementation.................................................................................................................... 15 1.7 Test Execution ............................................................................................................................ 16 1.8 Evaluating Exit Criteria and Reporting ........................................................................................ 18 1.9 Test Closure Activities................................................................................................................. 19 2. Test Management: Responsibilities for the Test Analyst - 90 mins............................................. 20 2.1 Introduction ................................................................................................................................. 21 2.2 Test Progress Monitoring and Control ........................................................................................ 21 2.3 Distributed, Outsourced and Insourced Testing.......................................................................... 22 2.4 The Test Analyst’s Tasks in Risk-Based Testing........................................................................ 22 2.4.1 Overview.............................................................................................................................. 22 2.4.2 Risk Identification................................................................................................................. 23 2.4.3 Risk Assessment ................................................................................................................. 23 2.4.4 Risk Mitigation...................................................................................................................... 24 3. Test Techniques - 825 mins. ........................................................................................................ 26 3.1 Introduction ................................................................................................................................. 27 3.2 Specification-Based Techniques................................................................................................. 27 3.2.1 Equivalence Partitioning ...................................................................................................... 27 3.2.2 Boundary Value Analysis..................................................................................................... 28 3.2.3 Decision Tables ................................................................................................................... 29 3.2.4 Cause-Effect Graphing ........................................................................................................ 30 3.2.5 State Transition Testing....................................................................................................... 30 3.2.6 Combinatorial Testing Techniques ...................................................................................... 31 3.2.7 Use Case Testing ................................................................................................................ 33 3.2.8 User Story Testing ............................................................................................................... 33 3.2.9 Domain Analysis .................................................................................................................. 34 3.2.10 Combining Techniques ...................................................................................................... 35 3.3 Defect-Based Techniques........................................................................................................... 35 3.3.1 Using Defect-Based Techniques ......................................................................................... 35 3.3.2 Defect Taxonomies.............................................................................................................. 36 3.4 Experience-Based Techniques ................................................................................................... 37 3.4.1 Error Guessing..................................................................................................................... 37 3.4.2 Checklist-Based Testing ...................................................................................................... 38 3.4.3 Exploratory Testing.............................................................................................................. 38
Certified Tester ISTOB Advanced Level Syllabus-Test Analyst Qualifications Board 39 ware Quality Characteristics-120 mins... h acteristics for Business Domain Testing 4.2.1 Accuracy Testing 43 4经3ab 44 44 49 Management-120 mins... e.1 nroducho 53 Test Tools-45 mins. 56 5 7.2.1 Test De Execution Tools ferences 8.1 Standards. 6, QB Documents 84 Other References. 62 9 Index.. 6 Version 2012 Page 5 of 64 19 October 2012
Certified Tester Advanced Level Syllabus - Test Analyst International Software Testing Qualifications Board Version 2012 Page 5 of 64 19 October 2012 © International Software Testing Qualifications Board 3.4.4 Applying the Best Technique ............................................................................................... 39 4. Testing Software Quality Characteristics - 120 mins.................................................................... 41 4.1 Introduction ................................................................................................................................. 42 4.2 Quality Characteristics for Business Domain Testing................................................................. 43 4.2.1 Accuracy Testing ................................................................................................................. 43 4.2.2 Suitability Testing................................................................................................................. 44 4.2.3 Interoperability Testing ........................................................................................................ 44 4.2.4 Usability Testing .................................................................................................................. 44 4.2.5 Accessibility Testing ............................................................................................................ 47 5. Reviews - 165 mins....................................................................................................................... 48 5.1 Introduction ................................................................................................................................. 49 5.2 Using Checklists in Reviews ....................................................................................................... 49 6. Defect Management – 120 mins................................................................................................... 52 6.1 Introduction ................................................................................................................................. 53 6.2 When Can a Defect be Detected? .............................................................................................. 53 6.3 Defect Report Fields ................................................................................................................... 53 6.4 Defect Classification.................................................................................................................... 54 6.5 Root Cause Analysis................................................................................................................... 55 7. Test Tools - 45 mins. .................................................................................................................... 56 7.1 Introduction.................................................................................................................................. 57 7.2 Test Tools and Automation ......................................................................................................... 57 7.2.1 Test Design Tools................................................................................................................ 57 7.2.2 Test Data Preparation Tools................................................................................................ 57 7.2.3 Automated Test Execution Tools......................................................................................... 57 8. References.................................................................................................................................... 61 8.1 Standards .................................................................................................................................... 61 8.2 ISTQB Documents ...................................................................................................................... 61 8.3 Books .......................................................................................................................................... 61 8.4 Other References........................................................................................................................ 62 9. Index ............................................................................................................................................. 63
Certified Tester ISTQB Internationa Advanced Level Syllabus-Test Analyst Qualifications Board Acknowledgements e Erik van Veenendaal. The core team thanks the review team and the National Boards for their suggestions and input. engatpahscelbenaseompeateAnceLoewotngGouphadthe Graham Bath,Rex Black.Maria Clara Choucair,Debra F Jan Sabak,Han s Schaefer,Mike Smith(Chair).Geoff Thomps on Erik van Veenendaal,Tsuyoshi Yumoto. The following persons participated in the reviewing,commenting and balloting of this syllabus Graham Bath,Ame Becher.Rex Bla Stefan Monacs eto Mueller Tho as Muelle P98Ca-suarReam8mSmaeaaSeosoganaaeomTbey,r Veymouth,( This document was formally released by the General Assembly of the ISTQBon October 19th,2012 Version 2012 Page 6 of 64 19October 2012
Certified Tester Advanced Level Syllabus - Test Analyst International Software Testing Qualifications Board Version 2012 Page 6 of 64 19 October 2012 © International Software Testing Qualifications Board Acknowledgements This document was produced by a core team from the International Software Testing Qualifications Board Advanced Level Sub Working Group - Advanced Test Analyst: Judy McKay (Chair), Mike Smith, Erik van Veenendaal. The core team thanks the review team and the National Boards for their suggestions and input. At the time the Advanced Level Syllabus was completed the Advanced Level Working Group had the following membership (alphabetical order): Graham Bath, Rex Black, Maria Clara Choucair, Debra Friedenberg, Bernard Homès (Vice Chair), Paul Jorgensen, Judy McKay, Jamie Mitchell, Thomas Mueller, Klaus Olsen, Kenji Onishi, Meile Posthuma, Eric Riou du Cosquer, Jan Sabak, Hans Schaefer, Mike Smith (Chair), Geoff Thompson, Erik van Veenendaal, Tsuyoshi Yumoto. The following persons participated in the reviewing, commenting and balloting of this syllabus: Graham Bath, Arne Becher, Rex Black, Piet de Roo, Frans Dijkman, Mats Grindal, Kobi Halperin, Bernard Homès, Maria Jönsson, Junfei Ma, Eli Margolin, Rik Marselis, Don Mills, Gary Mogyorodi, Stefan Mohacsi, Reto Mueller, Thomas Mueller, Ingvar Nordstrom, Tal Pe'er, Raluca Madalina Popescu, Stuart Reid, Jan Sabak, Hans Schaefer, Marco Sogliani, Yaron Tsubery, Hans Weiberg, Paul Weymouth, Chris van Bael, Jurian van der Laar, Stephanie van Dijk, Erik van Veenendaal, Wenqiang Zheng, Debi Zylbermann. This document was formally released by the General Assembly of the ISTQB® on October 19th, 2012
Certified Tester ISTOB n Advanced Level Syllabus-Test Analyst Qualifications Board 0.Introduction to this Syllabus 0.1 Purpose of this Document Thissllabus forms thebasisnresingiiction at the Advanced 1 To Natignal Boards.to translate into their oc ge and to accredit training providers National Boards may adapt the syllabus to their particular language needs and modify the 2 al pu ation questions in their local language adapted to the learning objectives for each syllabus. To traini ind). 5. elSgCe0meemotniesosets9abustroterpupcaspoweuheysekad 0.2 Overview The Advanced Level is comprised of three separate syllabi: Test Mar ger est Analyst The Advanced Level Overview document [ISTQB_AL_OVIEW]includes the following information: mes ch syllabus the syllabi Description of cognitive levels(K-levels) ·Appendices 0.3 Examinable Learning Objectives The Learning Objectives support the Business Outcomes and are used to create the examination for acnngena py cep:The e shown at the peai ning of the perti Version 2012 Page 7 of 64 19 October 2012
Certified Tester Advanced Level Syllabus - Test Analyst International Software Testing Qualifications Board Version 2012 Page 7 of 64 19 October 2012 © International Software Testing Qualifications Board 0. Introduction to this Syllabus 0.1 Purpose of this Document This syllabus forms the basis for the International Software Testing Qualification at the Advanced Level for the Test Analyst. The ISTQB® provides this syllabus as follows: 1. To National Boards, to translate into their local language and to accredit training providers. National Boards may adapt the syllabus to their particular language needs and modify the references to adapt to their local publications. 2. To Exam Boards, to derive examination questions in their local language adapted to the learning objectives for each syllabus. 3. To training providers, to produce courseware and determine appropriate teaching methods. 4. To certification candidates, to prepare for the exam (as part of a training course or independently). 5. To the international software and systems engineering community, to advance the profession of software and systems testing, and as a basis for books and articles. The ISTQB® may allow other entities to use this syllabus for other purposes, provided they seek and obtain prior written permission. 0.2 Overview The Advanced Level is comprised of three separate syllabi: Test Manager Test Analyst Technical Test Analyst The Advanced Level Overview document [ISTQB_AL_OVIEW] includes the following information: Business Outcomes for each syllabus Summary for each syllabus Relationships among the syllabi Description of cognitive levels (K-levels) Appendices 0.3 Examinable Learning Objectives The Learning Objectives support the Business Outcomes and are used to create the examination for achieving the Advanced Test Analyst Certification. In general all parts of this syllabus are examinable at a K1 level. That is, the candidate will recognize, remember and recall a term or concept. The learning objectives at K2, K3 and K4 levels are shown at the beginning of the pertinent chapter
Certified Tester ISTQB Internationa Advanced Level Syllabus-Test Analyst Qualifications Board 1.Testing Process-300 mins. Keywords assgsemtoehpteablep2gestcase,owueveteteasetescontol case exit criteria Learning Objectives for Testing Process 1.2 Testina in the Software Development Lifecvcle TA-1.21 (K2)Explain how and why the timing and level of involvement for the Test Analyst varies when working with different lifecycle models 1,TestweatomgPa nd Control performed by the Test Analyst in support of planning and controlling the testing and life 1.Test Design TA152 te gheoeemeriensthappoe use for low-level (concrete)and high-level (logical)test cases 1e,Testmee entation ffect the 1.7 Test Execution TA-1.7.1 (K3)For a given scenario.determine the steps and considerations that should be taken when executing tests 18,9t25aCmeaan1Re2et3 Explain why accurate test case execution status information is important 1.9 Test Closure Activities TA-1.9.1 (K2)Provide examples of work products that should be delivered by the Test Analyst during test closure activities Version 2012 Page 8 of 64 19 October 2012
Certified Tester Advanced Level Syllabus - Test Analyst International Software Testing Qualifications Board Version 2012 Page 8 of 64 19 October 2012 © International Software Testing Qualifications Board 1. Testing Process - 300 mins. Keywords concrete test case, exit criteria, high-level test case, logical test case, low-level test case, test control, test design, test execution, test implementation, test planning Learning Objectives for Testing Process 1.2 Testing in the Software Development Lifecycle TA-1.2.1 (K2) Explain how and why the timing and level of involvement for the Test Analyst varies when working with different lifecycle models 1.3 Test Monitoring, Planning and Control TA-1.3.1 (K2) Summarize the activities performed by the Test Analyst in support of planning and controlling the testing 1.4 Test Analysis TA-1.4.1 (K4) Analyze a given scenario, including a project description and lifecycle model, to determine appropriate tasks for the Test Analyst during the analysis and design phases 1.5 Test Design TA-1.5.1 (K2) Explain why test conditions should be understood by the stakeholders TA-1.5.2 (K4) Analyze a project scenario to determine the most appropriate use for low-level (concrete) and high-level (logical) test cases 1.6 Test Implementation TA-1.6.1 (K2) Describe the typical exit criteria for test analysis and test design and explain how meeting those criteria affect the test implementation effort 1.7 Test Execution TA-1.7.1 (K3) For a given scenario, determine the steps and considerations that should be taken when executing tests 1.8 Evaluating Exit Criteria and Reporting TA-1.8.1 (K2) Explain why accurate test case execution status information is important 1.9 Test Closure Activities TA-1.9.1 (K2) Provide examples of work products that should be delivered by the Test Analyst during test closure activities
Certified Tester Advanced Level Syllabus-Test Analyst Qualifications Board 1.1 Introduction oundation Level syllabus,the fundamental test process was described as including monitoring and control Analysis and desian Implementation and execution ria and reporting At the Advanced Level.some of these activities are considered separately in order to provide considered as follows: Planning,monitoring and control Analysis Evaluating exit criteria and reporting Test closure activities These activities can be implemented sequentially or some can be implemented in parallel.e.g..design oe dung the na den.mlemen e parallel w impl menta est A hile it t to under P activities of the testing project. Advanced testers face a number of challenges when introducing the different testing aspects described in this syllabus into the context of the e the to test 1.2 Testing in the Software Development Lifecycle The long life ed an es and the amount of involvement,time required.information avai lable an expectations can be quite varied as pnis of the ment-requirements reviews Project management-schedule input Configuration and change management-build verification testing.version control . efect e(i.e.,time from defect discovery to defect resolution) Technical support-accurate documentation for workarounds duction of tech documentation databas design specifications)input to these Version 2012 Page 9 of 64 19 October 2012 tional Software Testing Board
Certified Tester Advanced Level Syllabus - Test Analyst International Software Testing Qualifications Board Version 2012 Page 9 of 64 19 October 2012 © International Software Testing Qualifications Board 1.1 Introduction In the ISTQB® Foundation Level syllabus, the fundamental test process was described as including the following activities: Planning, monitoring and control Analysis and design Implementation and execution Evaluating exit criteria and reporting Test closure activities At the Advanced Level, some of these activities are considered separately in order to provide additional refinement and optimization of the processes, to better fit the software development lifecycle, and to facilitate effective test monitoring and control. The activities at this level are considered as follows: Planning, monitoring and control Analysis Design Implementation Execution Evaluating exit criteria and reporting Test closure activities These activities can be implemented sequentially or some can be implemented in parallel, e.g., design could be performed in parallel with implementation (e.g., exploratory testing). Determining the right tests and test cases, designing them and executing them are the primary areas of concentration for the Test Analyst. While it is important to understand the other steps in the test process, the majority of the Test Analyst’s work usually is done during the analysis, design, implementation and execution activities of the testing project. Advanced testers face a number of challenges when introducing the different testing aspects described in this syllabus into the context of their own organizations, teams and tasks. It is important to consider the different software development lifecycles as well as the type of system being tested as these factors can influence the approach to testing. 1.2 Testing in the Software Development Lifecycle The long-term lifecycle approach to testing should be considered and defined as part of the testing strategy. The moment of involvement for the Test Analyst is different for the various lifecycles and the amount of involvement, time required, information available and expectations can be quite varied as well. Because the testing processes do not occur in isolation, the Test Analyst must be aware of the points where information may be supplied to the other related organizational areas such as: Requirements engineering and management - requirements reviews Project management - schedule input Configuration and change management - build verification testing, version control Software development - anticipating what is coming, and when Software maintenance - defect management, turnaround time (i.e., time from defect discovery to defect resolution) Technical support - accurate documentation for workarounds Production of technical documentation (e.g., database design specifications) - input to these documents as well as technical review of the documents
Certified Tester ISTQB Advanced Level Syllabus-Test Analyst Qualifications Board ned with the enhosearsearmpe e fundamentai test process applied to the system test level could align as follows: System test plann with project planning.and test control continues until System test analysis and design occur concu ently with requirements specification.system and architectural (high-level)design specification,and component(low-level)design 10n h work on system test implementation activities stretching often until just days before are complete.System test execution continues until the system test exit criteria are met. a with and 0 etnea8hoe approach System test closure activities ater the system test exit aremet and system test are finished. r acceptance rermeatalmodesmayno may exclude iteration.Analysis and design.implementation and execution.and evaluation and reporting may be nereas planning is d lone at the eginning of the project and the cl osure uch closer cha asily withinh Agile is a "igh veight"process.the ere is less co nsive test docur nen aton in favor of having a more rapid suc s aily up quick. 0-15 minutes A9aepeetoaghe th developers as the do their initial architecture and de ign work. Reviews n nav no be for usually dedicated to single projects and are fully involved in all aspects of the project. aeraheoRemeneaoeseaeetemchenelaereopneherRoteeaneecaionohp3S edded iterative). terac s the and depl t for the Test A ctive role and vard that rather than depending on the Version 2012 Page 10 of 64 19 October 2012
Certified Tester Advanced Level Syllabus - Test Analyst International Software Testing Qualifications Board Version 2012 Page 10 of 64 19 October 2012 © International Software Testing Qualifications Board Testing activities must be aligned with the chosen software development lifecycle model whose nature may be sequential, iterative, or incremental. For example, in the sequential V-model, the ISTQB® fundamental test process applied to the system test level could align as follows: System test planning occurs concurrently with project planning, and test control continues until system test execution and closure are complete. System test analysis and design occur concurrently with requirements specification, system and architectural (high-level) design specification, and component (low-level) design specification. System test environment (e.g., test beds, test rig) implementation might start during system design, though the bulk of it typically would occur concurrently with coding and component test, with work on system test implementation activities stretching often until just days before the start of system test execution. System test execution begins when the system test entry criteria are all met (or waived), which typically means that at least component testing and often also component integration testing are complete. System test execution continues until the system test exit criteria are met. Evaluation of system test exit criteria and reporting of system test results occur throughout system test execution, generally with greater frequency and urgency as project deadlines approach. System test closure activities occur after the system test exit criteria are met and system test execution is declared complete, though they can sometimes be delayed until after acceptance testing is over and all project activities are finished. Iterative and incremental models may not follow the same order of tasks and may exclude some tasks. For example, an iterative model may utilize a reduced set of the standard test processes for each iteration. Analysis and design, implementation and execution, and evaluation and reporting may be conducted for each iteration, whereas planning is done at the beginning of the project and the closure reporting is done at the end. In an Agile project, it is common to use a less formalized process and a much closer working relationship that allows changes to occur more easily within the project. Because Agile is a “light weight” process, there is less comprehensive test documentation in favor of having a more rapid method of communication such as daily “stand up” meetings (called “stand up” because they are very quick, usually 10-15 minutes, so no one needs to sit down and everyone stays engaged). Agile projects, out of all the lifecycle models, require the earliest involvement from the Test Analyst. The Test Analyst should expect to be involved from the initiation of the project, working with the developers as they do their initial architecture and design work. Reviews may not be formalized but are continuous as the software evolves. Involvement is expected to be throughout the project and the Test Analyst should be available to the team. Because of this immersion, members of Agile teams are usually dedicated to single projects and are fully involved in all aspects of the project. Iterative/incremental models range from the Agile approach, where there is an expectation for change as the software evolves, to iterative/incremental development models that exist within a V-model (sometimes called embedded iterative). In the case with an embedded iterative model, the Test Analyst should expect to be involved in the standard planning and design aspects, but would then move to a more interactive role as the software is developed, tested, changed and deployed. Whatever the software development lifecycle being used, it is important for the Test Analyst to understand the expectations for involvement as well as the timing of that involvement. There are many hybrid models in use, such as the iterative within a V-model noted above. The Test Analyst often must determine the most effective role and work toward that rather than depending on the definition of a set model to indicate the proper moment of involvement