暫定版。後ほどアップデートしますが勉強会に先立ってまず公開します。USTREAMの方はスライド見にくいと思いますのでこちらをご活用いただければと。

1. ISO/IEC/IEEE 29119 勉強会
第3回 - Part 3 テストドキュメント
YAMASAKI Takashi 2014年1月9日

2. 本日のゴール

3. Organizational Test Process
Test Management Processes
Dynamic Test Processes
Test Policy
Organizational
Test Strategy
Test Plan
Test Status
Report
Test Completion
Report
Test Environment
Readiness
Report
Test Data
Requirements
Test
Environment
Requirements
Test Data
Readiness
Report
Incident
Report
Test Procedure
Specification
Test Case
Specification
Test Design
Specification
Actual
Result
Test
Result
Test Execution
Log

4. 種類

5. Figure 1 — The hierarchy of test documentation from ISO/IEC/IEEE 29119-2:2013
Dynamic Test ProcessesDynamic Test Processes Dynamic Test Processes
Test Management Processes
Test Management Processes
Organizational Test Process
Test Policy
Organizational
Test Strategy
Organizational
Test Strategy
Test Plan
(Project)
Test Plan
(Project)
Test Plan
(Project)
Test Plan
(Sub-process)
Test Plan
(Sub-process)
Test Plan
(Sub-process)
Test
Specification
Test Environment
Requirement
Test Data
Requirement
Test Environment
Readiness Report
Test Data
Readiness Report
Perform
Dynamic Test
Test Execution
Documentation
Incident Report
Test Status
Report
Test Completion
Report (Sub-proc.)
Test Completion
Report (Project)

6. TEST
PLAN
TEST
DESIGN
SPEC
TEST
CASE
SPEC
TEST
PROC
SPEC
ITEM
DOC
ITEM
DOC
ITEM
DOC
PROJECT
DOC
TEST
LOG
TEST
LOG
TEST
INCIDENT
REPORT
TEST
SUMMARY
REPORT
TEST
ITEM
TEST
EXECUTION
TEST ITEM
TRANSMITTAL
REPORT
TEST
INCIDENT
REPORT
TEST
DESIGN
SPEC
Figure. “Relationship of test documents to testing process” from IEEE Std 829-1998 IEEE Standard for Software Test Documentation).
IEEE Std
829 – 1998TEST
DESIGN
SPEC
Document specified by this
standard
Document NOT specified by
this standard
Test Item (NOT specified by
the standard)
Process NOT specified by this
standard

8. Test Management Processes
Dynamic Test Processes
Test
Planning
Process
Test
Monitoring &
Control
Process
Test
Completion
Process
Test Design &
Implementation
Process
Test
Environment
Set-up &
Maintenance
Process
Test
Incident
Reporting
Process
Test
Execution
Process
Organizational
Test
Process

9. Organizational
Test Process
Develop
Organizational
Test
Specification
(OT1)
Monitor and
Control Use of
Test
Specification
(OT2)
Update
Organizational
Test
Specification
(OT3)
Organization
Test
Specification
Controlled
Organizational
Test Specification
Updated
Organization
Test Specification
Figure 4 – Organizational Tet Process, from ISO/IEC/IEEE 29119-2 Test process, P-13
Inputs to activities in this process
may include:
• Views of primary stakeholders;
• Knowledge of the current test
practices within the
organization;
• Organization’s Mission
Statement;
• IT Policy;
• IT Project Management Policy;
• Quality Policy;
• Organizational Test Policy;
• Organizational Test Strategy;
• Feedback on Tet Specification;
• Typical test plans from the
organization; and
• Industry and/or government
standards.

10. 組織的なテストプロセス
組織的なテスト
仕様を開発する
(OT1)
(組織的な)
テスト仕様の
効用をモニターし
コントロールする
(OT2)
組織的なテスト
仕様を更新する
(OT3)
組織的な
テスト仕様
コントロールされた
組織的なテスト
仕様
更新された
組織的な
テスト仕様

11. Test Management Processes
Dynamic Test Processes
Test Plan
Test Status
Report
Test Completion
Report
Test Environment
Readiness
Report
Test Data
Requirements
Test
Environment
Requirements
Test Data
Readiness
Report
Incident
Report
Test Procedure
Specification
Test Case
Specification
Test Design
Specification
Actual
Result
Test
Result
Test Execution
Log
Organizational Test Process
Test Policy
Organizational
Test Strategy

12. 組織に関するテストについての情報を
提 供 す る ド キ ュ メ ン ト 、 た と え ば
プロジェクトに依らない情報など
• 組織的なテスト仕様の例
• 組織的なテストポリシー
(organizational test policies)
• 組織的なテスト戦略
(organizational test strategies)

13. • 経営層レベルのドキュメント
• 組織におけるテストの目的、ゴール、
全体的なスコープを記述
※ Part 3では”Organizational”がなく単に”Test Policy”となっている。
Organizationalのありなしで、意味が大きく変わる単語も多いのでちょっ
と違和感がある。

14. Document specification information
Introduction
Test policy statement
Objectives of testing (テストの目的)
Test process (テストプロセス)
Test organization structure (テスト組織構造)
Tester training (テスターのトレーニング)
Tester ethics (テスターの行動規範)
Standards (標準)
Other relevant policies (その他の関連のあるポリシー)
Measuring the value of testing（テストの価値の測定）
Test asset archiving and reuse（テスト資産の保管と再利用）
Test process improvement（テストプロセス改善）

15. • 組織で実行するすべてのプロジェクトにおいて
実行されるテストのための包括的(generic)な
要求を表現したドキュメントで、どのように
テストを実行するかの詳細を提供する
• 注１： 組織的なテストポリシーによって調整される
• 注２：著しく異なるプロジェクトコンテキストをカバー
するために、ひとつの組織がひとつ以上の組織的な
テスト戦略を持つことができる

16. Document specific information
Introduction
Project-wide organizational test strategy statements
(プロジェクト横断的な組織的テスト戦略ステートメント…って長いよ！）
Generic risk management（包括的なリスクマネジメント）
Test selection and prioritization (テストの選別と優先順位付け)
Test documentation and reporting (テストの文書化と報告)
Test automation and tools (テストの自動化とツール)
Configuration management of test work products (テスト成果物の構成管理)
Incident management（インシデント管理）
Test subprocesses（テストサブプロセス）
Test sub-process-specific organizational test strategy statements
(テストサブプロセス固有の組織的なテスト戦略ステートメント)
Entry and exit criteria（開始および終了基準）
Test complication criteria （テスト完了基準）
Test documentation and reporting （テストの文書化と報告）
Degree of independence （独立性の度合い）
Test design techniques （テスト設計技法）
Test environment （テスト環境）
Metrics to be collected （収集されるメトリクス）
Retesting and regression testing （再テストと回帰テスト）

18. Figure 5 – Example test management process relationship, from ISO/IEC/IEEE 29119-2 Test processes P-15
Organizational
Test Process
Test Management Processes
Test
Planning
Test
Monitoring
& Control
Test
Completion
Test Plan Updates
Test Plan
Test
Results
Test
Completion
Report
Test Management
Processes
Test Management
Processes
Dynamic Test
Processes
Dynamic Test
Processes
Dynamic Test
Processes
Test Policy &
Organizational Test Strategy
Test Plan,
Control
Directives
Test Plan,
Test Status Reports,
Test Completion Report,
Test Measures
Test Plan,
Control
Directives
Test
Measures
Test Plan,
Control
Directives
Test
Measures
Feedback on
Test Policy &
Organizational Test Strategy

19. Organizational
Test
Process
Test Management Processes
Dynamic Test Processes
Test
Monitoring &
Control
Process
Test
Completion
Process
Test Design &
Implementation
Process
Test
Environment
Set-up &
Maintenance
Process
Test
Incident
Reporting
Process
Test
Execution
Process
Test
Planning
Process

20. Figure 6 – Test Planning Process, from ISO/IEC/IEEE 29119-2 Test processes P-16
Test Planning
Process
Understand
Context
(TP1)
Organize
Test Plan
Development
(TP2)
Design Test
Strategy
(TP5)
Identify Risk
Mitigation
Approaches
(TP4)
Identify &
Analyse Risks
(TP3)
Communicate
Test Plan And
Make Available
(TP9)
Gain
Consensus on
Test Plan
(TP8)
Determine
Staffing and
Scheduling
(TP6)
Record
Test Plan
(TP7)
Scope
Test Plan
Development Schedule
Analyzed Risks
Mitigation
Approaches
Test
Strategy
Schedule,
Staffing Profile
Draft Test Plan
Approved
Test Plan
Test Plan
The process is shown as purely sequential, but in practice it may
be carried out iteratively, with some activities being revisited.
See text for detail.
Inputs to activities in this process may
include:
• Organizational Test Policy;
• Organizational Test Strategy;
• Regulatory Standards;
• Project Test Plan (if planning
testing for a specific phase or type
within a project);
• Incident reports;
• Project Management Plan;
• Applicable product documentation
(e.g. System Requirements, Test
Item Specification);
• Software Development Plan;
• Project and Product Risks; and
• Test Plan Updates.

21. Organizational
Test
Process
Test Management Processes
Dynamic Test Processes
Figure 2 – The multi-layer model showing all test processes, from ISO/IEC/IEEE 29119-2:2013 Test processes, P-11
Test
Planning
Process
Test
Completion
Process
Test Design &
Implementation
Process
Test
Environment
Set-up &
Maintenance
Process
Test
Incident
Reporting
Process
Test
Execution
Process
Test
Monitoring &
Control
Process

22. Figure 7 – Test monitoring and control process, from ISO/IEC/IEEE 29119-2 Test processes P-22
Test Monitoring &
Control Process
Set-Up
(TMC1)
<<instantiated>>
Inputs to activities in this process may
include:
• Test Plan(s);
• Applicable product documentation,
e.g. System Requirement, Contract,
etc.;
• Organizational Test Policy;
• Organizational Test Strategy
• Control Directive (from a higher
level Test Monitoring and Control
Process); and
• Measures (from the Test Process
being managed).
Monitor
(TMC2)
Report
(TMC4)
... Test Process ...
Test
Management
Processes
Dynamic Test
Processes
Control
(TMC3)
<<instantiated>>
Test
Measures
Control
Directives
Test Status Report
Test
Measures
Test
Plan
Test
Progress
Info
Test
Control Info [Testing Incomplete]
[Testing
Complete]
The process is shown as purely sequential, but
in practice it may be carried out iteratively,
with some activities being revisited..
See text fir details
Test Progress Info

23. Organizational
Test
Process
Test Management Processes
Dynamic Test Processes
Figure 2 – The multi-layer model showing all test processes, from ISO/IEC/IEEE 29119-2:2013 Test processes, P-11
Test
Planning
Process
Test
Monitoring &
Control
Process
Test Design &
Implementation
Process
Test
Environment
Set-up &
Maintenance
Process
Test
Incident
Reporting
Process
Test
Execution
Process
Test
Completion
Process

24. Figure 8 – Test Completion Process , from ISO/IEC/IEEE 29119-2 Test processes P-25
Test Completion
Process
Inputs to activities in this process may
include:
• Project Test Plan;
• Phase Test Plans;
• Incident Reports;
• Project Test Status Reports;
• Phase/Type Test Completion Reports; and
• Organizational Test Strategy (if relevant)
The process is shown as purely sequential, but in
practice it may be carried out iteratively, with
some activities being revisited..
See text fir details
Archive
Test Assets
(TC1)
Archived Test
Assets
Clean Up
Test
Environment
(TC2) Identify
Lessons
Learned
(TC3) Report
Test
Completion
(TC4)
Available Test
Environment
Lessons
Learned
Test
Completion
Report

25. Dynamic Test Processes
Test Environment
Readiness
Report
Test Data
Requirements
Test
Environment
Requirements
Test Data
Readiness
Report
Incident
Report
Test Procedure
Specification
Test Case
Specification
Test Design
Specification
Actual
Result
Test
Result
Test Execution
Log
Organizational Test Process
Test Policy
Organizational
Test Strategy
Test Management Processes
Test Plan
Test Status
Report
Test Completion
Report

26. Document specific information
Introduction
Context of the testing（テストのコンテキスト）
Project(s).test sub-process(es) (プロジェクト、テストサブプロセス）
Test item(s) (テストアイテム)
Test scope (テストスコープ)
Assumptions and constraints (前提と制約)
Stakeholders (利害関係者)
Testing communication(テストのコミュニケーション)
Risk register（リスク登録簿）
Product risks（プロダクトリスク）
Project risks（プロジェクトリスク）
Test strategy（テスト戦略）
Test sub-processes（テストサブプロセス）
Test deliverables（テスト成果物）
Test design techniques（テスト設計技法）
Test completion criteria（テスト完了基準）
Metrics to be collected（収集されるメトリクス）
Test data requirements（テストデータ要求）
Test environment requirements（テスト環境要求）
Retesting and regression testing（再テストと回帰テスト）
Suspension and resumption criteria（中断と再開基準）
Deviation from the Organizational Test Strategy（組織的なテスト戦略からの逸脱）
Testing activities and estimates（テスト活動と見積もり）
Staffing（人材の配置）
Roles, activities, and responsibilities（役割、活動、責任）
Hiring needs（雇用の必要性）
Training needs（トレーニングの必要性）
Schedule（スケジュール）

27. Document specific information
Introduction
Test status（テストステータス）
Reporting period（報告時期）
Progress against Test Plan（テストプランと比較した進捗）
Factors blocking progress（進捗を妨げる要因）
Test measures（テストメジャー）
New and changes risks（新規と変更のあったリスク）
Planned testing（計画されたテスト）

28. Document specific information
Introduction
Testing performed（実施したテスト）
Summary of testing performed（実施したテストの概要）
Deviations from planned testing（計画したテストからの逸脱）
Test completion evaluation（テスト完了評価）
Factors that blocked progress（進捗を妨げた要素）
Test measures（テストメジャー）
Residual risks（残存リスク）
Test deliverables（テスト成果物）
Reusable test assets（再利用可能なテスト資産）
Lesson learned（きづき・教訓）

30. Test Design &
Implementation
Test
Environment
Set-up &
Maintenance
Test
Incident
Reporting
Test
Execution
Test
Environment
Requirements
Test Specification
Test Environment
Readiness Report
Incident
Report
Test
Results
[Issue Noticed OR
Retest Result]
[No Issues Noticed]
Test Management Processes
Figure 9 – Dynamic test processes, from ISO/IEC/IEEE 29119-2 Test processes P-28
Test Plan Test
Measures
Control
Directives

31. Organizational
Test
Process
Test Management Processes
Dynamic Test Processes
Figure 2 – The multi-layer model showing all test processes, from ISO/IEC/IEEE 29119-2:2013 Test processes, P-11
Test
Planning
Process
Test
Monitoring &
Control
Process
Test
Environment
Set-up &
Maintenance
Process
Test
Incident
Reporting
Process
Test
Execution
Process
Test
Completion
Process
Test Design &
Implementation
Process

32. Test Design &
Implementation Process
Figure 10 – Tet Design and Implementation Process , from ISO/IEC/IEEE 29119-2 Test processes P-30
Identify
Feature
Sets
(TD1)
Derive
Test
Conditions
(TD2) Drive
Test
Coverage
Items
(TD3) Derive
Test
Cases
(TD4)
Assemble
Test
Sets
(TD5)
Drive
Test
Procedures
（TD6)
Test Design
Specification
Test Case
Specification
Test
Procedure
Specification
Feature
Sets
Test Conditions
Test
Coverage
Items
Test Cases
Test
Sets
Test Procedures
& Test
Scripts
Inputs to activities in this
process may include:
• Test Basis;
• Test plan;
• Test strategy;
• Test items; and
• Test design techniques.
The process is shown as purely sequential, but in
practice it may be carried out iteratively, with
some activities being revisited..
See text fir details

33. Organizational
Test
Process
Test Management Processes
Dynamic Test Processes
Figure 2 – The multi-layer model showing all test processes, from ISO/IEC/IEEE 29119-2:2013 Test processes, P-11
Test
Planning
Process
Test
Monitoring &
Control
Process
Test Design &
Implementation
Process
Test
Incident
Reporting
Process
Test
Execution
Process
Test
Completion
Process
Test
Environment
Set-up &
Maintenance
Process

34. Test Environment Set-Up
& Maintenance Process
Figure 11 – Test Environment Set-Up & Maintenance process , from ISO/IEC/IEEE 29119-2 Test processes P-34
Establish
Test
Environment
(ES1)
Maintain
Test
Environment
(ES2)
Test
Environment
Readiness
Report
Inputs to activities in this process may
include:
• Test Plan(s);
• Test Environment Requirements;
• Intended/operational environment
• Test basis
• Test procedure(s); and
• Test results (where available).
Test
Environment
Update(s)

35. Organizational
Test
Process
Test Management Processes
Dynamic Test Processes
Figure 2 – The multi-layer model showing all test processes, from ISO/IEC/IEEE 29119-2:2013 Test processes, P-11
Test
Planning
Process
Test
Monitoring &
Control
Process
Test Design &
Implementation
Process
Test
Environment
Set-up &
Maintenance
Process
Test
Incident
Reporting
Process
Test
Completion
Process
Test
Execution
Process

36. Figure 12 – Test Execution process , from ISO/IEC/IEEE 29119-2 Test processes P-36
Test Execution
Process
Execute
Test
Procedure(s)
(TE1)
Record
Test
Execution
(TE3)
Test
Outcome
Inputs to activities in this process may include:
• Test Plan(s);
• Test Procedure(s);
• Test item;
• Test basis;
• Test environment readiness report (where available); and
• Test environment update(s) (where available.
Test
Execution
Log
Compare
Test
Result
(TE2)
Test
Results
The process is shown as purely sequential, but in
practice it may be carried out iteratively, with
some activities being revisited..
See text fir details

37. Organizational
Test
Process
Test Management Processes
Dynamic Test Processes
Figure 2 – The multi-layer model showing all test processes, from ISO/IEC/IEEE 29119-2:2013 Test processes, P-11
Test
Planning
Process
Test
Monitoring &
Control
Process
Test Design &
Implementation
Process
Test
Environment
Set-up &
Maintenance
Process
Test
Execution
Process
Test
Completion
Process
Test
Incident
Reporting
Process

38. Test Incident
Reporting Process
Figure 13 – Test Incident Reporting process , from ISO/IEC/IEEE 29119-2 Test processes P-38
Analyze
Test
Result
(IR1)
Create/
Update
Incident
Report
(IR2)
Incident
Report
Updates
Inputs to activities in this process may
include:
• Test results;
• Test procedures(s)
• Test Case(s)
• Test item(s)
• Test basis; and
• Test execution logs (where available).
Incident
Report

39. Organizational Test Process
Test Policy
Organizational
Test Strategy
Test Management Processes
Test Plan
Test Status
Report
Test Completion
Report
Dynamic Test Processes
Test Environment
Readiness
Report
Test Data
Requirements
Test
Environment
Requirements
Test Data
Readiness
Report
Incident
Report
Test Procedure
Specification
Test Case
Specification
Test Design
Specification
Actual
Result
Test
Result
Test Execution
Log

40. Document specific information
Introduction
Notation convention(s) (表記法)
Feature sets（フィーチャーセット）
Overview（概要）
Unque identifier（UID）
Objective（目的）
Priority（優先度）
Specific strategy（特定の戦略）
Traceability（トレーサビリティ）
Test conditions（テスト条件）
Overview（概要）
Uniqque identifier（UID）
Description（説明）
Priority（優先度）
Traceability（トレーサビリティ）

41. Test Case Specification（テストケース仕様）
Document specific information
Introduction
Notation convention(s)（表記法）
Test coverage items（テストカバレッジアイテム）
Overview（概要）
Unique identifier（UID）
Priority（優先度）
Traceability（トレーサビリティ）
Test cases（テストケース）
Objective（概要）
Unique identifer（UID)
Traceability（トレーサビリティ）
Inputs（入力）
Expected results（期待結果）
Actual results and test result（実際の結果とテスト結果）

42. Document specific information
Introduction
Notation convention(s)（表記法）
Test sets（テストセット）
Overview（概要）
Unique identifier（UID)
Objective（目的）
Priority（優先度）
Contents (Traceability) （内容(トレーサビリティ)）
Test procedures（テスト手順）
Overview（概要）
Unique identifier（UID）
Objective（目的）
Priority（優先度）
Start up（開始）
Test cases to be executed (Traceability)（実行されるテストケース）
Relationship to other procedures（その他の手順との関連性）
Stop and wrap up（停止と要約）

43. Document specific information
Introduction
Detailed test data requirements
（詳細化したテストデータ要求）
Overview（概要）
Unique identifier（UID）
Description（説明）
Responsibility（責任）
Period needed（必要な期間）
Resetting needs（リセットの必要性）
Archiving or disposal（保管または破棄）

44. Document specific information
Introduction
Detailed test environment requirements
（詳細化されたテスト環境要求）
Overview（概要）
Unique identifier（UID）
Description（説明）
Responsibility（責任）
Period needed（必要な期間）

45. Document specific information
Introduction
Test data status（テストデータステータス）
Overview（概要）
Unique identifier（UID）
Description of status（ステータスの説明）

46. Document specific information
Introduction
Test environment readiness（テスト環境準備）
Overview（概要）
Unique identifer（UID）
Description of status（ステータスの説明）

47. 項目は未定義。

48. 項目は未定義。

49. Document specific information
Introduction
Events（イベント）
Overview（概要）
Unique identifier（UID）
Time（時間）
Description（説明）
Impact（影響度）

50. Document specific information
Introduction
Incident details（インシデント詳細）
Timing information（タイミング情報）
Originator（起案者）
Context（コンテキスト）
Description of the incident（インシデントの説明）
Originator's assessment of severity（起案者の深刻度評価）
Originator's assessment of priority（起案者の優先度評価）
Risk（リスク）
Status of incident（インシデントの状態）

51. IEEE Std 829-1998 IEEE Std829:2008TestPlan
TestDesignSpecification
TestCaseSpecification
TestProcedureSpecification
TestitemtransmittalReport
TestLog
TestIncidentReport
TestSummaryReport
MasterTestPlan
LevelTestPlan
LevelTestDesign
LevelTestCase
LevelTestProcedure
LevelTestLog
AnomalyReport
LevelInterimTestStatusReport
LevelTestReport
MasterTestReport
- - - - - - - - Test Policy - - - - - - - - - -
- - - - - - - - Organizational Test Strategy - - - - - - - - - -
x - - - - - - - Test Plan x x - - - - - - - -
- x - - - - - - Test Design Specification - - x - - - - - - -
- - x - - - - - Test Case Specification - - - x - - - - - -
- - - x - - - - Test Procedure Specification - - - - x - - - - -
- - - v - - - - Test Data Requirements - - - - v - - - - -
- - - v - - - - Test Environment Requirements - - - - v - - - - -
- - - - - - - - Test Data Readiness Report - - - - - - - - - -
- - - - - - - - Test Environment Readiness Report - - - - - - - - - -
- - - - - v - - Actual Results - - - - - v - - - -
- - - - - v - - Test Result - - - - - v - - - -
- - - - - x - - Test Execution Log - - - - - x - - - -
- - - - - - x - Incident Report - - - - - - x - - -
- - - - - - - - Test Status Report - - - - - - - x - -
- - - - - - - x Test Completion Report - - - - - - - - x x

52. Reference
ISO/IEC/IEEE 29119関連
• ISO/IEC/IEEE 29119-1:2013 Software testing – Part 1: Concepts and definitions
• ISO/IEC/IEEE 29119-2:2013 Software testing – Part 2: Test processes
• ISO/IEC/IEEE 29119-3:2013 Software testing – Part 3: Test documentation
• ISO/IEC/IEEE DF 29119-4 Software testing – Part 4: Test techniques
• 高信頼な組込みソフトウェアのためのテスト事例と最新動向 (羽田裕氏)
IEEE Std 829関連
• IEEE Std 829:1998 Standard for Software Test Documentation
• IEEE Std 829:2008 Standard for Software and System Test Documentation
ISTQB/JSTQB関連
• テスト技術者資格制度 Foundation Level シラバス 日本語版 Version 2011.J02
• ソフトウェアテスト標準用語集 （日本語版）Version 2.2.J01
ISO/IEC 関連
• ソフトウェアエンジニアリング関連の標準化 ～ ISO/IEC JTC1/SC7による国際標準化とSEC
の取組み～(高橋光裕氏)

53. END