1) The document outlines a high level test strategy that involves layering the project under test and identifying components in each layer. It describes identifying test basis documentation, creating a dependency matrix, and formulating an overall test "big picture". 2) Test packs will be designed based on project layers, and key documentation will be stored in a repository to facilitate test coverage analysis. A dependency matrix and big picture diagram will guide regression test selection. 3) Tools like DocIndex, InternetMiner and VisioDecompositer are used to extract and store information from documents, web pages and diagrams to generate the test basis repository, and inform the dependency matrix and big picture diagram.

1. High Level Test Strategy overview
Test Strategy
• “Layer” the project
• Identify the components in each “layer”
• Identify the test basis
• Draw up a Dependency matrix
• Formulate a testing “Big Picture”
• Test Packs
• Summarising and going forward ….

2. High Level Test Strategy overview
“Layer” the project
• Integration services runs on (currently) 3 Biztalk platforms: IS2000 (PoS), IS2002
(ISGM) and IS2004
• Each platform contains various abstract layers:
– Service layer
– Application layer
– Database layer
– Biztalk layer
Queue
ReceiveFunctions
Adapters
Orchestrations
MessageBoxes
ServerHosts
– Test Environment

3. High Level Test Strategy overview
Components in each layer
• Service layer
– IS01, IS02, IS03, IS04, BP01, BP02 etc.
• Application layer
– RuleExec, Janus Web, HOPPA etc.
• Database layer
– NDTRefData, Alerter, HAT etc.
• Biztalk layer
– Queue - Workqueue, Retry, Scheduled etc.
– ReceiveFunctions - Failed Messages, Receiver
– Adapters - FTP, HTTP, SOAP etc.
– Orchestrations – BP01, BP02 etc.
– MessageBoxes – MsgBox DB, BAM archive etc.
– ServerHosts – Orchestration host, Tracking host, SendHost adapters etc.
• Test Environment
– DEV, Systest 1, Systest 2, CATE 1, CATE 2, little CATE etc.

4. High Level Test Strategy overview
Test basis N
• This is the documentation that supplies the Expectations, Requirements and
Expected Results for ANY given piece of testing effort
• Prime source of these here is the SUMMIT methodology deliverables – i.e. Project
Charter, System Prospectus, System Delivery Spec (SDS), Tech System Design
(TSD), Tech Procedure Development (TPD) etc.
• Test basis should (ideally) consist of SIGNED-OFF DOCUMENTATION ONLY.
However – real-world here and it may include draft docs but increased risk of
inaccurate Requirements and/or Expected Results
• All pertinent test basis documents collated as “KeyDocs” and are secured into
Quality Center

5. High Level Test Strategy overview
Dependency matrix N
• This matrix will highlight the implicit and explicit relationship between IS
components on each abstract layer
• It is used for impact analysis and, hence, to identify likely candidates for
regression testing here in IS
• Where change is known to have occurred to one component in the abstract layer
model, the reader can quickly follow the implicit and explicit relationships to other
components that are identified in this matrix. As the reader navigates through the
matrix, further likely regression test candidates (and hence Test Packs) are
identified
• Change to one particular component may involve regression testing “most” of IS

6. High Level Test Strategy overview
Testing Big Picture N
• This is, in essence, a amalgamation of all of the useful diagrams identified as part
of the test basis
• This model offers a tabular view of the testing and will involve the decomposition
of those diagrams in the KeyDocs repository
• Data from the diagrams is dropped into a two-dimensional table and the
information from the diagram (Names in boxes, names against lines, where the
lines come from, where the lines go to etc) is then accessible AND, more
importantly, reusable from a testing perspective
• Impact analysis is greatly facilitated by using this Big Picture…You can not only
prove the Dependency matrix – which means you can accurately describe what
related, regression tests need to be run but you also have a pictorial view of
everything testing. INVALUABLE

7. High Level Test Strategy overview
Test Packs N
• The management will usually have a view at what level of granularity a Test Packs
should be designed. Integration Services (IS) have chosen “layers” within
platform. i.e ISGM Biztalk, or IS2004 Database, or IS01 service etc.
• The FEWER Test Packs you choose to have, the MORE has to go into each Test
Pack. This increases complexity, which increases the chance of failure in a Test
Pack, BUT the MORE Test Packs you have, the more resources required to
create, modify, prepare and execute them!
• If you “spread the load” by having more Test Packs (higher level of granularity),
then you have a more agile way of testing change BECAUSE you wont need to
regression test ALL Test Packs.

8. High Level Test Strategy overview
Summarising N
• Test Packs are used to regression test
• KeyDocs repository contains the (signed-off and master) test basis documentation which
will facilitate proof of test coverage (i.e. HAVE we covered everything in the documentation?)
• Dependency matrix helps you to decide (accurately when used in conjunction with the Big
Picture) exactly what else is candidate for test
• Big Picture gives pictorial representation of ALL things testing on a project/programme. The
data/information contained therein is ultimately reusable because it also exists in two-
dimensional form!
• If all this seems VERY COMPLEX, then wait till something CHANGES. i.e. documentation or
process. THEN it gets complex because you don’t know what impact that change will have!!
• The beauty of the above is that I have tools that will automatically decompose
documents/diagrams so “change” merely means re-extraction of the data from the changed
documents or diagrams (text from documents or text/context information from diagrams) into
KeyDocs or Big Picture……. Next slide elaborates more
• ….. But this is all talking the talk, now lets walk the walk

9. High Level Test Strategy overview
Walk the walk N
• DocIndex provides the engine that drives the creation of the KeyDocs (test
coverage and Expected Results).
• InternetMiner provides the engine that assists the creation of KeyDocs and
operates on Intranet/internet pages and hyperlinks to extract similar information to
DocIndex. This is more an ancillary tool that complements DocIndex and is
useful for gaining insight into specific areas of expertise.
• VisioDecompositer provides the engine that drives diagram information (process
flow and data flow) into the Big Picture.
• Here’s some screen shots to give you a better flavour….

10. High Level Test Strategy overview
DocIndex 1 of 10N
• DocIndex (viewer) main menu

11. High Level Test Strategy overview
DocIndex 2 of 10
• KeyDocs main menu

12. High Level Test Strategy overview
DocIndex 3 of 10
• Display a selectable list of KeyDocs

13. High Level Test Strategy overview
DocIndex 4 of 10
• Now display a summarised list of the sections in the selected KeyDoc

14. High Level Test Strategy overview
DocIndex 5 of 10
• Select one of the sections in the document and the text is displayed

15. High Level Test Strategy overview
DocIndex 6 of 10
• The power of this tool lies in the fact that ALL relevant project/programme-level
documentation is captured into the tool

16. High Level Test Strategy overview
DocIndex 7 of 10
• The contents of all of the KeyDocs are stored in two tables: tblSectionHeaders –
which is a summarised “table of contents” for a KeyDoc

17. High Level Test Strategy overview
DocIndex 8 of 10
• and File_Decomposition – which is textual body for all KeyDocs

18. High Level Test Strategy overview
DocIndex 9 of 10
• There is a keyword/key phrase Search facility for the group of KeyDocs, which allows
the user to quickly become acquainted with “project specific” terminology

19. High Level Test Strategy overview
DocIndex 10 of 10
• and the reader can, very quickly, become acquainted with the context in which the
specific search terms are used.

20. High Level Test Strategy overview
InternetMiner 1 of 10 N
• InternetMiner main menu

21. High Level Test Strategy overview
InternetMiner 2 of 10
• Enter a suitably relevant URL - in this case, found by doing a “biztalk performance test”
search string in Google UK

22. High Level Test Strategy overview
InternetMiner 3 of 10
• And the page(s) are decomposed into InternetMiner, in much the same way as
DocIndex

23. High Level Test Strategy overview
InternetMiner 4 of 10
• The list of decomposed URLs can be viewed (and refreshed either manually or automatically) so
that the information contained therein is always up to date

24. High Level Test Strategy overview
InternetMiner 5 of 10
• And the content from ALL of the decomposed URLs can be viewed at a glance

25. High Level Test Strategy overview
InternetMiner 6 of 10
• And relevant search keywords or key phrases can be entered

26. High Level Test Strategy overview
InternetMiner 7 of 10
• And the result be displayed (in various forms) - so that the user can see, contextually,
where the search term is used

27. High Level Test Strategy overview
InternetMiner 8 of 10
• Alternative views of the results (and search strings) can be made

28. High Level Test Strategy overview
InternetMiner 9 of 10
• And you can quickly build up a picture of the various (in this case technical Biztalk)
phrases

29. High Level Test Strategy overview
InternetMiner 10 of 10
• And how they relate to the overall topic (Biztalk Performance testing)

30. High Level Test Strategy overview
VisioDecompositer 1 of 2N
• VisioDecompositer takes a straightforward Visio diagram…..

31. High Level Test Strategy overview
VisioDecompositer 2 of 2N
• And converts it into two-dimensional information that can be “mapped” into test cases
or test scripts quite easily