Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

Tools for Test-Driven Product Modeling

Tools for Test-Driven Product Modeling

Herunterladen, um offline zu lesen

Test-driven product modeling is a methodology of product modeling which leads to quick development of high-quality product models. Automatizable test case(s) are written before the product model is implemented. Subsequently the product model is modified until no test cases fail. With subsequent refactoring steps the design and implementation is improved.

To apply this development process, product modelers need to write test cases themselves while implementing the product model. The test cases have to be created and maintained easily and tests have to be executed very quickly. After running the tests, it must be very easy to analyze test results. Furthermore, support for refactoring the product model and test cases is required.

Special tool support is essential to model products in a test-driven way. As an example of such tool support, we present the integration of the ConfigScan test engine into the Eclipse-based modeling environments VClipse and ConfigModeler.

VClipse is an open-source product-modeling environment. Product models can be specified using the textual language VCML. ConfigModeler is a customer-specific product-modeling environment built in a customer project at Nokia Siemens Networks. Product models can be specified in a high-level and customer-specific language.

2012 CWG European Conference - Berlin, Germany (Tim Geisler, webXcerpt Software GmbH, Christophe Faure, Fysbee SAS)

Test-driven product modeling is a methodology of product modeling which leads to quick development of high-quality product models. Automatizable test case(s) are written before the product model is implemented. Subsequently the product model is modified until no test cases fail. With subsequent refactoring steps the design and implementation is improved.

To apply this development process, product modelers need to write test cases themselves while implementing the product model. The test cases have to be created and maintained easily and tests have to be executed very quickly. After running the tests, it must be very easy to analyze test results. Furthermore, support for refactoring the product model and test cases is required.

Special tool support is essential to model products in a test-driven way. As an example of such tool support, we present the integration of the ConfigScan test engine into the Eclipse-based modeling environments VClipse and ConfigModeler.

VClipse is an open-source product-modeling environment. Product models can be specified using the textual language VCML. ConfigModeler is a customer-specific product-modeling environment built in a customer project at Nokia Siemens Networks. Product models can be specified in a high-level and customer-specific language.

2012 CWG European Conference - Berlin, Germany (Tim Geisler, webXcerpt Software GmbH, Christophe Faure, Fysbee SAS)

Weitere Verwandte Inhalte

Ähnliche Bücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Ähnliche Hörbücher

Kostenlos mit einer 30-tägigen Testversion von Scribd

Alle anzeigen

Tools for Test-Driven Product Modeling

  1. 1. Tools for Test-Driven Product Modeling Tim Geisler Christophe Faure webXcerpt Software GmbH Fysbee SAS tg@webxcerpt.com Christophe.Faure@fysbee.com CWG 2012, Berlin 2012-05-08
  2. 2. Context: ConfigModeler @ StandardizedStandardized productproduct descriptionsdescriptions
  3. 3. Context: ConfigModeler @ StandardizedStandardized productproduct descriptionsdescriptions FormalizedFormalized productproduct descriptionsdescriptions
  4. 4. Context: ConfigModeler @ StandardizedStandardized productproduct descriptionsdescriptions FormalizedFormalized productproduct descriptionsdescriptions ConfigModeler + VClipse Eclipse-based IDEs code generation CML as specification and documentation language CWG 2010 Vienna CWG 2011 Cologne
  5. 5. ConfigModeler & VClipse @ RFC Editor/IDE Editor/IDE ConfigModeler VClipse CML VCML Code Generation
  6. 6. Retrospective ● Product-model quality and modeler productivity increased by use of ConfigModeler. − 2/3 of all live models made with ConfigModeler ● But: Functional testing by business showed that model implementation is not always what was intended by business.
  7. 7. Problems with Testing ● Product models are not tested − Acceptance tests by business ● Product models are not tested systematically − Test plans ● Product models are not tested automatically − Automated testing tools: Fysbee ConfigScan, Sybit Model Tester ● Product models are tested too late − ???
  8. 8. Traditional Development and Testing Process Design Implement Test
  9. 9. Test Earlier Design Implement Test
  10. 10. Test-Driven Development Design Implement Test Test
  11. 11. Test-Driven Development Design Implement Test Test
  12. 12. Test-Driven Development Design Implement Test Test Refactor / Clean Up
  13. 13. Requirements ● Test cases should be written as soon as possible – even during the specification phase of the model. ● Certain test cases must be written by the modeler (together with the business?) ● Test cases must be executed quickly. ● No media or tool changes for writing test cases and executing tests.
  14. 14. Example: PC Configurator developed with Test-Driven Product Modeling (Screencast)
  15. 15. Developer's Tools
  16. 16. Developer's Tools ● rich test-language constructs − values, BOM, status, domains, visibility ● test engine tests „real thing“ − VC and IPC, on real system ● integrated into modeling IDE − navigation ● fast edit-and-test cycles: − model setup in SAP with one click − test execution from inside modeling environment ● refactoring support − simultaneous maintenance of model and test cases
  17. 17. Test Engine ConfigConfigScanScan ● For VC and IPC ● Test-case database inside SAP ● Rich test language ● RFC-enabled API − Call test engine from external modeling environment − Upload test cases as XML files to ConfigScan's test-case database for recurring test runs. − Execute XML streams as test cases with ConfigScan's test engine with or without storing them – even on remote systems
  18. 18. ConfigConfigScanScan's Test Language ● XML-based sequential language ● Simulate user's interaction with the product model ● Commands on cstic level − Set a value for a cstic − Verify that values are (not) in the cstics domain − Verify whether the cstic is (not) set to a specific value − Verify that a cstic is set, unset, visible, hidden, set by default, … ● Commands on BOM item level − Verify the quantity of a bom item − Verify that a BOM item is complete, consistent, instantiated ● Hierarchical nesting of test groups
  19. 19. Test Language CMLT ● Integrated with CML ● All ConfigScan test statements ● Hierarchical models through „context“ ● Nested test groups ● New compound statement „bom“
  20. 20. IDE for Editing CMLT tooltips refactoring templates outlinequickfix folding content assist navigation
  21. 21. Test Coverage ● „Static“ test coverage: which characteristics and materials are covered by test cases? ● Could be refined, e.g.: − Which value is covered by test cases? − Which dynamic domain is covered by test cases?
  22. 22. Integrating End-User Tests ● Model owner does acceptance tests typically in end-user environment: IPC ● Export configuration from IPC as .cfg file ● Automatically transform .cfg file into CMLT test cases − Represent only cstic settings, BOM, and status − No domain and user interaction sequence ● When defects are found, create small unit tests which reproduce failures.
  23. 23. Test Execution from ConfigModeler ● Execute ConfigScan simultaneously on multiple files and multiple systems ● Additional parameters for ConfigScan ● Store launch configuration
  24. 24. ConfigConfigScanScan View – Display Logs ● ConfigScan logs are displayed in a tree view ● Various actions: − Expand / collapse − Next error − Filter failed tests − Save / load log − History − Re-run (failed) tests
  25. 25. ConfigConfigScanScan View – Navigate to Test
  26. 26. ConfigConfigScanScan View – Group by Error ● Failures can be grouped by the error message. ● This enables quick problem analysis when one defect is detected by several failed test cases.
  27. 27. VClipse & ConfigConfigScanScan ● VClipse: Eclipse-based modeling environment for the Variant Configurator, textual language VCML ● VClipse is backend for ConfigModeler ● ConfigScan integration is part of VClipse ● The presented interface from ConfigModeler is just an instantiation of the VClipse / ConfigScan integration. ● VClipse is open-source (www.vclipse.org) − You can get the ConfigScan integration for free − Requirement: ConfigScan must be installed in your landscape
  28. 28. VCML and VCMLT Testing language VCMLT for VCML models (contributed to VClipse by Winfried Kung)
  29. 29. Adapt ConfigConfigScanScan Integration for Your Testing Language ● Create your own modeling language ● Create your own testing language ● You can use the ConfigScan integration by implementing some simple interfaces − Translation of your testing language to ConfigScan XML language − Optionally own styling code for tree view ● Example: VCMLT implementation on www.vclipse.org
  30. 30. Testing with ConfigConfigScanScan (Test Type)
  31. 31. Testing with ConfigConfigScanScan (Role/Type)
  32. 32. Testing With ConfigConfigScanScan ● From the beginning ConfigScan was designed for Test Driven Modeling (CWG Salzburg 2007) ● Test engines provide external API and detailed XML testing language ● Test are organized in standard SAP Catalogs ● Test can be executed in standard SAP JOB − Scheduled, periodic, … ● Dedicated mail notification of testing JOB ● Test logs are hierarchic and stored in SAP LOG
  33. 33. Test Management in ConfigConfigScanScan Testing framework with several components ● Test Engines (local/remote IPC and VC) − Dynamic testing language ● Test Editor/Importer (material centric) − Test document with nested test group ● Test Cockpit (catalog centric) − Organize tests in SAP Catalog − Schedule testing job − Mail notification ● Job Monitor (job centric) − Manage testing job
  34. 34. Testing with ConfigConfigScanScan (Architecture)
  35. 35. Comparison with Software Engineering Xunit test frameworks ● Test language + test-execution framework ● Integrated into development environment ➔ VClipse / ConfigModeler - ConfigScan integration Continuous Integration ● System which continuously builds code ● Test cases are run periodically ● Notifications, if tests fail ● CI monitoring ➔ ConfigScan's test management
  36. 36. Experiences and Uses at NSN ● Used also for model migration from legacy configurators − Model behavior of legacy configurator as reference for test cases ● Scenarios during product modeling: − Development tests (D) − Tests for product transport (D Q P) − Tests for traceability (P with saved logs) ● Regression testing
  37. 37. Summary ● Test-driven product modeling is feasible using an automated test engine and an external modeling environment with test-engine integration. ● The presented methodology and tools have been in use at Nokia Siemens Networks in production for selected products since July 2011. ● Integrating different products from CWG members with open interfaces can increase customer value.

×