Integrating tools and concepts

1. Automated Testing in Pipelines
Integrating Tools and Concepts
Layla Franke, Vlad Ilie, Alina Ionescu
dev.haufe.com
github.com/Haufe-Lexware
@HaufeDev
-Lexware

2. Scope of this Talk
• Coded UI for UI, Postman + Newman for API (LicenseManager – Layla)
• Testing Framework for Web UI in Java with Selenium (Umantis – Vlad)
• Framework in Java for API testing, Soap UI for API testing (Foundational Services – Alina)

3. 1 LicenseManager - Layla
Coded UI for UI, Postman + Newman for API

4. License Manager
• C# Web Application
• Inhouse Tool for Management of Serial Numbers
• Existing Continuous Delivery Pipeline in Go.CD

5. GUI Tests
Coded UI Tests in Visual Studio:
• Recording via Internet Explorer and built-in recorder
• Execution of tests from Commandline with MSTest
• Playback of Tests in various browsers

6. Build Pipeline
Compile
Deploy Application
Coded UI Test
Internet Explorer
View Test Reports
Do other stuff
Pulish App
QualityDev
Coded UI Tests – Pipeline integration
• Compile your code normally
• Deploy you application on test machine
• Start Coded UI Tests
You can use MSTest.exe on Go Agents
• Use parameterized URL and Browser
Define environment variables in GoCD
• Test Replay runs on Go Agent
• Coded UI Tests Finished
View HTML Test Reports and Coverage Report in Pipeline
Coded UI Test
Chrome
Coded UI Test
Firefox
Go Agent
Go Agent
Go Agent

7. Coded UI Test Reports - Overview

8. Coded UI Test Report - Failed

9. Coded UI Test Report - Passed

10. Create REST API Tests using Postman
Use Postman to create REST API Tests:
• Export Tests as a „Postman Collection“
• Have a variable URL using a „ Postman Environment”

11. Newman – Pipeline integration
• Deploy application on App Server
• Execute Newman Tests
Newman is a command line tool used for
automatic execution of tests created with
Postman
• View Test Results in Pipeline
Build Pipeline Go Agent
Newman Test
QualityDev
Pulish App
App Server
View Result
Compile
Deploy App to Server

12. CD pipeline integration
How to integrate in CD Pipeline?
Visual Studio Coded UI Tests
• Test DLL is created in Compile Job
• Tests run after deployment on test environment in stage QualityDev
• Tests for different browsers can be executed in parallel
Newman REST API Tests
• Can be executed after deployment on test environment

13. 2 Umantis - Vlad
Testing Framework for Web UI in Java with Selenium

14. About Umantis
• Web project (perl)
• 3 applications
(with stable and beta customers)
• Bugfix release every 2 weeks
• Feature release every 3 months
• Prio1 releases

15. Test Project Structure
• Java project
• Built using maven
• Selenium web driver
• Page Objects
• testNG
• Git/gerrit
• Jenkins

16. 1 LicenseManager - Layla
Coded UI for UI, Postman + Newman for API

17. 1 LicenseManager - Layla
Coded UI for UI, Postman + Newman for API

18. Page Objects
• Keep UI elements private
(don’t expose them to tests)
• Expose user actions
• Use data structures instead
of multiple parameters
public class JobApplicationExternalFormPage extends PageBase {
@FindBy(id = "value_153712")
WebElement dataReleaseRadioButtonGroup;
..............................
@FindBy(id = "Button_1161113")
private WebElement submitFinalApplicationButton;
@FindBy(id = "Button_112465")
private WebElement continueButton;
@FindBy(id = "errorcollectionlabel_3606")
private WebElement errorMessage;
.............................
private void fillJobApplicationForm(User applicant) {
UIActions.waitForElement(titleRadioButtonGroup);
UIActions.setValue(new RadioButtonGroup(titleRadioButtonGroup),
applicant.getTitleAsValue());
UIActions.setValue(firstNameEditBox, applicant.getFirstName());
......................................................................
UIActions.setValue(letterOfApplication, applicant.getLetterPath());
UIActions.setValue(resume, applicant.getCvPath());
UIActions.setValue(document1, applicant.getDoc1Path());
try {
UIActions.setValue(new RadioButtonGroup(dataReleaseRadioButtonGroup),
RadioButtonValue.DATA_NOT_AGREE.getValue());
} catch (NoSuchElementException ignored) {
}
UIActions.click(iAgreeCheckBox);
}
.............................

19. Page Objects - examples
• PageBase.java
• CreateNewEmployeePage.java
• AddPartialEventPage

20. UIActions
• Static Class
• Entry point for all ui actions
• setValue() and checkValue() for
different types of controls

21. Test Execution
• Jenkins – a job for each product and branch
• Local – using intelliJ
• On each commit (continuous integration) using gerrit and jenkins
• Maven
• Command line (can be used with any CI or CD tool):
java –jar umantis_test.jar ...

22. Test Environment
• Selenium Grid using docker containers
• Build test project container (based on maven image)
• Test execution container (based on jdk image)
• Selenium grid (based on hub and node selenium images)

23. Test Results - overview

24. Test Results – Detail
• Test failure with history
• Exception with stack trace
• URL, screenshot and html
where the failure occured

25. License Manager
• C# Web Application
• Inhouse Tool for Management of Serial Numbers
• Existing Continuous Delivery Pipeline in Go.CD

26. What we started from:
•Atlantic – the monolith
15 + clients…

27. Splitting the testing:
• 1. DEV Tests – Junit tests
• 2. System Tests - Junit tests
• 3. SOAP UI tests
• 4. Security Tests
• 5. Load Tests
• 6. Rollout (Docker) tests
• 7. UI Automated tests
• 8. Exploratory (manual)

28. The Testing Framework
Junit Test Framework
Atlantic
Code
Clients
Helpers
Tests
SoapUI Tests
DEV
Unit Tests
Security Tests
Load Tests
Rollout
Docker Tests
Atlantic
Client
Atlantic
Client
Atlantic
Client
HP ALM
UI Tests
Manual
Tests

29. All the endpoints:

30. Tests examples:
@Test
public void get_userWithUserId() {
UserDto user1 = UserClient.createNewPendingActiveUser();
Map<String, String> queryParams = new HashMap();
queryParams.put("userid", user1.getUserID().toString());
com.haufe.atlantic.old.usermanagement.rest.dtos.UserDto result =
RestUtilForOldResources.get(com.haufe.atlantic.old.usermanagement.rest.dtos.UserDto.class, headers,
queryParams, PathsHelper.umPaths().users());
assertUserValues(user1, result, true);

31. Where are they triggered from:
• After deployment, from GoCD there are triggered 2 Docker compose files:
• 1 with Maven for the Junit tests
• 1 with SoapUI for the SoapUI tests
They run in parallel.

32. Results in Rocket.Chat:

33. Things we learned
• Achieve “good enough” coverage
• Prioritize
• Take/create shortcuts if you can
• Bypass Logins if possible
• Write small “straight to target” tests (no end-to-end anymore)
• Use more code and API less UI
• Communication is key to good collaboration

34. Goal?
Faster, more reliable releases.

35. GUI Tests
Coded UI Tests in Visual Studio:
• Recording via Internet Explorer and built-in recorder
• Execution of tests from Commandline with MSTest
• Playback of Tests in various browsers