Weitere ähnliche Inhalte Ähnlich wie Need To Automate Test And Integration Beyond Current Limits? (20) Mehr von Ghodhbane Mohamed Amine (7) Kürzlich hochgeladen (20) Need To Automate Test And Integration Beyond Current Limits?1. Jakob Engblom, Product Line Manager, System Simulation, Wind River
Graham Morphew, Sr. Director of Product Management, System Simulation, Wind River
Need To Automate Test And Integration Beyond
Current Limits? Use Simulation
Moderator:
Brandon Lewis, OpenSystems Media
Speakers:
3. 3 © 2015 Wind River. All Rights Reserved.
Need To Automate Test and
Integration Beyond Current
Limits? Use Simulation
3
Maximize Velocity – Webinar Series
Agile, Continuous Practices and DevOps
Graham Morphew & Jakob Engblom
Wind River
4. 4 © 2015 Wind River. All Rights Reserved.
Embedded Software Trends
Software driving majority of
value and innovation
IoT driving future differentiation,
value & service
Customer expectations have
changed to demand new
features more often and faster
Pushing software organizations
to change and be more agile
and efficient
Change
Change
Change
5. 5 © 2015 Wind River. All Rights Reserved.
Faster Delivery Of Software To Customers
Agile Continuous Practices Cross Disciplined Teams DevOps
New development practices and organization styles
getting established for embedded software development
Key Challenges:
• Access
• Collaboration
• Automation
Automation is key because manual processes don’t scale.
Covered in previous webinar
Covered in upcoming webinar
6. 6 © 2015 Wind River. All Rights Reserved.
Developer changes
or adds code
Build and test:
single unit
Build and test:
multiple units
Build and test:
entire system
Code
OK
Immediate
Coffee break or lunch
Daily
Continuous Practices
Quick feedback to developers
Find different classes of errors at each level
Avoid big-bang integration & associated problems
Avoid regressions by rerunning old tests
= higher quality and shorter lead times
(Typical time frames)
Needs to be Automated!
7. 7 © 2015 Wind River. All Rights Reserved.
The Hardware Test Lab
bus, network, …
Data generator or
world model, running
on hardware
Target provisioning
and control
JTAG, serial,
Ethernet, flash
programmer, …
System under
test
Test manager
Build server
Lab test rig – critical resource
Automation based
on coordinating
several separate
pieces of hardware
8. 8 © 2015 Wind River. All Rights Reserved.
The Challenge of Automation in Hardware
Hardware test lab is difficult to build, scale and maintain
How many lab setups can you afford?
Reconfiguration is manual, even when tests are automatic
Automation is often brittle
Impact
• Large equipment expenses
• Unable to scale
• Automation of failure
9. 9 © 2015 Wind River. All Rights Reserved.
Simulation system – all just software
The Simulation Test Lab
Simulated bus,
network, …
Data generator or
world model, running
as software only
Target provisioning
and control
System under
test - simulated
Test manager
Build server
Simulator
tools
Model of
the world
Use the existing test
management system
Automation based on
software control, not
hardware control!
10. 10 © 2015 Wind River. All Rights Reserved.
Simulation Makes Automation a Reality
Automate any operation on the target system
Test in the context of any system or subsystem
Parallelize, scale & reconfigure test suites in any way
Use any server to run any test – flexibility*
Send the context of test failure to developers to solve*
Value using simulation for automation
• Make embedded test automation possible & cost effective
• Do more, effective testing and get higher quality releases
• Deploy faster and more frequently
11. 11 © 2015 Wind River. All Rights Reserved.
Parallelization and Performance
Simulation provides automation and access, which enables
more tests to be run in parallel
Test 1 Test 2 Test 4Test 3 Test 5
simulation
Test 1
Test 2
Test 4
Test 3
Test 5
Overnight tests take a few minutes
Weekly tests become daily tests
Test more often = less chance for bugs to creep
back into the software – higher quality
1/5 the time
12. 12 © 2015 Wind River. All Rights Reserved.
Test management server
Test run system
(such as Jenkins)
Test generation
and management
Test server 1
Simics
board
Networkboard
board
Testing
agent/
slave
node
Parallel Testing and
Automation Setup
Physics/Electronics/World
Test server n
Simics
board
Network
board
board
Testing
agent/
slave
node
Physics/Electronics/World
board
Test system launches
simulated test labs
Each test can be of a different
configuration
Sends out test cases &
collects results
13. 13 © 2015 Wind River. All Rights Reserved.
Simulation
Expanding Test Coverage: Variation
Simulation
Board A
Simulation
Board A
Network
Board B
Network
Board A
Board A
Board B
Board C
Simulation
Board B
Program 1
Program 2
Program 3
Program 4
generates
generates
generates
System setups are just
software
Change setup program
to get a new
configuration
Infinite supply of each
type of system
Randomly generate
setups for “didn’t think
of that” tests
System creation &
setup is automatable
Input file
World / Physics
Simulation
14. 14 © 2015 Wind River. All Rights Reserved.
Expanding Test Coverage: Faults
Fault injection is a natural match for simulation
Test fault detection, handling, recovery, reporting, …
POST, diagnostic software, firmware, applications, system, …
For example:
– Corrupt and drop network packets
– Feed bad data from the environment
– Crash processors
– Change memory contents
– Modify system configuration on the fly
Simulation makes faults:
– Repeatable
– Non-destructive
– Scriptable and automatable!
15. 15 © 2015 Wind River. All Rights Reserved.
Automatically capture test failures
Analyze using record and replay of tests
Run and debug in reverse
When Things go Wrong
More on Collaboration with simulation in our next webinar.
16. 16 © 2015 Wind River. All Rights Reserved.
Key Take-Aways
Continuous integration and testing requires automation
Automation is made easier by using simulation
Simulation provides the ability to automate any test
Simulation allows testing and automation of what
cannot be tested with hardware
18. Audience Q & A
Jakob Engblom,
Product Line Manager,
System Simulation,
Wind River
Graham Morphew,
Sr. Director of Product Management,
System Simulation,
Wind River
19. Thanks for joining us
Event archive available at:
http://ecast.opensystemsmedia.com/
E-mail us at: jgilmore@opensystemsmedia.com