QA: Challenge Accepted 3.0

1. How PENETRATION TESTING
techniques CAN HELP YOU IMPROVE
your QA skills
Marian HackMan Marinov
<mm@siteground.com>

2. Who am I?Who am I?
❖ Chief System Architect of Siteground.com
❖ Sysadmin since 1996
❖ Organizer of OpenFest, BG Perl Workshops,
LUG-BG and similar :)
❖ Teaching Network Security and Linux System
Administration at Sofia University

3. ☹☹☹☹☹☹
The QA position is underrated and heavily
undermined by the web industry

4. ☹☹☹☹☹☹
The QA position is underrated and heavily
undermined by the web industry
I really admire the work that QA people do!

5. Let's clear some stuffLet's clear some stuff
❖ Black box testing
❖ Gray box testing
❖ White box testing

6. Pentesting what?Pentesting what?
❖ Pentesting is actually QA testing :)
❖ The plans that pentesters prepare for
attacks are similar to the plans QA devise to
test the same software.
❖ The difference is the way the testing is
performed and what is evaluated for testing.

7. The fist part of the presentation
would be about Black/Grey box testing
The fist part of the presentation
would be about Black/Grey box testing
Simply because it shows some
very important differences
Simply because it shows some
very important differences

8. Pentester vs. QAPentester vs. QA

9. Pentesters start by gathering information
about the infrastructure and the software
Pentesters start by gathering information
about the infrastructure and the software
QA guys, usually relay on
Devs, Documentation and
Requirements for the same info
QA guys, usually relay on
Devs, Documentation and
Requirements for the same info

10. Pentesters define the expected successful
result by running one real life test and
writing down its result. Than every other
response is considered failure
Pentesters define the expected successful
result by running one real life test and
writing down its result. Than every other
response is considered failure
Every time a QA see an error in the
logs of the app or the server,
she is considering it, a failure
Every time a QA see an error in the
logs of the app or the server,
she is considering it, a failure

11. Pentesters try to find the places that are
most likely to be vulnerable
(common flaws and bugs)
Pentesters try to find the places that are
most likely to be vulnerable
(common flaws and bugs)
QAs tryies to test the whole applicationQAs tryies to test the whole application

12. Pentesters always look for chain of events,
that if triggered, will give them access
(Design vulnerabilities)
Pentesters always look for chain of events,
that if triggered, will give them access
(Design vulnerabilities)
QAs usually focus on the current layer
of the application and relay on the tests
of the next level to cover the whole
QAs usually focus on the current layer
of the application and relay on the tests
of the next level to cover the whole

13. Pentesters try to use and abuse
unspecified behavior in functions, and for
that they try generate all impossible inputs
Pentesters try to use and abuse
unspecified behavior in functions, and for
that they try generate all impossible inputs
QAs usually limit their tests to
a predefined set of inputs or
limited number of generated inputs
QAs usually limit their tests to
a predefined set of inputs or
limited number of generated inputs

14. Every time a test fails,
pentesters expand the search
Every time a test fails,
pentesters expand the search
With QAs, every time a test fails,
they return the code to the Dev team
With QAs, every time a test fails,
they return the code to the Dev team

15. Pentesters have more time and they
are working on the LIVE environment
Pentesters have more time and they
are working on the LIVE environment
QAs do their tests on
mockup/staging environments
QAs do their tests on
mockup/staging environments

16. Pentesters use tools for
generating wordlist and dictionaries
Pentesters use tools for
generating wordlist and dictionaries
QAs should start doing the same...QAs should start doing the same...

17. Sometimes pentesters get their hands on
binaries and they do try to reverse them,
to understand and break them
Sometimes pentesters get their hands on
binaries and they do try to reverse them,
to understand and break them
I'm yet to see a QA do some reversing...I'm yet to see a QA do some reversing...

18. ❖❖ Decision table testingDecision table testing
❖❖ All-pairs testingAll-pairs testing
❖❖ Equivalence partitioningEquivalence partitioning
❖❖ Boundary value analysisBoundary value analysis
❖❖ Cause–effect graphCause–effect graph
❖❖ Error guessingError guessing
❖❖ State transition testingState transition testing
❖❖ Use case testingUse case testing
❖❖ User story testingUser story testing
❖❖ Domain analysisDomain analysis
❖❖ Combining techniqueCombining technique
Black/Gray box testingBlack/Gray box testing

19. Generally pentesters use more tools for
a single test then QAs.
They try to combine more then one
completely valid use case, to construct
one test, that will break the system.
Generally pentesters use more tools for
a single test then QAs.
They try to combine more then one
completely valid use case, to construct
one test, that will break the system.

20. And here is something from the bookAnd here is something from the book
Exploiting SoftwareExploiting Software::
An attacker starts with breaking theAn attacker starts with breaking the
rules and undermining assumptions.rules and undermining assumptions.
One of the key assumptions to test isOne of the key assumptions to test is
the "implicit trust" assumption.the "implicit trust" assumption.
Attackers will always break any ruleAttackers will always break any rule
relating to when, where, and what isrelating to when, where, and what is
"allowed""allowed"
Black/Gray box testingBlack/Gray box testing

21. If you want to broaden
your test case horizons...
I advise you to read
the specific techniques
outlined in "Exploiting Software"
If you want to broaden
your test case horizons...
I advise you to read
the specific techniques
outlined in "Exploiting Software"

22. ❖❖ shell command injectionshell command injection
* argument injection* argument injection
* command delimiters* command delimiters
* multiple parsers and double escapes* multiple parsers and double escapes
❖❖ Plumbing pipes, ports and permissionsPlumbing pipes, ports and permissions
❖❖ Exploiting the filesystemExploiting the filesystem
* user-supplied variable passed to the filesystem calls* user-supplied variable passed to the filesystem calls
* postfix null terminator* postfix null terminator
* postfix null terminate and backslash* postfix null terminate and backslash
* relative path traversal* relative path traversal
❖❖ Manipulating environment variablesManipulating environment variables
❖❖ Leveraging poor session authenticationLeveraging poor session authentication
* Session ID, Resource ID and Blind trust* Session ID, Resource ID and Blind trust
* Bruteforcing session IDs* Bruteforcing session IDs
* Multiple paths of authentication* Multiple paths of authentication
* Failure to check error codes* Failure to check error codes
Specific techniquesSpecific techniques

23. ❖❖ Every QA is doing exactly thisEvery QA is doing exactly this
White box testingWhite box testing

24. ❖❖ Every QA is doing exactly thisEvery QA is doing exactly this
❖❖ But let's look at the differencesBut let's look at the differences
White box testingWhite box testing

25. ❖❖ Pentesters do whitebox testingPentesters do whitebox testing
* when they are testing opensource apps* when they are testing opensource apps
* if they were given the code* if they were given the code
❖❖ Pentesters usually try to find bugs in thePentesters usually try to find bugs in the
integration between the app they have theintegration between the app they have the
source to and the app which source they don'tsource to and the app which source they don't
❖❖ Pentesters check, does the software complyPentesters check, does the software comply
withwith allall the security guidelines for the specificthe security guidelines for the specific
language or systemlanguage or system
White box testingWhite box testing

26. ❖❖ Pentesters use debuggersPentesters use debuggers
* to inject code* to inject code
* change memory* change memory
* even change the flow of the application* even change the flow of the application
❖❖ Injecting code or otherwise influencing theInjecting code or otherwise influencing the
application is considered by many, not possible.application is considered by many, not possible.
* This is now a reality!* This is now a reality!
❖❖ Pentesters consider bug in one layer as anPentesters consider bug in one layer as an
open door for your whole appopen door for your whole app
White box testingWhite box testing

27. ❖❖ Code coverageCode coverage
* Control flow testing* Control flow testing
* Data flow testing* Data flow testing
* Branch testing* Branch testing
* Statement coverage* Statement coverage
* Decision coverage* Decision coverage
* Modified condition/decision coverage* Modified condition/decision coverage
* Prime path testing* Prime path testing
* Path testing* Path testing
White box testingWhite box testing
QA

28. ❖❖ Code static analysisCode static analysis
❖❖ Analysis ofAnalysis of
* memory states, finding possible entry* memory states, finding possible entry
points for malicious codepoints for malicious code
* interaction between different parts of* interaction between different parts of
the software(exchanging data betweenthe software(exchanging data between
APIs and Frameworks)APIs and Frameworks)
❖❖ Forcing race conditions, filling buffersForcing race conditions, filling buffers
and then examining the performance ofand then examining the performance of
the app and its memory structuresthe app and its memory structures
White box testingWhite box testing
QA

29. ❖❖ In order to be able to build moreIn order to be able to build more
extensive tests, you have to be aware ofextensive tests, you have to be aware of
the possible issues that may happenthe possible issues that may happen
with:with:
- your software- your software
- the language(s) that you are using- the language(s) that you are using
- the infrastructure that is hosting your- the infrastructure that is hosting your
applicationsapplications
Finally!Finally!

30. We are hiring!We are hiring!
THANK YOUTHANK YOU Marian HackMan Marinov
<mm@siteground.com>

31. Related booksRelated books Marian HackMan Marinov
<mm@siteground.com>

32. We are hiring!We are hiring!
THANK YOUTHANK YOU Marian HackMan Marinov
<mm@siteground.com>