2. Agenda
Introduction
The need of testing and it’s Terminology
Unit Testing
Right B.I.C.E.P.s.?
Benefits of Unit Testing
Naming standards for unit tests
3. Result of this problem
error - a mistake made by a human (in a software development
activity)
defect (or fault) - the result of introducing an error into a software
artifact (SRS, SDS, code, etc.)
failure - a departure from the required behavior for a system
A vicious cycle – the more pressure a you (Developer) feel, the fewer
tests you writes. The fewer tests you write, the less productive you are
and the less stable your code becomes. The less productive and
accurate you are, the more pressure you feel.
The Problem
4. Unit Testing
Unit testing is a software verification and validation method in which a
programmer tests if individual units of source code are fit for use.
The goal of unit testing is to isolate each part of the program and show that
the individual parts are correct.
Unit Testing checks that behaves correctly.
Static Testing
testing a unit without executing the unit code
Dynamic Testing
testing a unit by executing a program unit using test data
Some common definitions of a unit:
A unit is the smallest testable part of an application.
In procedural programming a unit may be an individual function or
procedure.
The smallest chunk that can be compiled by itself
A stand alone procedure of function
Something so small that it would be developed by a single person
5. Why Unit Testing????????????
Without unit tests defects show up in the end of development
Finding them is difficult. Removing them time-consuming and Complex
With unit tests we can find defects much sooner and easier, continuously
removing them before adding new features
We can also check if new features break our code
6. Unit Test:
Unit-tests
Should validate expected functionality
at the level of:
• individual class
• small groups of collaborating classes
• work package
Software
developer
CODE
Test
programs
Automatic
software test
run process
• Test execution framework
• Testing naming and structure
• Test case specification template
Unit-test Component
Code Documentation
Important code should have unit tests
Tests should be written together with the code
Code should pass minimum 70% of the unit tests
before it can be released
7. Unit-test: Test frameworks
Our constrains:
•TIME
Aim: to help developers:
Our inputs:
•What is available as free open source
code.
What we are doing:
•Trying Unit Testing Tools
in the our projects.
• Preparing “HowTo” documents to
make easier the installation of these
tools and the start with process
Our plans:
• Implementation of the Unit Testing
Procedure in the our project
environment.
• to produce code for unit-testing
• to run tests in automatic way
What we propose:
JUnit
NUnit
SimpleTest
8. Unit Testing makes your developer lives easier
Easier to find bugs
Easier to maintain
Easier to understand
Easier to Develop
You have already done Unit testing
Not structured
Not Repeatable
Not on all your code
Not easy to do as it should be
A framework is missing
9. Designing unit tests
Type of Unit Test
Positive Test Negatives
Test the function
with valid input data
Test the function
With invalid data
10. When to write the test
During Development - When you
need to add new functionality to the
system, write the tests first. Then you
will be done developing when the
test runs.
During Debugging - When
someone discovers a defect in your
code, first write a test that will
succeed if the code is working. Then
debug until the test succeeds.
“Whenever you are tempted to type something into a print
statement or a debugger expression, write it as a test
instead.”...
11. An individual
program unit
Test for both success
& failure.
For boundary
conditions
For general
functionality Etc..
Databases,
Stored
Procedures
, Triggers,
Tables,
Indexes
NT
Services
Database
conversion
.OCX,
.DLL,
.EXE and
other
binary
formatted
executables
Invariants:
things that must
always be true.
Post-conditions:
things that must
be true after you
run the code.
Pre-conditions:
Those things that
must be true
before you run
the code.
(Eg. The list size will
always be greater than or
equal to 0)
(Eg. The object’s buffer
has been initialized)
(Eg. method will never
return null)
(subprogram,
object class,
package, module)
What should be tested ?
12. Example of How To Think of Tests
The required functionality for a particular service
(getVersionsForProductId) is the ability to get a list of all versions
for a given product id. If there’s no product for that id, then an
exception is thrown. If there’s no versions for a valid product id,
then an empty list is returned.
The Tests That Are Needed
Starting from the easiest to the hardest:
If there’s no product for that id, then an exception is thrown (pre-
condition)
If there’s no versions for a valid product id, then an empty list is
returned (post-condition)
If there are versions for a product id, then a non-empty list of all the
versions is returned (post-condition)
13. Unit Testing Tasks and Steps:
Step 1: Create a Test Scenario
Step 2: Create Test Cases and Test Data
Step 3: If applicable create scripts to run test cases
Step 4: Once the code is ready execute the test cases
Step 5: Fix the bugs if any and re test the code
Step 6: Repeat the test cycle until the “unit” is free of all bugs
15. What Makes a Good Unit Test?
It sufficiently tests the contract. It doesn’t need to be complete, just
“sufficient.”
If it’s complex code, then have lots of tests.
If it’s a simple accessor, then you don’t need a test.
Most things are in between…
It runs quickly. Part of the point is to make it so that you’re able to run them
regularly, so speed plays into that.
It’s much easier to fix a bug you introduced five minutes ago than one you
did five weeks ago…
Tests are independent.
Loosely coupled functionality enables independent tests.
Write loosely coupled, highly cohesive code!
Run the tests regularly.
You can be “fearless” when you need to make changes, because you
don’t have to worry that you might’ve broken something.
16. When to Run Tests
When you write a new method……compile and run local unit tests
When you fix a bug……run the test that illustrates that bug.
Any successful compile……run local unit tests.
Before you check in……run all tests.
Continuously…...check out and build the project from scratch including
all unit tests.
Is incomplete (e.g. missing dependencies)
Doesn’t compile
Compiles but breaks other code
Doesn’t have unit tests
Has failing unit tests
Passes its tests but causes other tests to fail.
Do not check in code that…
17. Test Case Sample
Additionally the following information may also be captured:
a) Unit Name and Version Being tested
b) Tested By
c) Date
d) Test Iteration (One or more iterations of unit testing may be performed)
Comment
if any
Pass /
Fail
What actually
happens. This
column can
be omitted
when defect
recording tool
is used.
What
should
happen?
Input
Data
How to
Test
What to
Test
ID which can
be referred to
in other docs
like “TM”
“Root Cause
Analysis of
defects etc
RemarksPass /
Fail
Actual ResultExpected
Result
Input
Data
TC
Procedure
Test Case
Purpose
Test Case
18. Steps to Effective Unit Testing
Documentation: Early on document all the Test Cases needed to test your
code. Document the Test Cases, actual Results when executing the Test
Cases, Response Time of the code for each test case.
Important advantages if the test cases and the actual execution of test
cases are well documented.
Documenting Test Cases prevents oversight.
Documentation clearly indicates the quality of test cases.
If the code needs to be retested we can be sure that we did not
miss anything.
It provides a level of transparency of what was really tested during
unit testing. This is one of the most important aspects.
It helps in knowledge transfer in case of employee attrition.
Sometimes Unit Test Cases can be used to develop test cases for
other levels of testing
19. What should be tested when Unit Testing
It could be a screen or a component or a web
service
Test Cases to verify all the screen
elements that need to appear on the
screens.
Test cases to verify the
spelling/font/size of all the “labels” or
text that appears on the screen.
Test Cases such that every line of code
in the unit is tested at least once in a
test cycle.
Test Cases such that every condition in
case of “conditional statements” is
tested once.
Test Cases to test the
minimum/maximum range of data that
can be entered.
Test Cases to verify how various
errors are handled.
Test Cases to verify if all the
validations are being performed
20. Never write
a test that
succeeds the
1st
time
Start with
null case, or
something
that doesn’t
work
Try
something
trivial to
make the test
work
Loose
coupling &
testability go
hand in hand
Use mock
Objects
Write the
test first
Charles'
Six
Rules of
Unit
Testing
21. Keep unit tests small and fast
Unit tests should be fully
automated and non-interactive
Make unit tests simple to run
Measure the tests
Fix failing tests immediately
Keep testing at unit level
Start off simple
Keep tests independent
Keep tests close to the class
being tested
Name tests properly
Test public API
Think black-box
Think white-box
Test the trivial cases too
Focus on execution coverage first
Cover boundary cases
Provide a random generator
Test each feature once
Use explicit asserts
Provide negative tests
Design code with testing in mind
Don't connect to predefined
external resources
Write tests to reproduce bugs
Know the limitations
Unit Testing Guidelines
22. Are we testing the Right B.I.C.E.P.s.?
Testing without a strategy is futile.
R Are the results right?
B Are all the Boundary conditions correct?
I Can you test the Inverse relationship?
C Can you Cross-check using other methods/means?
E Can you force Error conditions to happen?
P Are Performance conditions within bounds?
s Always go for the whole set.
23. Right-BICEP
Right: Are the results right?
Validate results
Does the expected result match what the method does?
If you don’t know what “right” would be, then how can you test? How
do you know if your code works?
Perhaps requirements not known or stable
Make a decision. Your tests document what you decided.
Reassess if it changes later.
24. Right-BICEP
Boundary Conditions
Garbage input values
Badly formatted data like ’1970-23-12’ when 1970-12-23 is required.
Empty or missing values (0, null, etc.)
Values out of reasonable range
Duplicates if they’re not allowed
Unexpected orderings
Totally bogus or inconsistent values, such as filenames.
Validity like age in years between 0 and 90 e.g. 115.
Things that arrive out of order.
25. Right-BICEP
Check Inverse Relationships
If your method does something that has an inverse, then apply the
inverse
E.g. square and square-root. Insertion then deletion.
Beware errors that are common to both your operations
Seek alternative means of applying inverse if possible
26. Right-BICEP
Are the boundaries C.O.R.R.E.C.T.?
C Conformance. Is the format Ok?
O Ordering. Is the set (un)ordered as appropriate e.g. as dictated by
invariant.
R Is the value within acceptable Range?
R Does the code Reference anything external not under its direct control?
E Does the value Exist?
C Countability; Are there enough values?
T Tim(e|ing). Is everything happening in the right order?
27. Right-BICEP
Cross-check using other means
Can you do something more than one way?
Your way, and then the other way. Match?
Are there overall consistency factors you can check?
Overall agreement
If e.g. you are devising a square root for an embedded system without float capability,
you could test it (in the host environment) using the standard implementation, like
so:
public void testSquarerootUsingStd( ) {
double number=3830900.0;
double root1= mySquareRoot ( number ) ;
double root2= Math.sqrt ( number ) ;
assertEquals ( root2 , root1 , 0.0001 ) ;
}
28. Right-BICEP
Force Error Conditions; Can your module survive?
Strange things in time keeping (like a clock that has been set back or
summer and winter time).
Failures outside your code:
Invalid parameters, out of range values, etc.
Out of memory, disk full, network down, etc.
Can simulate such failures
High system load.
Limited color availability in user interface.
Very high or very low screen resolution.
29. Right-BICEP
Performance
A normal unit test should test for the functional requirements. But if the
non-functional requirement performance or speeds depend on the size of
the data, you should test that too.
Perhaps absolute performance, or
Perhaps how performance changes as input grows.
Perhaps separate test suite in JUnit
Example: a spam filter that tests against “naughty” or black-listed mail
servers. Test your code with different lists sizes. Typically a small list for
build testing, but maybe also with a longer list e.g. once a day.
30. Benefits of Unit Testing
It provides a strict, written contract that
the piece of code must satisfy.
It find problems early in the development
cycle.
It allows the programmer to refactor code
at a later date, and make sure the module
still works correctly (i.e. regression testing).
It may reduce uncertainty in the units
themselves and can be used in a bottom-up
testing style approach.
It provides a sort of living documentation of
the system. Developers looking to learn
what functionality is provided by a unit and
how to use it.
By testing the parts of a program first and
then testing the sum of its parts, integration
testing becomes much easier.
31. Result
# of Test Cases (Test Development Phase)
# of Test Cases Executed (Test Execution Phase)
# of Test Cases Passed (Test Execution Phase)
# of Test Cases Failed (Test Execution Phase)
# of Test Cases Under Investigation (Test Development Phase)
# of Test Cases Blocked (Test dev/execution Phase)
# of Test Cases Re-executed (Regression Phase)
# of First Run Failures (Test Execution Phase)
Total Executions (Test Reporting Phase)
Total Passes (Test Reporting Phase)
Total Failures (Test Reporting Phase)
Test Case Execution Time ((Test Reporting Phase)
Test Execution Time (Test Reporting Phase)
32. Result…Cont……
% Complete
% Defects Corrected
% Test Coverage
% Rework
% Test Cases Passed
% Test Effectiveness
% Test Cases Blocked
% Test Efficiency
1st Run Fail Rate
Defect Discovery Rate
Overall Fail Rate
33. Naming standards for unit tests
Test name should express a specific requirement. The
basic naming of a test comprises of three main parts
[MethodName_StateUnderTest_ExpectedBehavior]
Public void
Sum_simpleValues_Calculated ()
Test name should include the expected input or state and
the expected result for that input or state
Public int Sum(params int[]
values), Public int
Sum_NumberIsIgnored()
Test name should be presented as a statement or fact of
life that expresses workflows and outputs
Public void
SumNegativeNumber2()
Test Name should only begin with Test if it is required by
the testing framework or if it eases development and
maintenance of the unit tests in some way.
testCalculator()
Test name should include name of tested method or class.
[MethodName_StateUnderTest_ExpectedBehavior]
Public void
Sum_NegativeNumAs1stParam_E
xcepThrown()
Variable names should express the expected input and
state
BAD_DATA or EMPTY_ARRAY or
NON_INITIALIZED_PERSON
34. Naming standards for unit tests- Cont……
Target / Subject to refer to the piece of functionality that is testing.
Fixture Synonymous with "TestFixture", a fixture is a class that contains a set of
related tests.
Suite Test Suites are an older style of organizing tests. They're specialized fixtures
that programmatically define which Fixtures or Tests to run.
Test Methods within the Fixture that are decorated with the [Test] attribute and
contain code that validates the functionality of our target.
SetUP Test Fixtures can designate a special piece of code to run before every Test
within that Fixture. That method is decorated with the [Setup] attribute.
TearDown a method with the [TearDown] attribute is called at the end of every test
within a fixture.
Fixture Setup Similar to constructors.
Fixture
TearDown
Similar to finalizers
Category : The [Category] attribute when applied to a method associates the Test
within a user-defined category.
Ignore: Tests with the [Ignore] attribute are skipped over when the Tests are run.
Explicit: Tests with the [Explicit] attribute won't run unless you manually run them.
35. Naming standards for unit tests- Cont……
CONSIDER: Separating your Tests
from your Production Code.
If you have a requirement where you want to test in
production or verify at the client's side, you can
accomplish this simply by bundling the test library with
your release.
CONSIDER: Deriving common Fixtures
from a base Fixture
In scenarios where you are testing sets of common
classes or when tests share a great deal of duplication,
consider creating a base TestFixture that your Fixtures
can inherit.
CONSIDER: Using Categories instead
of Suites or Specialized Tests
For example, you could execute only
"Stateful" tests against an environment to
validate a database deployment.
Suites represent significant developer overhead and
maintenance. Categories offer a unique advantage in the
UI and at the command-line that allows you to specify
which categories should be included or excluded from
execution.
CONSIDER: Splitting Test Libraries
into Multiple Assemblies
(You can always go back)
AVOID: Empty Setup methods you should only write the methods that you need today.
Adding methods for future purposes only adds visual
noise for maintenance purposes
36. Naming standards for unit tests- Cont……
DO: Name Tests after Functionality
For example, a test with the name
CanDetermineAuthenticatedState provides more
direction about how authentication states are
examined than Login.
The test name should match a specific unit of
functionality for the target type being tested.
Some key questions you may want to ask
yourself: "what is the responsibility of this class?"
"What does this class need to do?" Think in
terms of action words.
DO: Document your Tests
A few comments here and there are often just the
right amount to help the next person understand
what you need to test and how your test
approaches demonstrates that functionality.
Most tests require special knowledge about the
functionality your testing, so a little
documentation to explain what the test is doing is
helpful.
CONSIDER: Use "Cannot" Prefix for Expected
Exceptions
Examples: CannotAcceptNullArguments,
CannotRetrieveInvalidRecord.
Since Exceptions are typically thrown when your
application is a performing something it wasn't
designed to do, prefix "Cannot" to tests that are
decorated with the [ExpectedException] attribute.
CONSIDER: Using prefixes for Different
Scenarios
Some have adopted a For<Scenario> syntax
(CanGetPreferencesForAnonymousUser). Other
have adopted an underscore prefix _<Scenario>
(AnonymousUser_CanGetPreferences).
If your application has features that differ slightly
for application roles, it's likely that your test
names will overlap.
37. Naming standards for unit tests- Cont……
AVOID: Ignore Attributes with no
explanation
Tests that are marked with the Ignore attribute
should include a reason for why this test has been
disabled.
AVOID: Naming Tests after
Implementation
If you changed your method name, would
the test name still make sense?
If you find that your tests are named after the
methods within your classes, that's a code smell
that you're testing your implementation instead of
your functionality.
AVOID: Using underscores as word-
separators
use_underscores_as_word_separators_fo
r_readability,
PascalCase should suffice. Imagine all the time you
save not holding down the shift key.
AVOID: Unclear Test Names
it's important to avoid giving them vague
test names that represent a some external
requirement like FixForBug133 or
TestCase21.
Sometimes we create tests for bugs that are caught
late in the development cycle, or tests to
demonstrate requirements based on lengthy
requirements documentation. As these are usually
pretty important tests (especially for bugs that creep
back in).
38. Naming standards for unit tests- Cont……
Finished with Unit Testing
Categories
DO: Limit the number of Categories
Using Categories is a powerful way to
dynamically separate your tests at runtime,
however their effectiveness is diminished when
developers are unsure which Category to use.
CONSIDER: Defining Custom Category
Attributes
As Categories are sensitive to case and spelling,
you might want to consider creating your own
Category attributes by deriving from
CategoryAttribute.
39. Summary
“Unit Testing” is the most important one. Detecting and fixing bugs early
on in the Software Lifecycle helps reduce costly fixes later on. The Above
article explains how Unit Testing should be done and the important points that
should be considered when doing Unit Testing.
Hinweis der Redaktion
Code refactoring is the process of changing a computer program&apos;s source code without modifying its external functional behavior in order to improve some of the nonfunctional attributes of the software. Advantages include improved code readability and reduced complexity to improve the maintainability of the source code, as well as a more expressive internal architecture or object model to improve extensibility.
Unit Testing is conducted by the Developer during code development process to ensure that proper functionality and code coverage have been achieved by each developer both during coding and in preparation for acceptance into iterations testing.
If you do all the stuff that you know you’re supposed to anyway (loose coupling, high cohesion, etc.), writing tests is really easy. :-)
Seriously: Loosely coupled and highly cohesive software is much easier to test than tightly coupled systems, or ones where it’s hard to tell what something is really for (low cohesion).
There’s a lot of Patterns for making that easier, like Inversion of Control, Strategies, etc.
Writing the test before you write the code guarantees that your code is “testable.”
With very few exceptions, code that is hard to test is a HUGE flag that the design is bad (i.e., tightly coupled, low cohesion).
If you have dependencies between tests, or have to do tons of setup, then it’s a pain to write tests and to run them.
From past experience, projects go to lengths to separate tests from code but don&apos;t place a lot of emphasis on how to structure Test assemblies. Often, a single Test library is created, which is suitable for most projects. However, for large scale projects that can have hundreds of tests this approach can get difficult to manage. I&apos;m not suggesting that you should religiously enforce test structure, but there may be logical motivators to divide your test assemblies into smaller units, such as grouping tests with third-party dependencies or as an alternative for using Categories. Again, separate when needed, and use your gut to tell you when you should.
1. Eventually, you&apos;ll want to circle back on these tests and either fix them or alter them so that they can be used. But without an explaination, the next person will have to do a lot of investigative work to figure out that reason. In my experience, most tests with the Ignore attribute are never fixed.