Weitere ähnliche Inhalte Ähnlich wie Data Strategies in Testing (20) Kürzlich hochgeladen (20) Data Strategies in Testing5. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Define Terms
Data Strategy - the combination of code,
procedure and infrastructure that affect how
tests interact with data to stimulate the
System Under Test (SUT).
6. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Define Terms
Signed Data - data which has been
modified so that it is easily recognizable.
For example, an account with a user name
like “TestUser” or an email address like
“automated@testing.com”
7. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Define Terms
Constraint - a characteristic that applies a
force in a way that is limiting to an
automated test or its environment.
10. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Examples of Constraints
Shared Environment
– Cost
– Policy
– Infrastructure
Downtime
– Policy
– Cost
– People
Technical Skills
– Cost
– People
11. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Examples of Constraints
Data Source Growth
– Cost
– Policy
– Infrastructure
Repeatable Tests
– Policy
– Test Expectations
– People
– Infrastructure
Data Variance
– Policy
13. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Strategies Have 2 Parts:
Creational Strategy - how and when data
is created.
Cleanup Strategy - the method by which
a data source is returned to a benign
state.
14. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Elementary Approach
Creation - Tests assume specific data in
SUT - no creational strategy
Clean up - No clean-up strategy, no
management of data
16. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Elementary Approach
Pros:
Simple
No implementation
(quick)
Cons:
Tests are not
repeatable
No isolation of data
Does not work on
different systems
17. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Refresh Data Source Approach
Creation - Tests assume specific data in
SUT
Cleanup - Data managed by refreshing the
data source prior to test execution
19. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Refresh Data Source Approach
Pros:
Clean test bed (data-
wise) per test
Uniform, one-size-fits-
all process
No variance of data
Less time/cost to
implement
Total control of data
Cons:
Timely
Requires Downtime
Blocks shared
environments
No variance of data
Stale data over time
Doesn’t work on
different systems
20. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Selfish Data Generation
Creation - Tests create the data they need
in SUT
Clean up - No management of data after
generation
May be random (e.g. via Faker)
Create with batch process or in test
Create via SUT or directly in Data Source
Create In-test or pre-test
22. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Selfish Data Generation
Pros:
Unique data per test
Does not block shared
environments
No downtime
No stale data
Tests can run anywhere
Tests become
repeatable
Cons:
Unchecked growth of
data source
Requires understanding
of data model
Tests need to know
about data
23. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Batch Cleanup
Creation - Data generated for each test
Cleanup - Data cleaned up after test run
Clean up may happen in data source or
via SUT
24. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Batch Cleanup
SUT
Execution 1
Execution 2
Execution 3
Data
Source
Generate Data
Generate Data
Generate Data
Batch Clean up
25. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Batch Cleanup
Pros:
Unique data per test
Does not block shared
environments
Minimal downtime
No stale data
Growth of data source
kept in check
Tests run anywhere
Tests are repeatable
Cons:
Requires deep
understanding of data
model
Must know what to clean
Significant scaffolding
must be created
26. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Immediate Cleanup
Creation - tests generate their own data
Clean up - for each test case immediately
after execution
Uses a TearDown method supplied by the
framework
27. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Immediate Cleanup
SUT
Execution 1
Execution 2
Execution 3
Data
Source
Generate Data
Generate Data
Generate Data
Clean up
Clean up
Clean up
28. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Data Generation & Immediate Cleanup
Pros:
Tests are repeatable
Unique data per test
Does not block shared
environments
Minimal downtime
No stale data
Growth of data source kept in
check
No data left over from other
tests
Tests run anywhere
Cons:
Requires deep
understanding of data model
Must know what to clean
Significant scaffolding must
be created
29. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Creational Strategies
Elementary Refreshed Data
Source
Data Generation
Shared
Environment
Low Downtime
Repeatable
Tests
Data Variance
Minimal Data
Growth
Significant
Technical Skills
30. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Cleanup Strategies
Elementary
(none)
Refresh
Data
Source
Batch Immediate (Signed,
Marked or Persisted)
Shared
Environment
Low
Downtime
Repeatable
Tests
Strong
Technical
Skills
32. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Signed Data
Record Name: “test_21323”
Field Name: “Record Name”
Format: “test_<random5digs>”
Persist in:
– a data source
– code
33. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Marked Data
An “isTestData” field or
Data that is otherwise known to be test
data because of its format or type, for
example:
– A “test user” created it
– Linked to known test denotation
35. www.beaufortfairmont.com 984.244.2313© 2016 Beaufort Fairmont, LLC All rights reserved
Externally Persisted Data
Keep a record in a data source:
– userIds:
• 12312
• 4322
• 23423
– emailIds:
• 12312
• 423423
• 234