Item cloning is the art and science of altering or reshaping test items and still have them represent similar measurement properties of the original items. The intent? To render the items unrecognizable to would-be cheaters and prevent them from using previously memorized test content to successfully complete an exam. Item cloning is not as easy as it sounds. It takes special skills to create cloned test items that fool even the most devious of test cheaters. You will learn: • Why testing organizations clone items • How cloning is used as part of item development and test security • Strategies for item cloning, including how to clone items and which items are best to clone. Caveon is excited to welcome guest presenter Lisa Hirtzinger, CIA, CRMA, CCSA, MBA, M.A. for this webinar. Ms. Hirtzinger is the Director, Exam Development and Analysis for The Institute of Internal Auditors (IIA). She manages exam development, translation and publication for The IIA’s flagship exam as well as four specialty designations. Ms. Hirtzinger will be joined by Caveon’s Patrick Martin, Assessment Project Manager, Test Development Services. Mr. Martin has helpful tips on which items to clone, and how to do so successfully.

1. Upcoming Caveon Events
• Caveon Webinar Series: Next session, September 18
Improving Testing with Key Strength Analysis
• EATP – September 25-27 in St. Julian’s, Malta.
– Caveon’s John Fremer and David Foster presenting the session:
What are we Accountable For? Security Standards and Resources for High
Stakes Testing Programs
• Handbook of Test Security – Now Available. We will share a discount code
at the end of this session.

2. Caveon Online
• Caveon Security Insights Blog
– http://www.caveon.com/blog/
• twitter
– Follow @Caveon
• LinkedIn
– Caveon Company Page
– “Caveon Test Security” Group
• Please contribute!
• Facebook
– Will you be our “friend?”
– “Like” us!
www.caveon.com

3. I Think I’m A Clone Now.
Item Cloning Strategies for Secure Testing
Lisa Hirtzinger, CIA, CRMA, CCSA, MBA, M.A
Director, Exam Development and Analysis
The Institute of Internal Auditors (IIA)
August 21, 2013
Caveon Webinar Series:
Patrick Martin
Assessment Project Manager
Test Development Services
Caveon

4. Our Story: The IIA
(Institute of Internal Auditors)
• A flagship examination: Certified Internal Auditor (CIA)
• Three-part multiple choice exam
• Computer based testing offered year round
• 105 countries; 17 languages
• Approximately 110,000 certified over 40 years
• Also four, one-part specialty exam (in fewer languages)

5. IIA Timeline
• In 2008 The IIA transitioned our exams to CBT
• In 2011 we renewed our focus on exam maintenance
post CBT transition
• In 2012 we researched cloning as a strategy
• In 2013 our efforts are focus on item writer training

6. Why Cloning Makes Cent$
The decision to try cloning was based on:
• Cost of Development
• Content Security
• Exam Relevancy and Validity
• Pre-Testing Strategies
• Leveraging Technology
• Use of Staff / Resources

7. The Process
• Partnered with Caveon professional item
writers
• Shared our core reference materials and
our objectives for cloning/refurbishing
• Provided the specific items we wanted
cloned
• Had Caveon writer attend in person review
with SMEs for comments and feedback
• Going forward assigned items to
professional writer; and had virtual SME
review/approval

8. Tips for Buy In
• Focus on augmenting the work of
your current valuable SMEs
• Allows focus on other
strategic priorities
• Host a working session with a
professional item writer and your
SMEs
• Share writing tips from the
professional writer with SMEs
• Ensure two-way communication

9. The Results
• Our SMEs found the process extremely
valuable and learned from it
• We increased our inventory of items for
le$$ than new development
• This changed my personal bias that
only SMEs from the profession can
write items

10. The Resulting Test Items
• Items have higher level of consistency and
clarity for exam candidates
• Enhances the overall professionalism
of the exam
• SMEs are great SMEs; but rarely
excellent technical writers
• In 2013 we are also partnering with
Caveon for additional item writer
training so we have an even better
pipeline of items to clone
• The overall objective is an item writing
maturity model (from trash to treasure
to treasure x 2)

11. Cloning Strategies

12. New Correct Answer
A trainer notices that during a heavy
barbell squat, the client’s upper back rolls
over as the client comes out of the bottom
of the movement.
Which verbal cue should the trainer deliver
to help correct the problem?
A: “Chest Up!”
B: “Eyes Up!”
C: “Shoulders Up!”
D: “Head Up!”

13. New Correct Answer
A trainer notices that during a heavy
barbell squat, the client’s upper back
rolls over as the client comes out of the
bottom of the movement.
Which verbal cue should the trainer
deliver to help correct the problem?
A: “Chest Up!”
B: “Eyes Up!”
C: “Shoulders Up!”
D: “Head Up!”
A trainer notices that during a heavy
goblet squat, the client tends to get
momentarily stuck at the bottom of
the movement.
Which verbal cue should the trainer
deliver to help correct the problem?
A: “Chest Up!”
B: “Eyes Up!”
C: “Shoulders Up!”
D: “Head Up!”

14. New Correct Answer
A woodworker is building an adjustable-
shelf bookcase for a client who has
requested that the frame be assembled
with miter joints and that no screw holes or
nail holes be visible when the cabinet is
finished.
Which joinery method should the
woodworker use to assemble the cabinet
as requested?
A: biscuits
B: pocket screws
C: dadoes
D: dovetails

15. New Correct Answer
A woodworker is building an adjustable-
shelf bookcase for a client who has
requested that the frame be assembled
with miter joints and that no screw holes or
nail holes be visible when the cabinet is
finished.
Which joinery method should the
woodworker use to assemble the cabinet
as requested?
A: biscuits
B: pocket screws
C: dadoes
D: dovetails
A woodworker is building an adjustable-
shelf bookcase for a client who has
requested that the frame be assembled
with simple butt joints and that no screw
holes or nail holes be visible when the
cabinet is finished.
Which joinery method should the
woodworker use to assemble the cabinet
as requested?
A: dowels
B: brads
C: tenons
D: rabbets

16. Flipping the Answer into the Stem
An analyst is evaluating a stock
for possible inclusion in a portfolio
designed to provide stable
income.
Which of the following metrics
should the analyst ascertain in
order to ensure that the stock will
meet the requirement of providing
stable income?
A: payout ratio
B: price to earnings
C: free cash flow
D: book value

17. Flipping the Answer into the Stem
An analyst is evaluating a stock for
possible inclusion in a portfolio
designed to provide stable income.
Which of the following metrics should
the analyst ascertain in order to
ensure that the stock will meet the
requirement of providing stable
income?
A: payout ratio
B: price to earnings
C: free cash flow
D: book value
An analyst is evaluating a stock for
possible inclusion in a portfolio
designed to provide stable income.
For what reason should the analyst
determine the payout ratio?
A: It is an indicator of the company’s
ability to maintain the dividend
payment.
B: It is a measure of the expected return
over a 5 to 10 year time period.
C: It is a gauge for determining whether
the stock is currently over or
undervalued.
D: It is a benchmark for measuring the
performance of the stock going forward.

18. Changing the Shape of the Stem
Which two statements are true
about brewing green tea?
(Choose two.)
A: It should be brewed at a
maximum temperature of
175°
B: It should be brewed at a
maximum temperature of
195°
C: It should be allowed to
steep for 1-3 minutes.
D: It should be allowed to
steep for 3-5 minutes.

19. Changing the Shape of the Item
Which two statements are true
about brewing green tea?
(Choose two.)
A: It should be brewed at a
maximum temperature of 175°
B: It should be brewed at a
maximum temperature of 195°
C: It should be allowed to steep
for 1-3 minutes.
D: It should be allowed to steep
for 3-5 minutes.
Simple tea uses electric kettles to
brew a variety of teas to order. A
customer has requested a large
plain green tea.
Which brewing instructions should
the barista follow?
A: Steep for 1-3 minutes at a
temperature of 175°
B: Steep for 3-5 minutes at a
temperature of 195.
C: Steep for 6-8 minutes at a
temperature of 205°
D: Steep for 8-10 minutes at a
temperature of 215°

20. Rewording or Keyword Swapping
Read the following passage
The film was criticized for its dull writing,
wooden acting, and for being too long.
Which revision corrects the error in the
sentence without changing the original
meaning?
A. The film was criticized for its dull
writing, wooden acting, and
excessive length.
B. The film was criticized for its dull
writing, wooden acting and for being
too long.
C. Having dull writing, wooden acting,
and being too long, the film was
criticized.
D. Audiences criticized the film for its
dull writing, wooden acting, and for
being too long.

21. Rewording or Keyword Swapping
Read the following passage.
The film was criticized for its dull writing,
wooden acting, and for being too long.
Which revision corrects the error in the
sentence without changing the original
meaning?
A. The film was criticized for its dull
writing, wooden acting, and
excessive length.
B. The film was criticized for its dull writing,
wooden acting and for being too long.
C. Having dull writing, wooden acting, and
being too long, the film was criticized.
D. Audiences criticized the film for its dull
writing, wooden acting, and for being
too long.
Read the following passage.
The employee was reprimanded for missing
deadlines, ignoring emails, and late arrivals.
Which revision corrects the error in the
sentence without changing the original
meaning?
A. The employee was reprimanded for
missing deadlines, ignoring emails,
and arriving late.
B. The employee was reprimanded for
missing deadlines, ignoring emails and
late arrivals.
C. After missing deadlines, ignoring emails,
and late arrivals, the employee was
reprimanded.
D. Management reprimanded the employee
for missing deadlines, ignoring emails,
and late arrivals.

22. Tips for Getting Stronger Clones from non-SMEs
• Give items with the potential for new correct answers
the priority.
• If applicable, provide a reference with each item.
• Require item writers to provide rationale statements
for every option.

23. Example Rationale
An analyst is evaluating a consumer
goods stock for possible inclusion in a
portfolio designed to provide stable
income.
Which of the following metrics should
the analyst ascertain in order to
ensure that the stock will meet the
requirement of providing stable
income?
A: payout ratio
B: price to earnings
C: free cash flow
D: book value
Answer Rationale
A: This answer is 100% correct because the
payout ratio measures the amount of earnings
paid to shareholders as dividends. If the ratio
is too high, the dividend may be unsustainable.
B: This answer is 100% incorrect because the
price to earnings ratio measures how
expensive a stock is relative to its earnings
and has no bearing on dividend payments.
C: This answer is 100% incorrect because free
cash flow measures the cash available after
capital expenses. In some cases analysts will
look at the FCF payout ratio to evaluate the
dividend stability of a utility company, but alone
it doesn’t provide enough information.
D: This answer is 100% incorrect because
book value is the total assets minus total
liabilities, which is more useful when
evaluating whether the stock is expensive.

24. HANDBOOK OF TEST SECURITY
• Editors - James Wollack & John Fremer
• Published March 2013
• Preventing, Detecting, and Investigating Cheating
• Testing in Many Domains
– Certification/Licensure
– Clinical
– Educational
– Industrial/Organizational
• Don’t forget to order your copy at www.routledge.com
– http://bit.ly/HandbookTS (Case Sensitive)
– Save 20% - Enter discount code: HYJ82

25. THANK YOU!
- Follow Caveon on twitter @caveon
- Check out our blog…www.caveon.com/blog
- LinkedIn Group – “Caveon Test Security”
Lisa Hirtzinger, CIA, CRMA, CCSA, MBA, M.A
Director, Exam Development and Analysis
The Institute of Internal Auditors (IIA)
lisa.hirtzinger@theiia.org
Patrick Martin
Assessment Project Manager
Test Development Services
Caveon
patrick.martin@caveon.com