This document summarizes a presentation on quality metrics and culture given at Habib University in Karachi. It discusses key quality metrics like lot acceptance rate, product complaint rate, and CAPA effectiveness. It also identifies attributes of a strong quality culture like leadership emphasis, message credibility, peer involvement, and employee ownership. Finally, it stresses that quality is everyone's responsibility and a mature quality system relies on systems thinking and a culture that prioritizes quality and continuous improvement.

1. Saturday, October the 10, 2015
Quality Metrics & Science
Habib University, Karachi
Obaid Ali & Roohi B. Obaid
Civil Service Officers at Drugs Regulatory Authority of Pakistan

2. 1 Math, Statistics and Computer in Quality Science
2Cultural Scores and Site Performance for key Quality Metrics
5 Quality is everyone's responsibility
6Closing words - Simple, clear and loud
3 Essential Attributes for Culture of Quality
4Roots and Principle of Quality Culture

3. Where everyone willingly takes
responsibility for patient protection and
quality of product
Where patient is centre of everything

4. Econometrics
Pharmacometrics
Chemometrics
Science
Rational

8. Data Integrity – Deviant Behaviors
Not
recording
activities
contempor
aneously
Backdating
Fabricating
data
Copying
existing
data as
new data
Re-running
samples
Discarding
data
Alicia M. Mozzachio, July 2014, FDLI, Washington DC

10. Trends help indeed to understand and predict
Lets have a visit to some slides of US-FDA

11. Time 
ProcessParameter
Lower Specification Limit (LSL)
Upper Specification Limit (USL)
today
Uncertainty
RISK: For a given severity of risk event, what are the chances (probability) of exceeding the USL in the next period of time?
G. Claycamp, FDA, Sept. 2005
Tomorrow ?

12. ProcessParameter
Lower Specification Limit (LSL)
Upper Specification Limit (USL)
Uncertainty
Tomorrow ?
G. Claycamp, FDA, Sept. 2005
RISK: Control options are scenarios for risk management. Note that this scenario shows the best estimate is below the USL.
Time  today

13. Time 
ProcessParameter
Lower Specification Limit (LSL)
Upper Specification Limit (USL)
today
Uncertainty
Take a cut at a moment in time:
Risk has a distribution.
Tomorrow ?
G. Claycamp, FDA, Sept. 2005

14. Quality
Metrics
Indicator of
operational reliability Quality Culture

15. Quality
Metrics
Lot
Acceptanc
e Rate
Product
Quality
Complaint
Rate
Right first
time Rate
Invalidate
d OOS
Rate
APR or
PQR on
time Rate
CAPA
Rate
Ability to
initiate
voluntary
CAPA

16. Lots rejected/total lots in a year
Lot Acceptance Rate

17. Lot Acceptance Rate
4/355=

18. Deviated lots/ total lots
in a year
Right First Time Rate

19. 38/355=
Right First Time Rate

20. APR triggered CAPA/ total
number of APRs
Corrective & Preventive Action (CAPA) Rate

21. Corrective & Preventive Action (CAPA) Rate
3/30=

22. Process capability or
performance index for each
critical quality attribute that
triggers a CAPA
Ability to initiate voluntary CAPA

23. Ability to initiate voluntary CAPA
Cp value

24. The number of batches for which
complaint received/ total number of
batches (very specific to the product)
Product Quality Complaint Rate

25. Product Quality Complaint Rate
6/30=

26. Invalidated initial OOS tests /
total number of tests in a year
Invalidated OOS Rate

27. Invalidated OOS Rate
200/9900=

28. The number of APRS within
30 days of annual due date /
the total number of products
produced during the year
APR or PQR on time Rate

29. APR or PQR on time Rate
05/40=

30. Cultural Scores & Site Performance Key Quality Metrics
Pharmaceutical OnlineKen Congdon, 26 May 2015
Stronger culture is correlated with
fewer action limit excursions in steriles
1/4 ISPE

31. Cultural Scores & Site Performance Key Quality Metrics
Pharmaceutical OnlineKen Congdon, 26 May 2015
Stronger culture scores are associated
with higher lot acceptance
2/4 ISPE

32. Cultural Scores & Site Performance Key Quality Metrics
Pharmaceutical OnlineKen Congdon, 26 May 2015
Sites with higher quality culture scores
have lower deviations recurrence
3/4 ISPE

33. Cultural Scores & Site Performance Key Quality Metrics
Pharmaceutical OnlineKen Congdon, 26 May 2015
Sites with higher quality culture scores
have higher CAPA effectiveness
4/4 ISPE

34. Identified Essential Attributes for Culture of Quality
Harvard Business ReviewSurvey April 2014
Leadership Emphasis
Message Credibility
Peer Involvement
Employee Ownership

35. Identified Essential Attributes for Culture of Quality
Harvard Business ReviewSurvey April 2014
Maintaining a leadership emphasis on quality
What they say & What they do
1/4

36. Even when executives have the best intentions, there are
often gaps b/w
What they say & What they do
As a result, employees get mixed messages about
whether quality is truly important. Company leaders must
first buy into quality improvement initiatives and clearly
demonstrate their own personal commitment to this effort
to employees

37. Identified Essential Attributes for Culture of Quality
Harvard Business ReviewSurvey April 2014
Ensuring Message Credibility
One Message
2/4

38. Quality messaging should be tailored in such a
way that it resonates with every employee.
Different messages may need to be developed to
resonate appropriately with employees at
different sites.
The key is to regularly test these messages with
employees, solicit their feedback, and refresh
these messages over time if necessary

39. Identified Essential Attributes for Culture of Quality
Harvard Business ReviewSurvey April 2014
Encouraging Peer Involvement
Participation
3/4

40. A variety of different tactics can be used to
create positive social pressure that encourages
employees to participate in (& even generate)
quality initiatives. These can include friendly
“quality competitions”.
The key is to test these tactics and solicit
feedback to see which are most effective with
employees.

41. Identified Essential Attributes for Culture of Quality
Harvard Business ReviewSurvey April 2014
Increasing Ownership & Empowerment
Engage in Decisions
4/4

42. Employees should be free to apply their own
judgment to situations that fall outside the
common directive. Providing the right level of
guidance is crucial to this effort
but they must know how to act when they
encounter specific quality issues.

43. Visible Signals of Quality Culture
Right tone at the top
Focus on the right
value
Constant display of
putting patient
above everything
Constant display of
concern for quality
Consistently
walks the talk
With great
people skill
& strong
value
More than
financial
rewards

44. Roots of Strong Quality Culture
Introduction of Improvement & Innovation for quality driven
productivity
Adaptation & Investment in Technology to ensure state of control
over system
Promotion of change & continuous improvement in Quality
System

45. Roots of Strong Quality Culture
Prompt adaptation of best practice, low staff rotation
Proactive & transparent behavior
Strong relationship based on dialogue, science & mutual respect

46. Principle of Quality Culture
Product & Process Understanding Sensible Business Management
Sufficient depth of science to
explain why deviations have
not impacted on quality of
product
Too fast growth & operating at
limit of capacity triggers often
issues & non-compliance

47. How mature is your quality system
1 2 3 4
Level
Richard L. Friedman, FDLI Workshop Washington DC, July 2014US-FDA
Small problems ultimately snowball into larger ones, and
management becomes aware only when there is a crisis

48. How mature is your quality system
1 2 3 4
Level
Richard L. Friedman, FDLI Workshop Washington DC, July 2014US-FDA
Nearly always reactive, but there is willingness to change.
Patchwork corrections are the norm

49. How mature is your quality system
1 2 3 4
Level
Richard L. Friedman, FDLI Workshop Washington DC, July 2014US-FDA
More proactive. Increasingly detects emerging adverse trends,
surfaces major issues, and makes lasting manufacturing and
system improvements

50. How mature is your quality system
1 2 3 4
Level
Richard L. Friedman, FDLI Workshop Washington DC, July 2014US-FDA
Routinely acts preventively as described in level 3. Fully
institutionalizes and reinforces (rewards) a vigilant culture that
makes meaningful manufacturing and systemic improvements

51. Systems Thinking
System is the product of interacting parts
Improving the parts taken separately will
not improve the system

52. Quality is everyone’s responsibility
Selected Quotes, W. Edwards Deming
Lack of
knowledge …
that is the
problem
If you do not
know how to ask
the right
question, you
discover nothing
If you can’t describe
what you are doing as
a process, you don’t
know what you are
doing
Rational behavior
requires theory.
Reactive behavior
requires only reflex
action
Whenever there
is fear, you will
get wrong
figures
Learning is not compulsory … neither is survival

53. Strong Quality Culture Efficient Quality System Consistent manufacturing
It’s the culture that decides the path
Poor Quality Culture Unreliable Quality System Defects and Recall

55. Soft gelatin capsule Paracetamol tablet
Pyrimethamine tragedy Metronidazole label

56. Disintegration
failure
Dissolution
failure
Materialmix
up
LabelMixup

57. One of the important
Hormonal Product

58. Contract
manufacturing
Changed in
color
Stability was
not done

59. Quality Agreement

60. Batch of 1
million tablets
22 batches Underweight
Caught after 2
weeks
System ???
or culture ???

61. Antibiotic &
steroid
cream
Analytical
method
unable to
quantify
Manipulated
method used
for years
Ended up
with
withdrawal
of product
from market

62. Lets Look Around …

63. 09 Oct, 2015, Tab Paracetamol - USA

64. Tablet Paracetamol
Wrongly labeled
325 mg instead of 500 mg
175 mg additional than claimed

65. What do you think what
should be the action?
End up with recall

66. Aug 2015
Puerto Rico &
Taiwan
Wood
fragments
found in
toothpaste
3.9 M tubes
recalled

67. Attix Canada April 2015

68. Attix Canada April 2015

69. Health Canada ….
FDA closed an API
repackager in Toronto
recall hundreds of
bulk APIs … cross-
contaminated by
penicillin
failed to use separate
facilities, or
equipment like hoods
and air handlers
cross-contamination
… employees move
freely b/w the
different areas
Attix stance: We have
carefully cleaned
everything before use
Regulatory Stance:
Cleaning is not a
substitute for
segregating products
Cross-contamination
with sensitizing agents
can initiate life-
threatening allergic
reactions
No safe level of
penicillin
contamination has
been determined to be
a tolerable risk
Attix Canada April 2015

70. Philippines-
Aug 2014
GMP non-
compliance in
Italy
Tetrahydrozoli
ne HCl RM
Used in
manufacturing
of ophthalmic
solution in
China
Phillippines
instructed
discontinuatio
n of
distribution,
sale & use
Many lots
recalled

71. Do you calculate a
cost of poor quality
at your site?
ISPE/PDA Survey Sep 2011

72. Have you evaluated the
cost of improving quality
vs. the cost of failure
(recalls, rejections, low
yield, downtime etc)?
ISPE/PDA Survey Sep 2011

73. Lets make it simple, clear & loud

74. Me, my kids &
parents use same
drug which I
manufacture
No batch has ever
failed

75. Drug manufactured in non-cGMP
environment
It does not mean that there is necessarily
something wrong with the drug
But “intention in question”

76. Please remember
Lack of adverse event reports, or
“something is wrong” is NOT the same
as evidence of no harm….

77. Closing words

78. Innocence
• Quality
mainly
outsourced
to regulators
Awareness
• Quality unit
will identify
the issue
Understanding
• Quality
system &
Metrics
reveals
reality &
drives action
Competence
• Quality
mind set in
all functions.
Design
quality in &
anticipate
issues
Excellence
• Competitive
advantage
through
prevention
of quality
issues
Journey of uncertainty to certainty matters

79. Linkage
Science &
compliance
Challenge
Traditional
validation &
continuous
verification
Maintain
Process
robustness &
drives
reliability
Product & Process
Criticality
Culture of Quality
beyond complianceNew technology

80. Even a great quality system will fail, if the
Quality Culture
is not equal standard

81. The question isn’t whether we will have standards …
The question is how useful can we make standards