Material Remains as Source of Ancient Indian History & Culture.ppt
Rutgers Green Belt
1. Certified Six Sigma Green Belt
Course
Brandon Theiss
Brandon.Theiss@gmail.com
2. Motivation
• Teaching the tools, techniques and Methods of Lean Six
Sigma is inherently difficult in academic setting.
• When taught in a industrial setting students have a
common motivation (the improved welfare of the
company), similar levels of education and knowledge of
domain specific information. Student are encouraged to
learn by applying the material to their daily activities.
• This is not possible in an academic setting particularly in a
mixed environment that includes everything from
undergraduate juniors through senior PhD researchers.
• In addition undergraduate students tend either lack
professional or have experience in Fields that are not
traditionally thought of as benefiting or implementing Six
Sigma (waitressing, check out clerk etc.)
3. Solution
• The beauty of the Six Sigma Methodology is that it can be applied
to any process.
• The definition of a process is quite broad and can be reduced to any
verb- noun combination.
• Therefore the collective process which the class studied and
improved was to
Pass [the]
ASQ
Certified Six Sigma Green
Belt Exam
• Therefore the foundational Six Sigma Concept of DMAIC (Define
Measure Analyze Improve Control) represents both the material
covered in the course as well as the pedagogical method used for
instruction
4. About the Course & Partnership
• Offered as a Non-Credit extracurricular course
at Rutgers University in Piscataway NJ
• Co-Sponsored by the Rutgers Student Chapter
of the Institute for Industrial Engineers (IIE) and
the Princeton NJ section of American Society for
Quality (ASQ)
• Open and advertised to all members of the
Rutgers Community (students, staff and faculty)
as well as the surrounding public
• Objective of the course was to train students to
pass the June 2nd 2012 administration of the
ASQ Certified Six Sigma Green Belt Exam
5. Class Demographics
• 71 Students Registered
– 57 At Student Tuition Rate ($296)
– 14 At Professional Tuition Rate ($495)
Histogram of Years Of Work Exprience
Highest Accademic Grade 3
Completed 20
40.0%
35.0% 15
30.0%
Frequency
25.0%
10
20.0%
15.0%
10.0% 5
5.0%
0.0%
0
Junior Year Senior BA/BS Some MA/MS/JD PhD/PE 2 4 6 8 10 12 14 16 18 20 22 24
Year Grdudate Years Of Work Exprience
7. Pre Test
• On the first night of classes students were
given an introductory survey of Six Sigma
by means of a worked example applying
DMAIC to the Starbucks Experience from a
Customers Prospective.
• Students were then given a copy of the
Certified Six Sigma Green Belt Handbook by
Roderick A. Munro
• Then given a 50 Question Multiple Choice
Test representative of the ASQ CSSGB Exam
• The Test was administered on two
successive nights (Monday and Tuesday)
8. Measurement System
• An Apperson GradeMaster™ 600 Test
Scanner was utilized which enabled test to
be scored and returned immediately upon
student submission at the exam site.
• In addition all of each answer to every
question was downloaded to connected
computer enabling further detailed analysis
10. Test Scores
Histogram of Test Scores
Normal
Mean 0.5589
9
StDev 0.1177
N 35
8
7
6
Frequency
5
4
3
2
1
0
36.00% 48.00% 60.00% 72.00% 84.00%
Test Scores
11. Test for Normality
Probability Plot of Test Score
Normal - 95% CI
99
Mean 0.5589
StDev 0.1177
95 N 35
AD 0.396
90
P-Value 0.352
80
70
Percent
60
50
40
30
20
10
5
1
0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0
Test Score
12. Is process in Control?
I Chart of Test Score
1.0
UCL=0.9468
0.9
0.8
0.7
Individual Value
0.6 _
X=0.5589
0.5
0.4
0.3
0.2
LCL=0.1709
0.1
1 4 7 10 13 16 19 22 25 28 31 34
Observation
13. Is the Process Capable?
Process Capability of Test Scores
LSL
P rocess Data Within
LS L 0.78 Overall
Target *
USL * P otential (Within) C apability
S ample M ean 0.558857 Cp *
S ample N 35 C PL -0.61
S tDev (Within) 0.120985 C PU *
S tDev (O v erall) 0.117718 C pk -0.61
O v erall C apability
Pp *
PPL -0.63
PPU *
P pk -0.63
C pm *
0.36 0.48 0.60 0.72 0.84
O bserv ed P erformance E xp. Within P erformance E xp. O v erall P erformance
P P M < LS L 971428.57 PPM < LS L 966214.72 P P M < LS L 969849.40
PPM > USL * PPM > USL * PPM > USL *
P P M Total 971428.57 PPM Total 966214.72 P P M Total 969849.40
15. Does the order the exams are turned in effect the
score?
Trend Analysis Plot for Test Score
Linear Trend Model
Yt = 0.5018 + 0.00317*t
0.9 Variable
Actual
Fits
0.8
Accuracy Measures
MAPE 15.9381
0.7 MAD 0.0840
Test Score
MSD 0.0124
0.6
0.5
0.4
0.3
3 6 9 12 15 18 21 24 27 30 33
Index
22. Does the order exams are turned in
effect test scores?
Trend Analysis Plot for Scores
Linear Trend Model
Yt = 0.5614 - 0.000138*t
0.9 Variable
Actual
Fits
0.8 Accuracy Measures
MAPE 13.9747
MAD 0.0779
0.7 MSD 0.0100
Scores
0.6
0.5
0.4
3 6 9 12 15 18 21 24 27
Index
24. Combined Test Scores
Histogram of Combined
Normal
20 Mean 0.5591
StDev 0.1099
N 64
15
Frequency
10
5
0
0.36 0.48 0.60 0.72 0.84
Combined
25. Test Scores
Histogram of Monday, Tuesday
Normal
0 0% 0 0% 0 0% 0 0% 0 0%
3 6. 4 8. 6 0. 7 2. 8 4.
Monday Tuesday Monday
9 9 Mean 0.5589
StDev 0.1177
8 8 N 35
7 7 Tuesday
Mean 0.5593
Frequency
6 6 StDev 0.1018
N 29
5 5
4 4
3 3
2 2
1 1
0 0
36 48 60 72 84
0. 0. 0. 0. 0.
26. Is there a difference Between Classes?
Boxplot of Monday, Tuesday
Monday Tuesday
0.9
0.8
0.7
0.6
0.5
0.4
0.3
27. Is there a statistical Difference?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Monday 35 19.56 0.558857 0.013857
Tuesday 29 16.22 0.55931 0.010357
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 3.26E-06 1 3.26E-06 0.000265 0.987056 3.995887
Within Groups 0.76114 62 0.012276
Total 0.761144 63
28. Brainstorming Techniques
• At the beginning of class students were asked as a group to
brainstorm ideas for why they failed the pre-test
– Only 4 ideas were proposed
• Students were taught the different brainstorming techniques
contained in the CSSGB Body of Knowledge
– Nominal Group Technique
– Multi-Voting
– Affinity Diagrams
– Force Field Analysis
– Tree Diagrams
– Cause and Effect Diagrams
• Students were then broken up into 6 different groups, assigned one
of the brainstorming techniques and given the task to brainstorm
why they failed the pre-test
33. Brainstorming Continued
• Students then were given told to return to
their groups and apply their “favorite” of the
brainstorming techniques to the task how can
you Pass the midterm exam
• Students Found the positive formulation of
the task much more challenging and most
groups stayed with the same technique they
used for the Negative version.
34. Team Dynamics
• The 3rd weeks lesson began with an
introduction of the Tuckman cycle of team
dynamics
• Students were asked
to reflect upon their
experience in the
brainstorming activity
to see if their
experiences paralleled
those predicted by the
model
35. Process Mapping
• The second portion of the 3rd Class was spent
introducing the process mapping strategies in
the CSSGB BoK
– SIPOC (Suppliers Inputs Outputs Customers)
– Process Mapping
– Value Stream
Mapping
36. Process Mapping Continued
• Students were again divided into 6 groups. Each group was assigned a map type and
told to Map the Exam Taking Process at either a Micro or Macro Level
• Micro Level Groups Handled the Physical steps of taking the exam such as reading
the question, locating the answer and filling in the bubbles
• Macro Groups Handled the all of the preparation leading up to taking the exam
• The point was to emphasize that the same tools techniques and methods can be
used on the very micro level (an operator tightening a bolt) to the very macro level
(the operations of a fortune 500 company)
37. Control Charts
• Class 4 Introduced Students to the Control Charts Covered in the
CSSGB BoK
– I-MR
– X Bar-R
– X Bar- S
– P
– NP
– U
– C
• Students were emailed prior to class a Microsoft Excel Workbook
containing the test results and told to bring their laptops to class
• Students were asked to do the following by hand (with Excel
helping for the calculations):
– I-MR Chart for Test Scores
– P Chart testing for “Bad Questions”
– NP Chart testing for “Bad Questions”
– C Chart for the number of wrong responses per exam
– U Chart for the number of wrong responses per exam
43. Does a T-Test Indicate there was improvement?
t-Test: Two-Sample Assuming Unequal Variances
Mid Pre
Mean 0.607234 0.561702
Variance 0.014373 0.01111
Observations 47 47
Hypothesized Mean Difference 0
df 91
t Stat 1.955429
P(T<=t) one-tail 0.0268
t Critical one-tail 1.661771
P(T<=t) two-tail 0.0536
t Critical two-tail 1.986377
44. Does ANOVA Indicate there was
Improvement?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Pre Total 64 35.78 0.559063 0.012082
Mid Total 53 31.72 0.598491 0.013705
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 0.045069 1 0.045069 3.516685 0.06329 3.923599
Within Groups 1.473823 115 0.012816
Total 1.518892 116
46. Is the Change in Control?
C Chart of Change in # of Correct Responses
15
10
UCL = 8.29
5
Mid= 2.28
0
LCL = -3.74
-5
-10
-15
47. Is the change in Scores Significant?
t-Test: Paired Two Sample for Means
Mid Pre
Mean 0.607234043 0.561702
Variance 0.014372618 0.01111
Observations 47 47
Pearson Correlation 0.689206844
Hypothesized Mean Difference 0
df 46
t Stat 3.475995635
P(T<=t) one-tail 0.000560995
t Critical one-tail 1.678660414
P(T<=t) two-tail 0.00112199
t Critical two-tail 2.012895599
51. Comparison of Results for Material
that has been Covered
Boxplot of Covered Scores
1.0
0.9
0.8
Covered Scores
0.7
0.6
0.5
0.4
0.3
Pre Covered Mid Covered
Subscripts
52. Comparison of Covered Material
Histogram of Pre Covered, Mid Covered
Normal
0.3 0.4 0.5 0.6 0.7 0.8 0.9
Pre Covered Mid Covered Pre Cov ered
Mean 0.5785
12 StDev 0.1252
N 64
10 Mid Cov ered
Mean 0.6516
StDev 0.1174
Frequency
8 N 53
6
4
2
0
0.3 0.4 0.5 0.6 0.7 0.8 0.9
53. Does ANOVA Indicate there was
improvement?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Pre Covered 64 37.02632 0.578536 0.015686
Mid Covered 53 34.53333 0.651572 0.013785
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 0.154648 1 0.154648 10.43065 0.001616 3.923599
Within Groups 1.70503 115 0.014826
Total 1.859678 116
54. Comparison of Results for Material
that has not been Covered
Boxplot of Scores
0.9
0.8
0.7
0.6
0.5
Scores
0.4
0.3
0.2
0.1
0.0
Pre Not Covered Mid Not Covered
Subscripts
56. Does ANOVA indicate the Exam was
harder?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Pre Not Covered 64 31.83333 0.497396 0.01785
Mid Not Covered 53 27.5 0.518868 0.024926
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 0.013367 1 0.013367 0.635003 0.427168 3.923599
Within Groups 2.420698 115 0.02105
Total 2.434065 116
58. Control Charts with Minitab
• Students were emailed a Microsoft Excel Workbook with the Mid-
Term data set
• It was heavily suggested that students purchase the Minitab
academic license and bring their laptops to class.
• Students then divided themselves into groups around those who
purchased the software and created the analysis control charts on
the preceding slides.
59. Hypothesis Testing Exercises
• In week 8 students were introduced to the hypothesis tests covered
in CSSGB BoK
– Z Test
– Student T
– Two Sample T (known variance)
– Two Sample T (unknown variance)
– Paired T Test
– ANOVA
– Chi Squared T
– F Test
• Students were emailed a data set containing both the Pre-Test and
Mid-Term data and asked to perform each of the listed test using
either Minitab or Microsoft Excel. The emphasis was placed on the
conclusions from the data
60. Confidence Intervals
• Not all students took the Mid-Term that took the
pre-test.
• This enabled students to utilize inferential
statistics to draw conclusions about the
population parameters (mean and variance
particularly)
• By using the class data set provided students
were able to calculate their confidence in the
overall population parameters for the average
test score as well as the standard deviation of the
entire class
61. Improve-Control
• Improve and Control are not an emphasis in the CSSGB BoK. For the
coverage of the material and extended example of the Starbucks
Experience from a customers perspective is presented.
• When introducing Lean and the types of Waste the process of making
various beverages are presented. Students then proposed improvement
strategies to minimize the ‘Muda’
Triple Tall Half Hot Half Cold Americano
(Future State)
Triple Tall Half Hot Half Cold Americano
(Current State)
67. Was The Difference Significant?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Pre 64 35.78 0.559063 0.012082
Mid 47 28.54 0.607234 0.014373
Final 40 30.43 0.76075 0.020084
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 1.029282 2 0.514641 34.534 4.91E-13 3.057197
Within Groups 2.205562 148 0.014902
Total 3.234844 150
69. Was the Individual Improvement
Significant?
t-Test: Paired Two Sample for Means
Final Pre
Mean 0.750556 0.556667
Variance 0.019743 0.010023
Observations 36 36
Pearson Correlation 0.342582
Hypothesized Mean Difference 0
df 35
t Stat 8.199954
P(T<=t) one-tail 5.8E-10
t Critical one-tail 1.689572
P(T<=t) two-tail 1.16E-09
t Critical two-tail 2.030108
71. Pareto Chart on Topic
Pareto Chart of Question Topic
16
100
14
12 80
10
Percent
60
Count
8
6 40
4
20
2
0 0
Question Topic s r l
at sis ro bl
ity s
va r ts EA
St he Er a am er ha FM
c ot ap Te In
t
lC
si yp C e ro
Ba H s c nt
es en Co
oc fid
Pr n
Co
Count 3 3 2 2 2 1 1 1
Percent 20.0 20.0 13.3 13.3 13.3 6.7 6.7 6.7
Cum % 20.0 40.0 53.3 66.7 80.0 86.7 93.3 100.0
74. Results
• Students test scores improved on average 19.4%
• The Passage Rate on the actual ASQ Administered
Certified Six Sigma Greenbelt Exam Far exceeded
the national average*
• 68.75% of respondents to an online survey
ranked their level of satisfaction with the course
at a 5 or higher on a 7 point scale
• Increased ASQ Princeton Membership by 62
members
75. Lessons Learned
• Using the passing the exam process as a class exam for
the implementation of the tools and techniques of Six
Sigma is an effective methodology
• There is demand for teaching Six Sigma in an academic
setting
• The joint venture between Rutgers and ASQ is feasible
and mutually beneficial.
• Having a diverse student population increases the
overall performance of the group.
• Students need to be adequately qualified to sit for ASQ
exam prior to taking the course.
Hinweis der Redaktion
Six Sigma is a problem solving tool kit that seeks to improve the quality of process outputs by identifying and removing the causes of defects (errors) and minimizing variability in manufacturing and business processes.Six Sigma Green Belts are the tactical leads on improving functions within a job function that are able to apply the Lean Sigma Concepts to their daily work.The methods are universally applicable to anything where a customer is being serviced.
This is a unique pedagogical approach and from philosophically is quite “meta”. The objective under examination is in fact the actor performing the examination.The most brilliant of teacher can write the most profound equation on a chalkboard, and the most diligent of students can take pristine notes. However learning only occurs when the student is able to apply the material. Johann Wolfgang von Goethe was correct when he said “Knowing is not enough; we must apply.”Given the diversity of the composition of the students in terms of education, life experience, income and industry finding a common task in which to apply the LSS would have been impossible. The only true commonality between the group was that they were all humans and wanted to earn their greenbelt. We were able to leverage this fact in developing the instructional roadmap for course.Also the utilization of Shewhart Control Charts which are used to differentiate between common cause and special cause variation, is fairly novel in academic settings.
The instructor for the course, Brandon Theiss, is a Senior Member of ASQ and a Graduate student at Rutgers University. Currently there is not a course offered in the undergraduate Industrial and Systems Engineering Program at Rutgers. This course provided an opportunity for students to not only be exposed to the material but also to earn a nationally recognized certification in the tools techniques and methods of Six Sigma. It represented a first of its kind partnership between the student chapter of the IIE and ASQ Princeton section. Part of the proceeds for the course were used to fund the IIE trip to their national conference in Orlando.
The cost of the course for students included the textbook and ASQ student membershipThe professional rate only included the text.The ASQ Certified Six Sigma Green Belt Requires 3 or more years of work experience in one of more areas of the Body of Knowledge. There was a very long and at times heated exchange with the ASQ certification committee about what constitutes work experience. A compromise was ultimately reached however there were still a large number of qualified students that were denied the right to sit for the exam
The course met once per week over an 11 week period from 6:30 to 9:30PM. There were two sessions per week and students were free to attend either the Monday or Tuesday class based upon which ever was more convenient for their schedule
Students were notified via email prior to the first night of the course that an exam would be administered on the first night.This provided both a baseline for the future improvement as well as showing students directly the level of mastery they would need to obtain to become certified.
Feedback in any system is critically important. With a course that only meets once per week, having students wait a week would be to long. By providing students immediate feedback they were able to best utilize their time to study as well as not mis-learn material thinking that they had been correct on a question when in fact they were not.
A simple histogram of the exam results from the Monday section with a normal distribution fit. It does appear to be normal but has a very large standard deviation 11.8%
The probability plot indicates that there is insufficient data to reject the null hypothesis that the data is normally distributed. This is indicated by the P value which indicates the probability that the difference between the measured data and the model occurred by pure chance. The null hypothesis of normality would have been rejected if the value had been less than alpha (5%) representing a 95% confidence level.
It is technically debatable if the test scores are continuous or discrete variable and if a I chart is appropriate. However the point is to introduce students to control charts and an Individuals chart.Since no point lies about the Upper or Lower Control Limit, the process is in a state of “statistical control”. However common sense shows that this is nonsensical as the range of the limits is between 17% and 95%. This was caused by the large standard deviation observed.This was used as an opportunity to discuss the difference between statistical significance and actual significance. This reinforces the concept that the math does not know where the numbers came from and can at best direct teams to derive the true underlying meaning.
Again there is a technical point if the test scores are discrete or continuous. The above Process Capability study requires that the data be considered continuous. Process capability is essentially the probability of producing a product that will meet your customers specification. In this case the passing score (78%) sets that limit. As you can see in the above chart for every 1,000,000 students from the Monday population that took the pre-test exam ~970,000 students will fail.
Everyone has taken a test where the test taker believes there was a question that either had the wrong answer or was too difficult. By using a NP (or P) control chart, one can easily distinguish if a question was statistically significantly too difficult above the UCL or too easy below the LCL
There were several students who handed in their exams very quickly. We wanted to see if the amount of time a student spent on the exam effected their scores. And for the Monday data set it appears it did.
A histogram of the Tuesday data set
Again the data is normal as indicated by a P value greater than 5%. It is however notable in the above plot that there is a clear outlier.
Again we can see that there is clearly an outlier in the data set.
The Tuesday process is very similar in its inability to produce a unit meeting customers expectations and again will generate ~970,000 failures for every million students from the population that take the exam
In the above graph it does appear that there were questions that a statistically significant number of students got wrong.
Interestingly, the order in which a student turned in their exam did not have an effect on the Tuesday data set.
Combined Histogram of the results
Both distributions look somewhat similar.
The above shows a box plot comparing the two classes. The median appears to be higher in the Tuesday class. However is the difference significant?
An ANOVA analysis was performed which results in a very high p value which means that there is not a statistically significant difference between the two population means.
Nominal Group -> when individuals over power a groupMulti-Voting -> Reduce a large list of items to a workable number quicklyAffinity Diagram -> Group solutionsForce Field Analysis -> Overcome Resistance to ChangeTree Diagram -> Breaks complex into simpleCause- Effect Diagram -> identify root causes
Most Common Model of group Development was proposed by Bruce Tuckman in 1965.In order for the team to grow, to face up to challenges, to tackle problems, to find solutions, to plan work, and to deliver results. They must go through the cycleFormingTeam members getting to know each otherTrying to please each otherMay tend to agree too much on initial discussion topicsNot much work accomplishedMembers orientation on the team goalsGroup is going through “honeymoon period”StormingVoice their ideaUnderstand project scope and responsibilitiesIdeas and understanding cause conflictNot much work gets accomplishedDisagreement slows down the teamNormingResolve own conflictsCome to mutually agreed planSome work gets doneStart to trust each otherPerformingLarge amount of work gets doneSynergy realized Competent and autonomous decisions are madeAdjourningTeam is disbanded, restructured or project re-scoped.Regression to Forming stage
Control Charts are used to differentiate between common cause (normal) and special cause (abnormal) variation.
There does not appear to be a large change between the Pre Test and the Mid Term
A T-Test indicatesthat there is significant improvement, as indicated by the one tail P value.
ANOVA on the other hand indicates that there is not a difference between the two means.
Displays a histogram of the changes in scores, about 40% of the students went down and 60% increased their score.
This is a somewhat novel adaptation of a C chart that allows for negative values. However there appear to be students that did much better and much worse than the other students.
Looking at a Paired-T test there was absolutely a statistically significant improvement.
Why did the test scores not improve more dramatically? Well the exams cover all of the material in the CSSGB BoK the course was only half complete. When we looked at the material covered up to the midterm on both the pre-test and the mid term the above pie charts show the percentage of the covered material on each exam.
Not surprisingly students performed better on the material that was covered as compared to the material that was not covered.
However the students also scored better on that same material on the pre test.
So was there actual improvement?
The change in the means indicates a ~8% improvement. However is that statistically significant?
ANOVA does indicates that there is a difference in the means. The students did in fact learn the material that was covered.
There does not appear to be a difference in the scores in the material that was not covered yet in the course.
There was a small increase in the means ~2% is that significant?
No. There is not a statistically significant difference between the pre-test and mid-term scores on the material that was not covered. As a result it would indicate that the exams were roughly the same difficulty.
The process is still incapable of generating a passing score on the test.
Minitab is the de facto industry standard for statistical process control. Unfortunately the undergraduate program at Rutgers does not include any training in the software suite. It is fairly intuitive however students needed additional instruction.
Unfortunately, as this courses primary purpose was to act preparation for the Greenbelt Exam a larger focus could not placed on this material. However in an industrial setting most projects fail in the control phase. Regression to the mean is the natural trend. Anyone that has ever tried to lose weight or quit smoking knows that the trouble is always in sustaining the improvement.
The above histogram does not quite look normal and has a very large standard deviation 14%.
A dot plot again shows a strange pattern.
The distribution is in fact bimodal. Unfortunately due to ASQ’s interpretation of the meaning of work, a large number of qualified application were unable to sit for the actual Greenbelt exam and became disenchanted with the course and represent the lower distribution. This assumption was supported by a post hoc online survey.
However the test scores did appear to approve (even with the lower distribution)
And the improvement was very significant as indicated P value of 4.91 x 10^-13
On average the students improved 19.4% only a few students scores decreased,
The Paired T Test Results also confirm that the students test scores improved!
A P Chart was again used to detect difficult questions.
A Pareto Chart above shows the topics that generated that special cause variation in the prior P chart.
The initial process capability was quite poor, producing defects ~970,000 failures per 1,000,0000
The final process capability though still not best in class, is much better, producing 475,000 failures per million (the observed is used since the data was already proven to be non normal as it is bimodal)
*Actual data has not yet been released for the national average yetAs Confucius says “I hear and I forget. I see and I remember. I do and I understand.”