LTE Release Management Process
Application of PMGB tool sets for improvement
PMGB Workbook V34_engl
Application of PMGB to...
Description of Release Management
System
Acceptance
Quality
MS
Supplier (LTE)
shipment
Integration
and Test
Pilot RolloutI...
Release Management model – what is the target
performance?
Total Number of defects identified in Lab & Field
No. of
Defect...
SIPOC: Release Management
Do we have a common understanding of how the process works today?
Process Name: LTE Release Mana...
CTQ / CTB Matrix
What is critical for the customer’s satisfaction?
Critical to Customer (CTQ)
Voice of the Customer Centra...
CTQs / CTBs
Output Variable Matrix
Through which output variables would CTQs and CTBs be best described?
CTQ
Each SW relea...
Data Collection Plan
How do we get the data in order to measure the project performance?
Measurement Type of
measureme
nt
...
Graphic display of the Output Measurements
How well are we fulfilling customer requirements today?
Diagram
Operational def...
Graphic display of the Output Measurements
How well are we fulfilling customer requirements today?
Diagram
Operational def...
Graphic display of the Output Measurements
How well are we fulfilling customer requirements today?
Diagram
Operational def...
Graphic display of the Output Measurements
How well are we fulfilling customer requirements today?
Diagram
Operational def...
Process Function Diagram (‘Swim Lane’)
What does the current process flow look like?
Division Process stepsStart Stop
LTE ...
Value Stream Map
How does material and information flow within the process?
Supplier CustomerSupplier Release ManagerTest ...
Results of the process analysis
What are the critical factors in the process flow – where have you identified potential pr...
Process RACI Matrix
Who is responsible for what in the process?
Process name / description
Process Steps Divison / Area
In...
Failure Mode and Effect Analysis (FMEA)
Where in the process flow can defects occur?
Procedure:Procedure:
1. Determine pro...
Failure Mode and Effect Analysis (FMEA)
“Where can defects occur in the process flow?
Process
step
Potential
defect
possib...
Process Management Storyboard – Results & learning’s
During System Introduction Operability and Customer experience are tr...
Glossary
RN – Radio Networks ; central skill centre tasked
with Release Management
BU – Business Unit
FTE – Full Time Empl...
Required (‘7+1’)Tools for PMGB Workbook
5S (incl. photos before/after)
CAN DO - Tools
(select one for certification)
SIPOC...
Nächste SlideShare
Wird geladen in …5
×

6 sigma LTE release management process improvement

4.724 Aufrufe

Veröffentlicht am

This is my 6 Sigma case study pertaining to software introduction –commonly termed Release Management. This example captures LTE Radio System updates as an iterative (N+1) process in the scenario whereby a Supplier’s SW is underperforming necessitating more robust and efficient Release Testing.
It is supplier agnostic and can be applied to any Platform & SW Release Management regime where SDLC is the primary methodology and where system & customer data points are measurable in the environments.

Veröffentlicht in: Technologie
0 Kommentare
1 Gefällt mir
Statistik
Notizen
  • Als Erste(r) kommentieren

Keine Downloads
Aufrufe
Aufrufe insgesamt
4.724
Auf SlideShare
0
Aus Einbettungen
0
Anzahl an Einbettungen
246
Aktionen
Geteilt
0
Downloads
146
Kommentare
0
Gefällt mir
1
Einbettungen 0
Keine Einbettungen

Keine Notizen für die Folie

6 sigma LTE release management process improvement

  1. 1. LTE Release Management Process Application of PMGB tool sets for improvement PMGB Workbook V34_engl Application of PMGB tool sets for improvement 06.07.2013Process name : LTE System Release Management 1 6 Sigma Process Management : Jeffrey Gardner
  2. 2. Description of Release Management System Acceptance Quality MS Supplier (LTE) shipment Integration and Test Pilot RolloutI/P Quality MS Quality MS Quality MS System Development Life cycle New Release (N+1) Start Stop Preamble Release Management is a sub process of the Software Delivery Life-cycle (SDLC) LTE (Long Term Evolution) is a Global standard of 4th Generation Network deployed by Mobile Operators to provide communications and broadband services. A centralised Skill Centre owns the SDLC and Release Management approval through a ‘Test- Release Management process PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 2 A centralised Skill Centre owns the SDLC and Release Management approval through a ‘Test- it-once’ approach. Release Management process is sub-optimal with No. of defects above target level. Objective is to modify the Process to achieve Operational & Service excellence. The target outcome is better quality for internal customers and end users/consumers. The optimised Release Management process is repeatable.
  3. 3. Release Management model – what is the target performance? Total Number of defects identified in Lab & Field No. of Defects RFA RFP RRO SAS Time Joint System Test Pilot Rollout Improve Test Improve Improve Quality PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 3 Target outcome :deal with the root cause and reduce impact of managing the effects Reduction in effort expended to identify defects during lab testing and remedial actions in the field. Measurable improvement in defects detected in Lab compared to those in the field. RFA MS RFP MS RRO MS SAS MS Improve Test Quality Improve Quality Improve Quality
  4. 4. SIPOC: Release Management Do we have a common understanding of how the process works today? Process Name: LTE Release Management Supplier Input Process Output Customer START Supplier of LTE System/s System Release Patch Execute Patch Release Testing Patch under Pilot Patch passes Pilot Patch Rollout START PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 4 Complete Rollout System Release Acceptance Recommendation to National Companies Mobile Network Operators/multiple markets STOP
  5. 5. CTQ / CTB Matrix What is critical for the customer’s satisfaction? Critical to Customer (CTQ) Voice of the Customer Central Topic CTQ / Requirements I want the minimum number of SW faults in the Network Quality/Operational Each SW release shall be thoroughly tested prior to release to Field Critical to Business (CTB) SW faults in the Network release to Field Faults in the network should be within acceptable levels Efficiency Defect statistics (KPI’s) are within Target Minimum delivery time for SW patch Quality/Time Minimum delivery time for SW patch Faults are increasing accumulatively over time Quality Problem Reports are as few as possible Voice of the Business Central Topic CTB / Requirements PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 5 Voice of the Business Central Topic CTB / Requirements The total effort is increasing over the lifecycle Cost efficiency Effort shall be minimised with no more than 4 FTE’s allocated at one time. The defects may impact customer experience CEM/Revenue Minimise the impact to end customers
  6. 6. CTQs / CTBs Output Variable Matrix Through which output variables would CTQs and CTBs be best described? CTQ Each SW release shall be thoroughly tested prior to release to Pilot 0 3 9 0 0 Fault statistics (KPI’s) are within Target 0 9 9 3 0 Minimum delivery time for SW patch 0 0 3 9 0 Problem Reports are as few as possible 0 3 9 0 0 CTB PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 6 CTB Effort shall be minimised with no more than 4 FTE’s allocated at one time (applicable to RN Business Unit) 9 0 0 3 9 Minimise the impact to end customers 3 3 9 0 0 Correspondence 9 strong, 3 medium, 1 low, 0 none
  7. 7. Data Collection Plan How do we get the data in order to measure the project performance? Measurement Type of measureme nt (Output/ Process/ Type of data (continuous/ discrete) Operational Definition Responsible for data collection Method of data collection Data /time / frequency of measure- ment Data source / place of measurement Target value (Output/ Process/ Input) ment No. Priority 1, 2 PR’s in Pilot Output Continuous Fault removal efficiency PR ratio Lab/Field Release Manager Trouble Ticket System (TIS) Date/time stamped Release Manager/K PI calculation 3:1 Ratio (Lab to Field) No. Priority 1,2 PR’s in Rollout Ouput Continuous Quality Risk exposure PR ratio Lab/Field Release Manager Trouble Ticket System (TIS) Date/time stamped Release Manager/K PI calculation 9:1 Ratio (Lab to Field) PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 7 Lab/Field Patch availability for Field Output Continuous Supplier ;First- in/First-out Test Leader Observed Date/time stamped Test Leader </=5 days Contain effort to minimum Output Continuous FTE run rate Release Manager Reporting of actual effort Monthly ITG/Remaid Max 4 x FTE
  8. 8. Graphic display of the Output Measurements How well are we fulfilling customer requirements today? Diagram Operational definition of the measurement No. of Priority PR’s in Pilot : Relationship between defects found in the Lab and defects found in the field Pilot of the new release. Formulaic based on number of faults generated in the 2 phases. ResultDiagram Bar Graph shows the Target results for Testing efficiencies over the last 4 System Releases. In all cases the target was exceeded although the internal customer experience was unsatisfactory (internal effort, disruption and unplanned events). The customer experience (VOC) Result 100 50 PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 8 The customer experience (VOC) requires reduction in Operational impact. Efficiency KPI defined as a measure of the effectiveness of test assignment:- Efficiency KPI = No. of faults identified in test phase A Total No. of faults identified during testing Efficiency Release A Target 75% Target range 0 – 100 % 80.95 0 96.3 Efficiency Relese B Efficiency Release C Efficiency Release D 83.3392.86
  9. 9. Graphic display of the Output Measurements How well are we fulfilling customer requirements today? Diagram Operational definition of the measurement No. of priority PR’s in Rollout : Relationship between Defects found in the Lab and Defects found in the early Operational lifecycle of the new Release. Formulaic based on number of faults generated in the 2 phases. ResultDiagram Result 75 Bar Graph shows the Target results for Testing efficiencies over last 4 releases. In all cases the target was not achieved and the internal customer experience was unsatisfactory (internal effort, disruption and unplanned events). The customer experience (VOC) requires reduction in Operational impact. In target achievement the quality is a step 25 50 PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 9 Quality Risk Exposure Release A Target 10% Target range 5 – 25 % 35.71 0 30.2352.6321.6 Quality Risk Exposure Release B Quality Risk Exposure Release C Quality Risk Exposure Release D In target achievement the quality is a step improvement to the first Pilot Phase. Total defects reduce with increase in quality. Quality risk exposure KPI defined as adherence to quality targets and value add:- QRI KPI= No. of unidentified faults during testing Total No. of faults identified over all testing
  10. 10. Graphic display of the Output Measurements How well are we fulfilling customer requirements today? Diagram Operational definition of the measurement Effort : Radio Network BU - Adherence to Planning of effort - relationship between planned and actual effort. Calculation is proportion of planned vs actual effort (Effort = actual/planned) ResultDiagram Result 1 2 Bar Graph shows the Target achievement for project planning over the last 4 System Releases. In 1 out of 4 projects the target was not achieved due to overrun. The delay resulted in increased staffing costs - OPEX estimation. Total FTE allocation shall be maximum of 4 FTE’s (planned & actual). PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 10 Effort Release A Target 1 Target range 0,5 – 1,5 0.98 0 1.34 Effort Release B Effort Release C Effort Release D 0.850.85 of 4 FTE’s (planned & actual). Cost efficiencies are upheld. Effort KPI defined as ratio of planned vs actual:- Effort KPI= Actual Test Effort (person days) Planned Test Effort (person days)
  11. 11. Graphic display of the Output Measurements How well are we fulfilling customer requirements today? Diagram Operational definition of the measurement Patch availability : FIFO model (First-in, first-out), time period (days) between receiving patch from supplier to availability for Pilot/Rollout. Calculation is time based and variable in units of days. ResultDiagram Result 5 10 Bar Graph shows the average turnaround (Test process cycle) time for Patch releases (Release D). Operates on FIFO basis with Lab Test execution (FTE’s) as sole constraint. Rapid turnaround time increases fault likelihood. Better to define SLA as 5 day turnaround and offer <5d delivery on a Yr 2012 Release D PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 11 JAN Target 5d Target range 2d –5d 0 turnaround and offer <5d delivery on a per case basis. Cost efficiencies and Quality are upheld. FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC 6.7 2.7 1.7 2.7 2 2 10 3 4 4 3.5 5
  12. 12. Process Function Diagram (‘Swim Lane’) What does the current process flow look like? Division Process stepsStart Stop LTE System System release patch available New patch required New patch required RN Business Unit Operations (Central) LTE System Supplier: Optimization Confirm Pilot Pass y N patch available VE 1d required NVA <40d required NVA <40d Execute Patch Release Testing VA 11d Patch under Pilot VA 10 d 1 2 Patch Rollout VA 7d NOK 3 Confirm as Rollout gating NVA 2d 4 Patch Rollout completion VE 2d 5 PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 12 Optimization Functional Team VA Value Adding VE Value Enabling NVA Non Value Adding Confirm Rollout OK NVA 0.5d MNO Market Units OK NOK System Release Acceptance NVA 2d
  13. 13. Value Stream Map How does material and information flow within the process? Supplier CustomerSupplier Release ManagerTest Lead 1 FTE – 6 days effort 1 FTE – 8 days effort Delivery to Test Lab Lab Install Define Test Template 1 FTE – 1 days effort Defect 3 FTE - 5 days effort 1 FTE – 0.5 days effort 2.5 FTE – 1 day PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 13 System Test Assess Pass/Fail Roll out 1 FTE – 0.5 days effort Test Report Pilot Readiness 1-7 days (Tues) 3 FTE - 5 days effort Major Release 5 FTE – 5 days effort 1 days – 1 FTE’s Pilot 1 FTE – 7 days effort 1 FTE – 75 days effort 2.5 FTE – 1 day effort
  14. 14. Results of the process analysis What are the critical factors in the process flow – where have you identified potential problems? Number Process Step Problem 1 Execute Patch Release Testing The Quality of the SW received from Supplier is not ‘error free‘. Supplier provides Change note information to enable captive Testing to be scoped within known constraints of environment and manpower. 2 Patch under Pilot The Pilot activity offers the best opportunity to establish defects to Operations and/or end customers. Depending on Fault (PR) priority the resulting leadtime to provide a resolution is between 1-40 days. 3 Patch passes Pilot As a quality gate (MS) the Pilot network experience provides data to support the Ready for Rollout decision. It does not full fill the following :- a) Guarantee of a Rollout free from defects b) A time extension of the Pilot phase 4 Patch Rollout During the rollout phase new defects may occur which often results in a request for new SW patch/es. This has 2 primary effects :- a) As SW footprint expands so do the related defects with broader impact PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 14 4 impact b) If rate of patches through the supply channel exceeds the rollout plan (patch deploy rate), the network suffers a dip in SW performance. 5 Patch Rollout complete Ideal outcome is one final patch satisfying customer quality expectation. In practice emerging defects neccessitate further release management (extension of project into Operations ‘maintenance mode‘) with extended delays and effort for MNO Business Unit.
  15. 15. Process RACI Matrix Who is responsible for what in the process? Process name / description Process Steps Divison / Area Input Output CommentsDescription of the Operations Optimization Input Output Comments Nr. Description of the process Step Supplier RN Business Unit Operations Central Optimization MNO Markets 1 Execute Patch Release Testing C R/ A I I Release patch 2 Patch under Pilot A R R I 3 Patch passes Pilot R/ A C C I 4 Patch Rollout A R C I Recommend Market MNO have sole authority for PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 15 R = Responsible - Person who is competent fof executing the task. A = Accountable – Person who has the responsability over the task. C = Consulted – Person who must be consulted about a decision. I = Informed – Person who must be notified of a decision. 5 Patch Rollout complete A C R R SAS to MNO Markets sole authority for SAS milestone decision
  16. 16. Failure Mode and Effect Analysis (FMEA) Where in the process flow can defects occur? Procedure:Procedure: 1. Determine process steps 2. Determine which defects may occur 3. Determine what the consequences of the defects are 4. Determine the magnitude of the consequences of the defects (Intensity from 1-10) 5. Determine what the possible causes for the defect are 6. Determine the likelihood of occurrence of the identified causes (Occurrence from 1-10) 7. Determine which existing controls are in place 8. Determine the likelihood of detection of the defect causes (Detection from 1-10) 9. Calculate the Risk Priority Number (I x O x D) PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 16 9. Calculate the Risk Priority Number (I x O x D) 10. Define and take appropriate measures 11. Calculate/Estimate new Risk Priority Number
  17. 17. Failure Mode and Effect Analysis (FMEA) “Where can defects occur in the process flow? Process step Potential defect possibility Potential effect Intensity Defect cause Occurrence Existing controls Detection RPN Actions Intensity Occurrence Detection RPN Execute Patch Release Testing Defect/s not identified in Lab Testing Defect slips to Pilot. Operability and/or end customer impact 5 Patch release too rapid. Lab not fully representative of Field Pilot (limited investment policy). 8 Y: appropriate allocation of Resource and expertise 4 160 Detection; patch content based decision that allows appropriate time for Patch testing. Drive ‘Release when ready’ of patch. 5 8 3 120 Patch under Pilot Defect not identified in Field Pilot Defect slips to Pilot. Operability and/or end 5 Unstable Pilot network. Patch related event data lost in high number of 8 Y : Implemented Operational Network surveillance 4 160 Detection; statistically rule out ‘interferers’ by implementing network freeze of Pilot area. 5 8 2* 80 PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 17 and/or end customer impact number of network events. surveillance unit Patch passes Pilot Performanc e reports (KPI) not available No data to evaluate Pilot exit 5 Data not reported by in- house Performance Management unit 8 Y : Implemented Operational Network surveillance unit 3 120 Detection; Source Performance data from LTE Supplier 5 8 2* 80 *Preliminary assessment – actual implementation will provide measurement data
  18. 18. Process Management Storyboard – Results & learning’s During System Introduction Operability and Customer experience are treated with equal importance. Historically the quality KPI’s measured during Rollout phase are not achieved. The RN Release Management capability is fit-for-purpose (skill set + manpower) but with Lab/Test Plant constraints The CTQ’s and Function are acceptable to Central Op’s and thus are mapped to Output Matrix and data collection. Improvement opportunities in Lab test coverage (scope and time) and Pilot extension are focus of 6 Sigma study. Partial implementation of improvements in current ‘N’ Release XXYY (Patch turnaround time + Effort). Full implementation of improvements with benefits expected in ‘N+1’ Release Introduction (AABB).Full implementation of improvements with benefits expected in ‘N+1’ Release Introduction (AABB). Key Learning’s from describing & Measuring Process Analysis of defects found in field - 65% of defects are reproducible in Lab Results of key measures since applying PMGB tools and techniques Effected extension of Test process cycle time (5d minimum) with measured increase in Lab defect ‘capture’ rate Effort KPI contained to ca 3.75 x FTE’s (Work force reporting Feb 2012 –Dec 2012) 35% Not Reproducible PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 18 (Work force reporting Feb 2012 –Dec 2012) Investment uplift in Lab necessary to increase defect capture rate 35% 65% Reproducible
  19. 19. Glossary RN – Radio Networks ; central skill centre tasked with Release Management BU – Business Unit FTE – Full Time EmployeeFTE – Full Time Employee SLA – Service Level Agreement PR – Problem Report KPI – Key Performance Indicator SW – Software MS – Milestone Lab – Representative system for Release Testing PMGB Workbook V34_engl 06.07.2013Process name : LTE System Release Management 19 Lab – Representative system for Release Testing System – Supplier’s aggregate of equipment & SW LTE – Long Term Evolution : 4th generation 3GPP standardised system for Mobile communication
  20. 20. Required (‘7+1’)Tools for PMGB Workbook 5S (incl. photos before/after) CAN DO - Tools (select one for certification) SIPOC Seven MUST DO – Tools (all are required for certification) ++ 5S (incl. photos before/after) Value Analysis Time Analysis (incl. process balancing) FMEA Poka Yoke SIPOC VoC CTQ (Tool 1) Output Measurement Variables (Tool 2) Data Collection / Measurement Plan ( incl. Operational Definition) Graphical display of process performance Value Stream Map/Swim-Lane PMGB Workbook V34_engl Process name : LTE System Release Management 20 Value Stream Map/Swim-Lane RACI (Responsibility Matrix) 06.07.2013

×