Performance Evaluation is the ultimate “Check” in the quality management Plan, Do, Check, Act methodology. Per API 1173, the pipeline operator shall establish and maintain a procedure to identify key performance indicators (KPIs) whose measurement details the effectiveness and adequacy of the PSMS.
What is Performance Evaluation?
- The determination of the effectiveness in meeting the stated objective for any requirement
What Areas are Subject to Performance Evaluation
- Stakeholder Engagement
- Risk Management
- Operational Controls
- Incident Investigation
- Safety Assurance
- Management Review and Continuous Improvement
- Emergency Preparedness and Response
- Competence, Awareness and Training
- Documentation and Recordkeeping
- The comprehensive, systematic, integration of PSMS
- The state of the “Safety Culture”
- The effectiveness of the Audit Process
- The Maturity of the PSMS
Does the Performance of One Tenant Effect the Other Tenants?
- PSMS tenants need to do more than overlap, they need to be integrated within the PSMS because the output from any given tenant may serve as an input to another tenant resulting in its performance potentially having a much greater effect than just on the tenant being analyzed
What is the Objective of Performance Evaluation?
- To analyze, document and report effectiveness in meeting objectives
- The output of the analysis of Performance Evaluation (Check) is the primary input to Management Review and Continuous Improvement (Act)
- The determination of PSMS implementation maturity
What is a Performance Metric?
- A key performance indicator that signifies how effectively objectives are being met
- KPI can be either leading (execution) or lagging (results)
What is a Performance Measure?
- The measured values of each metric
How can we Manage Performance Evaluation?
- Identify the objective
- Define the metrics that support the determination of effectiveness in meeting the objective
- Define the analytics to be applied to the metrics
- Determine the means by which the measure of each metric will be collected
ICAM – is a web-based, configurable Process and Workflow Management platform utilized to manage, schedule, track, document, measure performance and report the execution of the PSMS. In this scenario, ICAM is used to manage Performance Evaluation; however, the platform can also be utilized to manage the actual execution of the various tenants.
ICAM was initially funded by PHMSA in 2004 to support the process management requirements associated with pipeline integrity. Pipeline operators currently enjoy the benefits of ICAM in the management of gas transmission, gas distribution, liquid and storage field integrity, public awareness, control room management and aerial patrol.
The following example demonstrates performance metrics (measures) for SOAR. These Stakeholder Engagement measures are automatically collected in pSEc and served up in a variety of dashboards for analytical purposes.
- Safety Observation and Reporting (SOAR)
- Leading KPI
- Number and nature of submissions
- Number and nature of items reported
- Lagging KPI
- Number of submissions w/ disposition
- Number of submissions slated for corrective action*
- Number of corrective actions taken*
- Measured effectiveness of corrective actions*
- Leading KPI
* Once disposition has been completed, the resolution process/workflow is managed in ICAM