ART Scientific Newsletter: Key Performance Indicators & Benchmarking

ART Scientific Newsletter: Key Performance Indicators & Benchmarking

Introduction

A quality management system (QMS) requires meaningful and measurable parameters to be established and documented. Specific parameters such as key performance indicators (KPIs), may be measured, recorded and assessed as an invaluable component of total quality management. As such, KPIs may be used to monitor both clinic and individual performance.

However, it is necessary to have universally accepted externally referenced benchmarks against which to assess each clinic or individual’s competencies and performance. Hence, the main purpose of this article is to review benchmarking of staff and clinic performance.

Performance Indicators

The Vienna consensus also concluded that there are five performance indicators (PIs) that might reflect the quality of sperm preparation and insemination (Figure 2). It is important to appreciate that with sperm preparation, sperm motility is a more reliable PI than sperm recovery due to the variability in sperm preparation methods.

Furthermore, blastocyst quality on day 5 (D5) of development partly depends upon the quality of the paternal genome whereas early cleavage embryo development primarily reflects the maternal genome, embryonic genome activation not occurring until around day 3 (D3) of development in the human. The proportion of monopronucleate (1PN) oocytes following ICSI is expected to be lower than following IVF since MII oocytes are selected for ICSI whereas no such selection occurs prior to IVF.

In addition to the five PIs listed in Figure 2, it might also be desired to benchmark competency in semen analysis, 5-10% variation from the mean sperm concentration, motility and morphology being acceptable. Also, sperm cryopreservation could be another PI, >50% of the pre-freeze motility being deemed a generally accepted competency benchmark for slow freezing.

ART-Scientific-KPI-fig2jpg

Key Performance Indicators

The Vienna consensus concluded that there are 12 KPIs that might be considered integral to a QMS since they encompass the most important steps within the IVF lab process (Figure 3).

ART-Scientific-KPI-fig2

Since there is a tendency among some practitioners to include triploid (three pronucleate; 3PN) and polyploid (four or more pronuclei; ≥ 4PN) zygotes in their calculation of fertilization rates, it is important to note that the recommended KPI for fertilization rate only includes those zygotes displaying two pronuclei (2PN) and two polar bodies (2PB). Since the number of mature oocytes is unknown at the time of insemination by IVF, it is the long held belief of this author that the only way to scientifically and accurately compare IVF and ICSI fertilization rates is for the denominator to be the same for both, in which case this should be the number of COCs retrieved, and that belief is reflected in the same competency and benchmark values provided in Figure 3, which is an amendment from the values recommended by the Vienna consensus.

In much the same way that reporting cycle outcomes per cycle stimulated more faithfully reflects the overall quality of cycle management, reporting normal fertilization rate (number of 2PN + 2PB zygotes) per COC retrieved more faithfully reflects the overall quality of ovarian stimulation, oocyte handling (eg oocytes may be compromised or lost during denudation prior to ICSI but would not be included in assessment of fertilization rate when expressed per MII oocyte injected) and insemination, whether that be via IVF or ICSI. Although insemination procedures may differ, the initial number of COCs and final number of normally fertilized oocytes are equally relevant and it is important to be able to compare this KPI properly using what should be the same benchmark for both IVF and ICSI.

Previously, various KPIs for cryopreservation, including competencies and benchmarks, have been recommended.3 In this respect, it is important to differentiate between cryosurvival and viability.

The cryosurvival and viability of cleavage stage embryos can be determined by the number of intact blastomeres and number of thawed/warmed embryos developing on to the blastocyst stage, respectively. With blastocysts, however, determination of cryosurvival is less straightforward and viability can only be indirectly assessed via the timing and extent of blastocoele reexpansion, and only in those blastocysts exhibiting a collapsed blastocoele on thawing/warming.

Summary

There are three different types of indicator, RI, PI and KPI that may be used to monitor and assess the competency and performance of staff and organizations. The recently published list of LPIs provides a useful checklist for the purposes of lab audits and troubleshooting.2

References

  • ESHRE Guideline Group on Good Practice in IVF Labs. Revised guidelines for good practice in IVF laboratories (2015). Hum Reprod, 2016, 31: 685-686 (full report in supplementary data).
  • ESHRE Special Interest Group of Embryology and Alpha Scientists in Reproductive Medicine, The Vienna consensus: report of an expert meeting on the development of ART laboratory performance indicators. Reprod BioMed Online, 2017 (in press).
  • Alpha Scientists in Reproductive Medicine. The Alpha consensus meeting on cryopreservation key performance indicators and benchmarks: proceedings of an expert meeting. Reprod BioMed Online, 2012, 25, 146-167.
  • Reprinted from Reprod BioMed Online, Vol 35, Issue 5, ESHRE Special Interest Group of Embryology and Alpha Scientists in Reproductive Medicine, The Vienna consensus: report of an expert meeting on the development of ART laboratory performance indicators, 494-510, Copyright (2017), with permission from Elsevier.

Addendum

Various terms are used in benchmarking, which can be confusing, so it helps to understand their meaning, which is as follows:

Benchmark: Externally referenced result expected under optimal conditions.
Benchmarking: Comparison of results against internationally recognized gold standards.
Clinical performance indicator (CPI): Parameters used to monitor and assess clinical processes.
Competency: Minimum standard required to achieve proficiency and ongoing performance.
Competitive benchmarking: Comparison of results against local or national competitors.
Functional benchmarking: Comparison of processes against internationally recognized gold standard processes.
Internal benchmarking: Comparison of results between units within a related group.
Key performance indicator (KPI): Parameter used to monitor and assess crucial steps in a process.
Laboratory performance indicator (LPI): Parameter used to monitor and assess lab processes.
Laboratory performance measure (LPM): Synonymous with LPI.

Performance indicator (PI): Parameter used to monitor and assess steps in a process.
Program performance indicator (PPI): Parameter used to monitor and assess the overall organization’s processes.
Quality assurance (QA): All the systematic activities implemented within the quality system that can be demonstrated to provide confidence that a product or service will fulfil requirements for quality.
Quality control (QC): The operational techniques and activities used to fulfil requirements for quality.
Quality improvement (QI): Actively progressive increased effectiveness and efficiency within a quality system.
Quality management: All activities that determine the quality policy and its implementation via quality planning, QC, QA and QI within a quality system.
Quality management system (QMS): Quality policies, objectives, instructions and procedures.
Reference indicator (RI): Parameter used to indirectly monitor and assess steps in a process.
Total quality management (TQM): Quality management that embraces all staff, their patients, their suppliers and society at large.

About the author

After completing his doctorate in 1987, he undertook postdoctoral research at the Royal North Shore Hospital in Sydney, and from 1993-1997 he was appointed Lecturer in Obstetrics and Gynaecology at the University of Nottingham in England, where he established the world’s first Master’s degree in Assisted Reproduction Technology with Simon Fishel PhD.

‘From 1998-2008 Steven was the Scientific Director of Westmead Fertility Centre in Sydney and appointed Senior Lecturer in Obstetrics and Gynaecology, and subsequently an Honorary Associate, at the University of Sydney. He is the recipient of numerous research grants, and an author and editor of several books, book chapters and peer reviewed journal articles. Steven’s research interests include cryopreservation, endometrial physiology, endometriosis and oocyte maturation, among others.

Steven Fleming PhD

Director Embryology, CooperSurgical

Subscribe to ART Scientific

ART Scientific is a community with science at its heart.

* indicates required

Like you, we hate spam. Your details will only be used for ART Scientific updates and nothing more. By subscribing you agree to our Privacy Policy and our Terms and Conditions.

Related Content

AS_Meena

ART Scientific Newsletter: The BIG FREEZE – Latest trends in oocyte vitrification

Introduction Human oocyte cryopreservation (OC) has gained immense popularity in the last decade. Post removal of the experimental label of…

AS_Clinical-benchmark

ART Scientific Newsletter: Clinical Performance Indicators and Benchmarking in ART

Introduction An inherent component of total quality management (TQM) is the measurement, documentation and assessment of ongoing practice using specific…

AS_sperm-washing

ART Scientific Newsletter: Optimizing Therapeutic Sperm Washing Medium

Optimizing Therapeutic Sperm Washing Medium: Why are there clear differences in sperm progression and velocity between products? There are numerous…

AS_IVF-culture-Media

ART Scientific Newsletter: IVF Culture Media – Is It Time To Go Back To Basics?

Introduction Some of us will still remember the ‘good old days’ when, every two weeks, we would neatly line up…

Related Products

Zymot-hero-image

ZyMōt™ Sperm Separation Device

Sperm preparation made easy ZyMōt™ represents a fundamental change in thinking about how to effectively separate high-quality sperm1 for ART…

GMcsf-bottle-heros

SAGE 1-Step™ GM-CSF Culture and Transfer Medium

Providing a new opportunity for patients in need Based on the long-established and clinically proven SAGE 1-Step medium, SAGE 1-Step…

Denudation-handling-pipettes

Denudation and Handling Pipettes

Pipettes for manipulation and transfer of cumulus-oocyte complexes (COCs), oocytes, embryos and biopsy samples The Stripper™ and EZ-Range™ denuding and…

product-EmbryoOptions-hero

Embryo Options℠

We think cryo billing, so you can think patient care Embryo OptionsSM is a digital cryopreservation billing and management platform…

Get In Touch With Us

We’d love to hear from you. How can we help?