ART Scientific Newsletter: Clinical Performance Indicators and Benchmarking in ART

ART Scientific Newsletter: Clinical Performance Indicators and Benchmarking in ART

Introduction

An inherent component of total quality management (TQM) is the measurement, documentation and assessment of ongoing practice using specific parameters known as performance indicators (PIs). Within assisted reproduction technology (ART), agreement on the definition and use of PIs for practitioners of ART lab processes has been previously reached, known as The Vienna Consensus.1 To complement the Vienna Consensus, agreed clinical PIs for ART medical practitioners have also been recently reported, known as The Maribor Consensus.2 This article attempts to summarize and review the outcome of The Maribor Consensus, highlighting its successes and limitations.

Definition of Clinical Performance Indicators

The remit of the clinical PI working group was to identify all the PIs relevant to a stimulated cycle of in vitro fertilization (IVF), be it insemination via IVF or intracytoplasmic sperm injection (ICSI). In this respect, PIs for intra-uterine insemination (IUI) and ancillary IVF procedures such as surgical sperm recovery (SSR) were not included, though they are common practice in clinical care and play a significant role in the successful treatment of infertility. The following essential steps in clinical management were reviewed to draw up a preliminary list of potential PIs for training and competency:

1. Diagnosis and indications for treatment of infertility
2. Controlled ovarian hyperstimulation (COH)
3. Monitoring of ovarian response, triggering of oocyte maturation, and oocyte pick-up (OPU)
4. Embryo transfer (ET) and clinical outcomes

Once identified, the literature was searched and reviewed to establish strengths, weaknesses and definitions, and a consensus meeting was held to agree upon a list of candidate PIs for broader consideration. An acceptability survey, using a five-point Likert scale, was sent out to the European Society of Human Reproduction and Embryology (ESHRE) Committee of National Representatives (CNR) and to over 2,000 members of the ESHRE Special Interest Group (SIG) in Reproductive Endocrinology to determine which candidates might be considered as ‘universally’ accepted PIs. A final list of PIs was derived from those which proved to be acceptable to 70% or more of respondents. In addition, national and central registry data from 11 countries were interrogated to establish competency (acceptable) and benchmark (aspirational) values for each PI.

Clinical Performance Indicators

Given the wide variation in clinical practice, only six clinical PIs received the approval of at least 70% of the respondents to the acceptability survey (Table 1). Unsurprisingly perhaps, no PIs relating to the differential diagnosis of infertility and indications for treatment were included, though there was 80% acceptance of general statements relating to this initial step in clinical management.

The importance of a diagnostic fertility workup was recognized, as was the relevance of burden, effectiveness, safety, and cost to ART treatment decision-making.3 For COH, the agreed PIs included cycle cancellation prior to OPU and the incidence of moderate/severe ovarian hyperstimulation syndrome (OHSS) in poor, normal and high responders, the OHSS rate also being separated into gonadotrophin-releasing hormone agonist and antagonist cycles. For OPU, the agreed PIs included the percentage of mature (metaphase II; MII) oocytes observed prior to ICSI and the complication rate following OPU, the former having previously been proposed in the Vienna Consensus as a reference indicator for the efficacy of COH.1 Somewhat surprisingly, the oocyte recovery rate, accepted as a reference indicator in the Vienna Consensus, was not accepted by the Maribor Consensus due to the technical feasibility and variability in ultrasound assessment of follicle size and numbers.

For ET, the agreed PIs included the clinical and multiple pregnancy rates (CPR and MPR) but not the ectopic pregnancy or live birth rate (LBR). Though these PIs could have been defined in ways to better reflect clinical and lab performance, the CPR per ET was agreed by most respondents (69%) as the most appropriate indicator of operator skill. Furthermore, a third of respondents did not consider ectopic pregnancy to be related to ET or operator performance and LBR was considered inappropriate for monitoring clinical practice since it is dependent upon retrospective data.

AS-Graph1
AS-Graph2

Reference Population for Competencies and Benchmarking

To assess clinical performance more objectively, it is necessary to define a reference population for which competency and benchmark values can be set. For consistency, it was decided that the Vienna Consensus reference population should also be adopted by the Maribor Consensus, as follows:

’Patients <40 years of age using autologous oocytes and fresh or frozen ejaculated spermatozoa for IVF or ICSI, excluding couples electing to have preimplantation genetic testing (PGT).’

It was suggested that, where relevant, the reference population might further be stratified into poor, normal and high responders to COH. Hence, consensus competence and benchmark values for all six clinical PIs were derived from published and calculated data reported by CNR members (Table 2). Since the benchmark for the percentage of MII oocytes prior to ICSI had previously been agreed in the Vienna Consensus1, the same value was adopted in the Maribor Consensus. Standard competence and benchmark values could not be agreed upon for the CPR because of heterogeneity in national data acquisition and validation. Therefore, it was proposed that values specific to the local context should be adopted, such as those reported to the European IVF Monitoring Consortium. Moreover, it was recommended that such values be adjusted for fresh and frozen cleavage stage and blastocyst ET.

Usage of Clinical PIs for Training and Achieving Competence

Naturally, clinical PIs provide an ideal measure for assessing the learning curve and attainment of competency of trainee practitioners. However, it is helpful to have some idea of the number and frequency of procedures that should be considered optimal to achieve competency, whether arbitrary or calculated using learning curve cumulative summation.4 Despite being challenged by members of the ESHRE CNR and SIG in Reproductive Endocrinology, the working group of the Maribor Consensus recommended a specific number and frequency of various clinical procedures for assessment of training and ongoing competency of practitioners (Table 3).

The consensus was that training in various procedures should be achieved within two years, in which time it is expected that a trainee would have gained experience with a minimum of 100 stimulated cycles and 75 cycles of OPU and ET. Ongoing competency in these clinical skills is determined by monitoring relevant PIs at the recommended time intervals or following a minimum number of cycles (Table 3), whichever occurs earliest. Should a gap between actual and expected competencies (Table 2) exist following initial training or at ongoing competency assessments, it was assumed that appropriate remedial action would be taken.

Summary

AS-Graph3

The Maribor Consensus has delivered six clinical PIs that complement the laboratory PIs proposed by the Vienna Consensus. Together, these consensus documents provide a beneficial framework for subspeciality training in reproductive endocrinology and infertility.

Furthermore, they constitute an integral component of TQM for enhancing clinical services in ART. However, it is hoped that future working groups will find greater acceptability of additional clinical PIs relevant to ART, including those related to other procedures such as ultrasound scanning, IUI and SSR.

References

  • 1. ESHRE Special Interest Group of Embryology and Alpha Scientists in Reproductive Medicine (2017) The Vienna Consensus: report of an expert meeting on the development of ART laboratory performance indicators. Reprod Biomed Online 35(5): 494-510.
  • 2. ESHRE Clinic PI Working Group et al. (2021) The Maribor Consensus: report of an expert meeting on the development of performance indicators for clinical practice in ART. Hum Reprod Open 2021(3): hoab022.
  • 3. Dancet EA et al (2014). “Patient-centered fertility treatment”: what is required? Fertil Steril 101(4): 924-926.
  • 4. Lopez MJ et al (2014). Individualized embryo transfer training: timing and performance. Hum Reprod 29(7): 1432-1437.

About the author

After completing his doctorate in 1987, he undertook postdoctoral research at the Royal North Shore Hospital in Sydney, and from 1993-1997 he was appointed Lecturer in Obstetrics and Gynaecology at the University of Nottingham in England, where he established the world’s first Master’s degree in Assisted Reproduction Technology with Simon Fishel PhD.

‘From 1998-2008 Steven was the Scientific Director of Westmead Fertility Centre in Sydney and appointed Senior Lecturer in Obstetrics and Gynaecology, and subsequently an Honorary Associate, at the University of Sydney. He is the recipient of numerous research grants, and an author and editor of several books, book chapters and peer reviewed journal articles. Steven’s research interests include cryopreservation, endometrial physiology, endometriosis and oocyte maturation, among others.

Steven Fleming PhD

Director Embryology, CooperSurgical

Subscribe to ART Scientific

ART Scientific is a community with science at its heart.

* indicates required

Like you, we hate spam. Your details will only be used for ART Scientific updates and nothing more. By subscribing you agree to our Privacy Policy and our Terms and Conditions.

Related Content

AS_Meena

ART Scientific Newsletter: The BIG FREEZE – Latest trends in oocyte vitrification

Introduction Human oocyte cryopreservation (OC) has gained immense popularity in the last decade. Post removal of the experimental label of…

AS_sperm-washing

ART Scientific Newsletter: Optimizing Therapeutic Sperm Washing Medium

Optimizing Therapeutic Sperm Washing Medium: Why are there clear differences in sperm progression and velocity between products? There are numerous…

AS_IVF-culture-Media

ART Scientific Newsletter: IVF Culture Media – Is It Time To Go Back To Basics?

Introduction Some of us will still remember the ‘good old days’ when, every two weeks, we would neatly line up…

AS_GM-csf

ART Scientific Newsletter: GM-CSF in IVF Embryo Culture

Introduction One of the greatest unmet challenges in Assisted Reproductive Technology is to formulate an IVF embryo culture medium that…

Related Products

GMcsf-bottle-heros

SAGE 1-Step™ GM-CSF Culture and Transfer Medium

Providing a new opportunity for patients in need Based on the long-established and clinically proven SAGE 1-Step medium, SAGE 1-Step…

Denudation-handling-hero

Denudation and Handling Pipettes

Pipettes for manipulation and transfer of cumulus-oocyte complexes (COCs), oocytes, embryos and biopsy samples The Stripper™ and EZ-Range™ denuding and…

product-EmbryoOptions-hero

Embryo Options℠

We think cryo billing, so you can think patient care Embryo OptionsSM is a digital cryopreservation billing and management platform…

G603-new-microscope

G603 Work Chamber

Minimise the stress on embryos The G603 combines the characteristics of a K-Systems workstation with an incubator like environment making it…

Get In Touch With Us

We’d love to hear from you. How can we help?