<h4>1 - What is fitness for purpose?&nbsp;</h4>The test meets the necessary performance criteria for a clinical binary examination to ensure it is suitable for making accurate and reliable clinical decisions in the intended patient population. Among others, performance criteria include clinical sensitivity and clinical specificity.&nbsp;<h4>2 - How to assess clinical performance?&nbsp;</h4>Assessing the clinical performance of a binary examination, stating results as 'positive' or 'negative,' involves calculating the examination's Clinical Sensitivity and Clinical Specificity. From the physician's point of view, clinical performance is evaluated by using predictive values, i.e., how likely is it that a positive or negative result corresponds to the presence or absence of the condition, which is also influenced by the prevalence of the condition in the population being tested.&nbsp;<h4>3 - What is Clinical Sensitivity?&nbsp;</h4>Clinical sensitivity (SS), also known as the True Positive Rate, measures the proportion of true positives correctly identified by the test. Data from a group of subjects who have been confirmed to have the disease or condition through diagnosis or a gold standard test (D1 in Table 1) are needed to calculate clinical sensitivity.<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Table_1_2x2_Contingency_table_for_clinical_diagnosis_241735a171.jpg" alt="Eurachem: Table 1. 2x2 Contingency table for clinical diagnosis." srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Table_1_2x2_Contingency_table_for_clinical_diagnosis_241735a171.jpg 245w," sizes="100vw" width="1401" height="461">The formula for clinical sensitivity is:&nbsp;SS = tp / (tp + fn), with:&nbsp;<ul><li>tp, True Positives are the cases where the test correctly identifies the presence of the disease.&nbsp;</li><li>fn, False Negatives are the cases where the test incorrectly identifies the absence of the disease when it is actually present.&nbsp;</li></ul>Table 1 represents the components of clinical sensitivity and specificity. Figure 1 illustrates an example of the overlapping of the number of components. The symbol "∩" represents the intersection of groups. For example, D1∩n denotes the set of negative results obtained for known positive cases, i.e., false negatives.<h4>4 - What is Clinical Specificity?&nbsp;</h4>Clinical Specificity (SP), also known as the True Negative Rate, measures the proportion of true negatives correctly identified by the test. Data from a group of subjects who have been confirmed not to have the disease or condition through a gold standard test (D0 in Table 1) are needed to calculate clinical specificity.The formula for clinical specificity is:&nbsp;SP = tn / (tn + fp), where:&nbsp;<ul><li>tn, True Negatives are the cases where the test correctly identifies the absence of the disease.&nbsp;</li><li>fp, False Positives are the cases where the test incorrectly identifies the presence of the disease when it is actually absent.&nbsp;</li></ul><img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Figure_1_Graphical_representation_of_an_example_of_85ab82a16b.jpg" alt="Eurachem: Figure 1. Graphical representation of an example of tp (D1∩p), tn (D0∩n), fn (D1∩n), and fp (D0∩p). " srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Figure_1_Graphical_representation_of_an_example_of_85ab82a16b.jpg 245w," sizes="100vw" width="1300" height="736"><h4>5 - How to calculate the uncertainty of clinical sensitivity and specificity?</h4>Calculating clinical sensitivity and specificity uncertainty involves statistical methods that estimate each parameter's confidence interval (CI). These intervals provide a range of values within which the true sensitivity or specificity is expected to lie with a certain level of confidence, typically 95 %. Please refer to the AQA:2021 for information about statistical models to compute the 95 % CI [1].<h4>6 - How to define targets?&nbsp;</h4>Target minimum values for clinical sensitivity and specificity should be identified. These values may originate from regulations or criteria based on the intended use of the binary results. A target or minimum value for the lower limit of the 95 % CI for result rate, i.e., SS or SP, (LLSS.95 or LLSP.95) or the upper limit of the 95 % CI for result rate (HLSS.95 or HLSS.95) should also be defined according to the purpose of the analysis. The target or minimum value (LLSStg or LLSPtg) is particularly significant when the impact of false results is critical. For example, for blood components used in transfusions, the screening for infectious diseases should be performed with tests associated with an LLSS.95 close to 100 %.&nbsp;Figure 2 illustrates five different tests, each assessed for clinical sensitivity and its associated uncertainty. These cases reflect different levels of compliance with a target sensitivity of 0.5. Case 1 is non-compliant, Cases 2 to 4 are partially compliant, and Case 5 is compliant. From a simplified statistical power view, the laboratory should always take into account the minimum number of samples when defining its target. This number can be estimated by simulation. For example, for a clinical sensitivity of 90 % the evaluation will require a minimum of 10 samples. It should be emphasized that samples must be representative of the target population for a clinical power evaluation - for example, epidemiologically representative samples of the types and subtypes of a given virological agent are required.<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Figure_2_Compliance_assessment_with_clinical_sensitivity_a21ece043c.jpg" alt="Eurachem: Figure 2. Compliance assessment with clinical sensitivity uncertainty using the lower 95% confidence interval limit for five performance assessment scenarios. " srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Figure_2_Compliance_assessment_with_clinical_sensitivity_a21ece043c.jpg 245w," sizes="100vw" width="1500" height="648">

Performance Assessment of Binary Output Examinations in Medical Laboratories

<h4>Introduction&nbsp;</h4>Millions of tests, measurements and examinations are made every day in thousands of laboratories around the world. There are innumerable reasons underpinning them, for example: as a way of valuing goods for trade purposes; supporting healthcare and construction; checking the quality and safety of food and feed; and in forensic analysis and environmental monitoring. Virtually every aspect of society is supported in some way by analytical work.&nbsp;The cost of carrying out these measurements is high and additional costs may arise from decisions made on the basis of the results. For example, tests showing food to be unfit for consumption may result in compensation claims. In addition, tests confirming the presence of banned drugs could result in fines or imprisonment. Clearly it is important to make a correct measurement and be able to show that the result is correct.<h4>The validation/verification process</h4>Most analysts know that method validation/verification is important, but exactly why, how and when it should be done is not always clear.&nbsp;Method validation/verification is the process whereby the laboratory demonstrates whether or not a method is ‘fit for purpose’ (Fig. 1). This means that the tests carried out should be appropriate with respect to uncertainty, cost, time etc. The final report should present analytical data in such a way that the customer can, readily, interpret it and draw appropriate conclusions.<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_The_validation_verification_82742c1029.jpg" alt="Eurachem: Fig. 1. The method validation/verification process. The laboratory ‘translates’ the customer’s problem into an analytical requirement, i.e. the method performance required to solve the problem. Method validation/verification includes a stage where various performance characteristics are evaluated and then compared with analytical requirements." srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_The_validation_verification_82742c1029.jpg 124w," sizes="100vw" width="821" height="1029"><h4>The Eurachem Guide&nbsp;</h4>A <a target="_blank" rel="noopener noreferrer" href="https://lcms.labrulez.com/products/2242">Guide from Eurachem</a> (Fig. 2) gives practical advice on how method validation/verification can be accomplished. The Guide and its supplements [1-4]:&nbsp;<ul><li>Introduce the basic concepts related to method validation and method verification;&nbsp;</li><li>Indicate how to plan, record and report validation and verification studies to best support the statement of ‘fitness for purpose’;&nbsp;</li><li>Provide key definitions and the rationale behind the experiments for assessing the various performance characteristics and related topics (Fig. 3);&nbsp;</li><li>Include quick reference tables that suggest experiments together with the necessary statistical calculations for evaluation and reporting each performance characteristic;&nbsp;</li><li>Provide support to the analyst on how to make the best use of method validation data for setting up an internal quality control plan.</li></ul><img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_The_Fitness_for_Purpose_of_Analytical_Methods_95fc1a74e3.jpg" alt="Eurachem: Fig. 2. The Eurachem Guide is available free of charge at www.eurachem.org." srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_The_Fitness_for_Purpose_of_Analytical_Methods_95fc1a74e3.jpg 135w," sizes="100vw" width="1062" height="1231"><h4>How should methods be validated/verified?</h4>The guide focuses on a single-laboratory approach to method validation/verification and gives guidance on planning, carrying out and reporting method validation and verification studies. Any method validation or verification study will require the laboratory to investigate several performance characteristics:<ul><li>Precision<ul><li>Repeatability&nbsp;</li><li>Intermediate precision</li></ul></li><li>Trueness</li><li>Selectivity</li><li>Working range</li><li>Analytical sensitivity</li><li>Limit of detection</li><li>Limit of quantification</li><li>Ruggedness</li></ul>Exactly which characteristics are studied will depend on the analytical application. Verification of the performance of a standard method requires substantially less work than validation of a method developed in-house. Legislative/sectoral requirements must also be considered.&nbsp;Sampling and subsampling can be part of the measurement or testing procedure and must, in those cases, be validated.

The importance of being fit for purpose

<h4>Types of comparisons&nbsp;</h4>Interlaboratory comparisons mean organisation, performance and evaluation of measurements and tests on the same or similar items by two or more laboratories in accordance with predetermined conditions. Comparisons are organised at all scientific levels, but the objectives, protocols and participants vary. In certification trials, the measurements are used to assign values to reference materials. In method validation studies (collaborative trials), the performance, e.g. trueness and precision of methods, is evaluated. The most accurate measurements are compared world-wide in ’key comparisons’.&nbsp;Proficiency testing (PT) schemes – also known as ‘external quality assessment (EQA) schemes’ or ‘laboratory performance studies’ – are one means of assessing the quality of routine measurements. PT schemes are the most common, and perhaps the most important, type of interlaboratory comparisons.&nbsp;<h4>Advantages of PT&nbsp;</h4>Participation in PT enables you to compare your results with those from other laboratories. It can also provide you with:&nbsp;<ul><li>Regular, objective and independent assessment of the quality of your routine analyses;&nbsp;</li><li>Feedback that stimulates improvement of the technical work;</li><li>Comparative information about method and instrument performance;</li><li>An overview of the quality of specific analyses in a sector, country or region.&nbsp;</li></ul><h4>Limitations of PT&nbsp;</h4>Ideally, PT samples are similar in nature to routine samples, and sufficiently homogeneous and stable not to influence the evaluation of participants’ performance. Due to practical aspects, PT samples are sometimes processed, e.g. stabilised and/or freeze-dried. Participants should be aware of this.&nbsp;PT schemes can be organised and evaluated in many different ways. There is no perfect protocol! For laboratories and their customers, and for accreditation and regulatory bodies, it can be important to know if the same result is judged differently by different PT providers.<h4>Appropriate PT schemes&nbsp;</h4>The huge number of analytes, and the large variety in the way tests are performed, means that it is not always possible to find a scheme that meets a laboratory’s exact requirements. Before signing up for a scheme, carefully check that the test materials, and the analytes and their levels fit your routine measurement procedures. Is the frequency appropriate and does the provider’s report give you sufficient information?&nbsp;<h4>Role of PT in measurement quality&nbsp;</h4>Correct measurements require both internal and external ‘tools’. During the validation step, the performance of the method is established. Subsequent use of control charts will show if the measurements are under statistical control. Many laboratories choose to accredit their services, thereby agreeing to implement a quality management system and accepting regular external audits of their work. Participating in PT is an effective external means to check that the procedures are fit for purpose!&nbsp;<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Proficiency_testing_schemes_and_other_interlaboratory_comparisons_2_7688414cd9.jpg" alt="Eurachem: Proficiency testing schemes and other interlaboratory comparisons" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Proficiency_testing_schemes_and_other_interlaboratory_comparisons_2_7688414cd9.jpg 206w," sizes="100vw" width="848" height="642"><h4>Educational aspects&nbsp;</h4>PT offers opportunities for education and training. Many providers have regular user meetings to discuss results and problem areas. Virtual PT schemes, using digital images of the samples, enable an unlimited number of participants, immediate feedback and repeated assessments.<h4>Accreditation of PT providers&nbsp;</h4>Some providers choose to accredit their PT schemes. The work with organising and evaluating the results will then be assessed according to international guidelines, similar to calibration and testing laboratories.<h4>Use of PT results&nbsp;</h4>Because PT schemes provide an overview of the analytical quality for specific applications, the results are increasingly used by laboratories’ customers, and by accreditation and regulatory bodies. PT helps identify measurement problems, which have a direct impact on trade, environmental monitoring, and health and safety.&nbsp;<h4>More information / further reading&nbsp;</h4>Information about PT providers and schemes can be obtained from your national accreditation body, from the <a target="_blank" rel="noopener noreferrer" href="https://www.eptis.org/">EPTIS website</a> or from other national or international organisations.

Proficiency testing schemes and other interlaboratory comparisons

Proficiency testing (PT) is applicable to quantitative, qualitative and interpretative assessments, but this leaflet will concentrate on PTs for quantitative tests. Participation in PT is an essential part of the quality assurance in analytical laboratories and provides them with many benefits. In PT the provider evaluates the performance of participants against pre-established criteria defined in the design of the PT scheme.<h4>Performance evaluation&nbsp;</h4>The majority of PT schemes involve some form of performance score, such as the z or similar scores1 , and corresponding assessment criteria. An assigned value (xpt) and a standard deviation for proficiency assessment (spt) are determined and used to calculate the performance score of the laboratory result (xi), e.g. the z score with z = (xi - xpt ) / spt<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_How_can_proficiency_testing_help_my_laboratory_2_c1bf0b596f.jpg" alt="Eurachem: How can proficiency testing help my laboratory?" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_How_can_proficiency_testing_help_my_laboratory_2_c1bf0b596f.jpg 195w," sizes="100vw" width="952" height="761">Assessment of z scores is based on the following criteria:&nbsp;<ul><li>|z| ≤ 2.0 is regarded as satisfactory;&nbsp;</li><li>2.0 &lt; |z| &lt; 3.0 is regarded as questionable (‘warning signal’);&nbsp;</li><li>|z| ≥ 3.0 is regarded as unsatisfactory (‘action signal’).&nbsp;</li></ul>This is based on the concept that normally distributed analytical results lie within two standard deviations with a probability of ca. 95 %, and within three standard deviations with a probability of 99.7 %.&nbsp;PT providers have several options to determine spt such as prescribed/perceived desirable analytical performance or the observed distribution of data. The spt used by the PT provider may not be appropriate for all laboratories. If justified, the participants may then calculate their own z score using an alternative spt value which is fit for their purpose.&nbsp;<h4>Corrective actions</h4>Unsatisfactory performance scores (‘action signal’) indicate possible problems in the analysis undertaken. The laboratory must investigate this (e.g. by checking for transcription/calculation errors, trueness and precision) and, if necessary, address the problems through appropriate corrective actions. Participation in the PT provides very limited benefits to the laboratory, if unsatisfactory performance scores are not acted upon.<h4>Evaluation of results over time&nbsp;</h4>In addition to internal quality control, regular participation in PT enables laboratories to monitor their performance over time and to identify trends before these become problems. Performance scores obtained from subsequent PT rounds can be plotted in a control chart.&nbsp;<h4>Method comparisons&nbsp;</h4>Where PT schemes require participants to report details of the method used, the PT report may enable the participant to compare the performance of their method with that of other methods used.&nbsp;<h4>Use of PT data to estimate bias&nbsp;</h4>The bias of a method should be established using certified reference materials (CRMs) or comparison with a reference method. However, these may not be available for all matrices, analytes and levels, or CRMs may not be fully representative of the real test samples. Participation in PT provides the opportunity to check the bias taking into account the effects of matrix and concentration variation, provided that a reliable estimate of the “true value” is assigned in the PT. Participation in several PT rounds also provides information on the bias variability which can be used as a contribution in the laboratory evaluation of the measurement uncertainty.&nbsp;<h4>Use of PT to check measurement uncertainties&nbsp;</h4>The ζ (zeta) score can help to check the plausibility of the laboratory’s measurement uncertainty estimate. It is calculated as follows:<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_zeta_score_calculation_b70d8b1187.jpg" alt="Eurachem: Zeta score calculation" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_zeta_score_calculation_b70d8b1187.jpg 245w," sizes="100vw" width="423" height="188">where xi is the laboratory’s result, xpt the assigned value, and u(xi) and u(xpt) are their respective standard uncertainties.&nbsp;The assessment criteria for satisfactory, questionable and unsatisfactory results are the same as for the z score. The uncertainty reported by a laboratory for its measurement result is an estimate of the accuracy that the laboratory claims to reach. If ζ scores are outside the acceptable range, this shows that the laboratory is not able to fulfill its own requirements. In other words the measurement uncertainty is underestimated.&nbsp;Additionally, the standard measurement uncertainty of a laboratory result can be expected to be lower than the reproducibility observed in the proficiency test. If the measurement uncertainty is much lower the uncertainty estimate should be reviewed.&nbsp;<h4>Demonstration of competence&nbsp;</h4>Successful participation in PT (| z | ≤ 2.0) is often seen as a proof of competence to customers, accreditation bodies and regulatory authorities. PT can also provide a valuable educational element to the laboratory, for example it can indicate the success of staff training or where additional training is required.&nbsp;<h4>More information / further reading&nbsp;</h4>B. Brookman and I. Mann (eds.) Eurachem Guide: Selection, Use and Interpretation of Proficiency Testing (PT) Schemes (3rd ed. 2021). Information about PT providers and schemes can be obtained from your national accreditation body, from the EPTIS website (www.eptis.org) or from other national or international organisations.

How can proficiency testing help my laboratory?

Participation in Proficiency Testing (PT) is an important part of assuring the quality of test results in a laboratory. The time and effort required can be costly, especially for laboratories performing many different tests, so selecting the most appropriate PT scheme is very important. Several PT schemes are often available for the same area of testing, so this leaflet focuses on key questions that can help laboratories choose those PT schemes that are best suited for their needs.&nbsp;<h4>Parameters included in the PT&nbsp;</h4>Are the matrices, analytes, and/or concentration levels of the test items offered by the PT scheme similar to those of samples encountered in the everyday practice of the laboratory? For example:&nbsp;Example 1: The levels of contaminants in a PT scheme for drinking water will be quite different from those expected in industrial wastes.&nbsp;A laboratory testing industrial wastes could:&nbsp;<ul><li>Participate, taking into account the limitations;&nbsp;</li><li>Not participate at all.&nbsp;</li></ul>Example 2: PT schemes for sequencing of DNA may offer either tissue samples or DNA extracts.&nbsp;Depending on its choice, the laboratory’s competence will be assessed for:&nbsp;<ul><li>The whole test;&nbsp;</li><li>The sequencing step only.&nbsp;</li></ul><h4>Strategies for data collection and analysis&nbsp;</h4>Are the strategies applied by the PT provider suitable for the needs of the laboratory? Factors to be considered include:&nbsp;<ul><li>Description of the statistical design applied;&nbsp;</li><li>Number of test items to be analysed and/or number of replicates requested;&nbsp;</li><li>Procedures for data collection from participants (e.g. submission by fax, e-mails or web-portals);&nbsp;</li><li>Procedures for comparison of results obtained by different measurement procedures;&nbsp;</li><li>Number and origin of participants;&nbsp;</li><li>Number of participants using the same measurement procedure as the laboratory; and&nbsp;</li><li>Methods and criteria used for performance assessment.&nbsp;</li></ul>The laboratory should also consider whether its customers, accreditation bodies and/or regulatory bodies have any specific requirements on statistical design.&nbsp;Example 3: A laboratory determines the fat content in milk powder, cereals and feed using three operationally defined measurement procedures (Röse Gottlieb, direct fat extraction, and fat determination by hydrolysis). Each measurement procedure could give different results for each matrix. It is important for the laboratory to check whether the different measurement procedures are taken into consideration for each matrix in the PT scheme<h4>Evaluation of performance&nbsp;</h4>Is the method used for assessing the participants’ performance (e.g. z score) clearly described by the PT provider and understood by the laboratory?Example 4: A laboratory may be interested in reporting the measurement uncertainty on its PT results so this can be taken into account in the performance evaluation, but the approach of the PT provider does not include this.&nbsp;Are the criteria used by the PT provider for performance evaluation compatible with the type of work undertaken by the laboratory and/or the relevant legislation?&nbsp;Example 5: A laboratory performing confirmatory analyses may look for more restrictive criteria for performance evaluation than laboratories using simpler screening techniques.&nbsp;<h4>Trust in the PT provider&nbsp;</h4>How can a laboratory assess the competence of a PT provider? Consider:&nbsp;<ul><li>Compliance with the requirements of ISO/IEC 17043, e.g. accreditation;&nbsp;</li><li>Experience;&nbsp;</li><li>Reliability of the assigned values; and&nbsp;</li><li>Fitness for purpose of criteria for proficiency assessment.&nbsp;</li></ul><h4>Communication&nbsp;</h4>Is the interaction between the PT provider and the participant suitable? Consider:&nbsp;<ul><li>Language used;&nbsp;</li><li>Accessibility and clarity of information;&nbsp;</li><li>Reports provided in a timely manner;&nbsp;</li><li>Regular communications, e.g. annual user meetings;&nbsp;</li><li>Level of support provided; and&nbsp;</li><li>Implementation of the customer’s wishes for PTs.&nbsp;</li></ul>Practical guidance for the selection of PT schemes is provided in Annex A of the Eurachem Guide [1].<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Selection_Use_and_Interpretation_of_Proficiency_Testing_PT_Schemes_by_Laboratories_2021_21f294fc1c.jpg" alt="Eurachem: Selection, Use and Interpretation of Proficiency Testing (PT) Schemes by Laboratories (2021)" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Selection_Use_and_Interpretation_of_Proficiency_Testing_PT_Schemes_by_Laboratories_2021_21f294fc1c.jpg 110w," sizes="100vw" width="790" height="1120"><blockquote>Information about PT providers and schemes can be obtained from your national accreditation body, from the <a target="_blank" rel="noopener noreferrer" href="https://www.eptis.org/">EPTIS website</a> or from other national or international organizations.</blockquote>

Selecting the right proficiency testing scheme for my laboratory

A laboratory should have a strategy for its participation in proficiency testing (PT) detailing which schemes it should enrol in (level) and how often (frequency). This is explained in a Eurachem Guide [1].&nbsp;<h4>A balanced selection of tools&nbsp;</h4>Quality related to technical work is dealt with in several ways and is specific to each laboratory. Thus, each laboratory should define its own level and frequency of PT participation after careful analysis of its other quality assurance (QA) measures, such as:&nbsp;<ul><li>Regular use of reference materials (RMs) and certified reference materials (CRMs);&nbsp;</li><li>Comparison of results by independent measurement procedures;</li><li>Participation in method development/validation and/or RM characterisation studies;&nbsp;</li><li>Use of internal quality control (IQC) measures;&nbsp;</li><li>Participation in other interlaboratory or intralaboratory comparisons, e.g. analysis of blind samples within the laboratory.&nbsp;</li></ul>These ‘tools’ are complementary but not perfect and they do not automatically ensure fit-for-purpose results! Important limitations should be identified, e.g. problems in obtaining a stable IQC sample or CRMs/RMs whose composition deviates from that of routine test samples. Also note that legislation may stipulate a minimum frequency of PT participation in certain areas. Frequently, some PT providers offer a flexible participation, e.g. 2, 4, 6 or 12 rounds per year; in rare cases, participation in PT may not be feasible at all.&nbsp;<h4>Areas of technical competence&nbsp;</h4>When planning PT participation, the laboratory starts the process by listing its areas of technical competence, defined in terms of three parameters:&nbsp;<ul><li>A measurement procedure;&nbsp;</li><li>A characteristic;&nbsp;</li><li>A product.&nbsp;</li></ul>Two examples are “a measurement procedure using quantitative real-time PCR (qPCR) for the determination of DNA sequences of pathogens in meat” and “a measurement procedure using inductively coupled plasma atomic emission spectroscopy (ICP-AES) for the determination of magnesium concentration in human serum”.&nbsp;<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Proficiency_testing_How_much_and_how_often_2_3d8da61ebf.jpg" alt="Eurachem: Areas of technical competence " srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Proficiency_testing_How_much_and_how_often_2_3d8da61ebf.jpg 245w," sizes="100vw" width="1372" height="630">An area of technical competence may encompass different, but equivalent and comparable, measurement procedures, different characteristics and/or different products. The laboratory can refer to the scope of a standardized procedure, or its method validation data, when planning its level of PT participation. If suitable PT schemes are available, the laboratory is expected to participate at least in a proficiency test related to each of its areas of technical competence.<h4>Risk assessment&nbsp;</h4>In order to decide on a suitable level and frequency of PT participation, the laboratory should conduct a simple risk assessment by considering, for example:&nbsp;<ul><li>Limitations in methodology, e.g. instrument instability or interferences from matrix components;&nbsp;</li><li>Extent of validation and/or verification;&nbsp;</li><li>Experience, knowledge and turnover of technical staff;&nbsp;</li><li>Quality and availability of RMs, etc.;&nbsp;</li><li>How the results will be used, e.g. forensic science and environmental control, and the consequences of a wrong result being reported to a customer;&nbsp;</li><li>Number of tests/calibrations/measurements undertaken between PT rounds;&nbsp;</li><li>Complexity of the measurement procedure and changes in requirements, e.g. lower compliance limits.&nbsp;</li></ul><img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Proficiency_testing_How_much_and_how_often_3_c13e83248b.jpg" alt="Eurachem: Risk assessment" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Proficiency_testing_How_much_and_how_often_3_c13e83248b.jpg 121w," sizes="100vw" width="337" height="434"><h4>Case studies&nbsp;</h4><h5>1. A laboratory determines a wide range of pesticides in fruits and vegetables.</h5>Depending on the specific pesticide, the laboratory uses two technically different measurement procedures, LC-MS and/or GC-MS. Additionally, different sample preparation techniques are required depending on whether the matrices are high water content (e.g. cucumbers, pears etc.) or low water content (e.g. chillies, peanuts). Thus, the laboratory would divide its activities into four areas of technical competence for which the laboratory would need to participate in PTs. The laboratory predominately assesses fruits and vegetables of high water content, so it chooses to participate in a PT scheme for high water content fruits/vegetables more frequently.<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Proficiency_testing_How_much_and_how_often_4_2bfc641a2b.jpg" alt="Eurachem: Table of Case study 1. * = pesticides analysed in the laboratory by LC-MS (1) or GC-MS (2)" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Proficiency_testing_How_much_and_how_often_4_2bfc641a2b.jpg 245w," sizes="100vw" width="1297" height="321"><h5>2. A company has two laboratories, at different sites, both determining a range of minerals and trace elements in various meat, fish and cereal products, using ICP-MS but with different sample preparation techniques for meat/fish matrices and cereal products.&nbsp;</h5>Thus, each laboratory would define its PT participation in terms of two areas of technical competence, both based on an ICP-MS procedure: (i) content of minerals/trace elements in meat/fish; and (ii) content of minerals/ trace elements in cereals. Each laboratory would need to participate in PT schemes that cover both matrices. However, the staff in one laboratory are generally less experienced due to greater staff turnover. The company decides that this laboratory should participate at a greater frequency than the other laboratory.<h4>The PT strategy&nbsp;</h4>Once the level and frequency of participation is established, the PT strategy, as part of the laboratory’s overall quality control plan, is in place. The PT strategy, for an accredited laboratory, could cover, at least, the period between full reassessments, being reviewed on an annual basis. During audits, the laboratory should be prepared to justify the technical arguments that have led to its decision on the “level” and “frequency” of PT participation.Information about PT providers and schemes can be obtained from your national accreditation body, from the<a target="_blank" rel="noopener noreferrer" href="https://www.eptis.org/"> EPTIS website</a> or from other national or international organizations.

Proficiency testing – How much and how often?

Routine chemical analysis typically involves several steps, e.g. selection of procedure, sampling, sample preparation, measurement, result calculation, uncertainty estimation, and reporting to customer. Based on the result, important decisions and actions are taken. The steps before and after the measurement are often referred to as “pre- and post-analytical”. There are many sources of uncertainty associated with this work, some of which can actually limit the overall quality.&nbsp;Proficiency testing (PT) and external quality assessment (EQA) schemes are effective means of assessing the quality of routine analyses. However, if there are specific procedures, either before or after the measurement, which are applied to the routine samples, but not to the PT test sample, the result reflects only the quality of part of the measurement process. Pre- and post-analytical PT/ EQA schemes are already widely used in laboratory medicine. The standard ISO 15189, that underpins accreditation of medical laboratories, stresses that EQA schemes should “... have the effect of checking the entire process including pre- and post-examination procedures ...”&nbsp;The examples in this leaflet illustrate the potential and importance of pre- and post-analytical PT/EQA. They are intended as an inspiration for the providers to further develop their external quality control activities.<h4>Pre- and post-analytical PT/EQA – a means for harmonisation&nbsp;</h4>By illustrating, e.g. lack of adherence to guidelines, or variation in sample preparation, or inconsistency in interpretation, pre- and post-analytical PT/EQA schemes pinpoint problems not apparent in other schemes. This may lead to improved harmonisation and changes to guidelines and standards.<h4>Example 1: Interpreting clinical chemistry information&nbsp;</h4>Patient background information together with analytical results can be used to check agreement of interpretational advice given by medical doctors. This type of scheme can, therefore, include both pre- and post-analytical aspects. The provider, e.g. through an expert group, ranks the quality of participants’ response and suggestions, and presents them in the report with comments.<h4>Example 2: Sampling requests in microbiology&nbsp;</h4>Using the scenario of food poisoning, pre- &amp; post-analytical aspects of sampling requests in a microbiology scheme can be highlighted. The provider gives information about food type, symptoms, and incubation time with a simulated sample. Participants choose which organism(s) to test for. The associated laboratory work involves examination of samples, report on method(s) of choice and results. Participants could, e.g. be asked to specify which toxins or pathogens are likely to cause the food poisoning and state whether or not they will disseminate information, e.g. to hospitals or food suppliers. The feedback report would then show the results of each participant together with the comments of the provider.&nbsp;<h4>Example 3: Toy testing&nbsp;</h4>Faulty design, usage and chance play a role in injury and death incidents related to children’s toys. Toy safety is the practice of ensuring that toys are safe, usually through the application of set safety standards. The PT provider sources a toy, performs visual assessment of its homogeneity, and distributes the samples. Participants are requested to assess the toy against the European standard EN 71 and they have to decide which clauses of the standard apply. The clauses chosen define the test(s) to be carried out. The feedback report will contain a “model answer” including clauses, tests and labeling requirements (when applicable).&nbsp;<h4>Example 4: Monitoring drinking water quality&nbsp;</h4>A post-analytical application can be underpinned by legislation and national regulations (Directive 98/83/EC). Participants receive a synthetic drinking water sample with properties close to one or more limiting values. They are requested to deliver results, uncertainties and a “pass/fail” recommendation to the customer. The recommendations of the provider are listed in the feedback report for educational purposes. Information can be fed back also to regulators to serve, e.g. as basis for further discussions.&nbsp;<blockquote><h5>More information / further reading&nbsp;</h5>Information about PT providers and schemes can be obtained from your national accreditation body, from the <a target="_blank" rel="noopener noreferrer" href="https://www.eptis.org/">EPTIS website</a> or from other national or international organisations.</blockquote>

Pre- and post-analytical proficiency testing

This leaflet gives some hints on the application of ISO/IEC 17043 [1] for PT providers organising PT schemes for sampling. If there is a comparison between participants and a mechanism for performance evaluation which meets the objective of the PT scheme for sampling, then ISO/IEC 17043 is applicable.&nbsp;<h4>Types of PT schemes for sampling&nbsp;</h4><h5>Type 1</h5>Only the sampling procedure is taken into consideration and evaluated. Performance assessment can be done through a preestablished scoring system or set of criteria. The performance can be assessed by deviations from a standard procedure or through an audit process where experts judge the performance of the participant.&nbsp;<h5>Type 2</h5>Samples collected by the participants are tested by a single laboratory chosen by the PT provider who must ensure that validated test methods with low variability are used. Thus, the variability obtained is attributed to the sampling and not to the test method.&nbsp;<h5>Type 3</h5>The performance of the participant is based on the testing results, and comprises both sampling procedures and test methods. Here the participant can perform the test at the sampling site or at their laboratory. The use of an additional appropriate reference material, ideally a certified reference material, provided by the PT provider to each participant, enables the analytical bias to be determined. Thus, the performance assessment is based on the sampling procedures and test methods combined or separately.&nbsp;<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Proficiency_testing_schemes_for_sampling_3_73f9231ef6.jpg" alt="Eurachem: Proficiency testing schemes for sampling" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Proficiency_testing_schemes_for_sampling_3_73f9231ef6.jpg 104w," sizes="100vw" width="578" height="862"><h4>How to apply ISO/IEC 17043 to sampling PT&nbsp;</h4>The following requirements from ISO/IEC 17043 might need some particular consideration for sampling PT:&nbsp;<ul><li>Personnel: The demonstration of the competence (knowledge of the planning of sampling, sampling techniques and preparation of sampling sites) of the personnel involved in organizing the sampling PT scheme.&nbsp;</li><li>Equipment, accommodation and environment: Environmental conditions should be taken into consideration by including them in the performance evaluation or by minimising or eliminating their influence.&nbsp;</li><li>Planning: Production, quality control, storage and distribution of proficiency test items for sampling PTs can be interpreted as “requirements for the sampling site” and handling/transportation of the samples once the sampling is performed.&nbsp;</li><li>Preparation of PT items: The sampling site must be prepared to ensure that each participant performing the sampling has an equivalent challenge (possible influences: rain, wind, temperature, participant, etc.).&nbsp;</li><li>Homogeneity and Stability: The item that is being sampled should be as similar as possible for all participants during the sampling exercise. Special care should be taken to minimise the influence of any previous participants in the exercise, for example by causing drill holes. Dynamic systems such as a river by their nature are constantly changing and therefore may not be homogeneous or stable.</li><li>Statistical design: ISO 13528 [2] should be considered when establishing the statistical design. It is important to distinguish between the sampling procedure and the test method in the statistical design, which will depend on the type of sampling PT. Sample transport effects that could have an influence should also be considered.&nbsp;</li><li>Assigned value / Evaluation criteria: The determination of any assigned value will depend on the type of sampling PT. The evaluation criteria should also consider the pre-sampling (e.g. container used) and the post-sampling (e.g. sample storage and transportation) aspects.&nbsp;</li><li>PT items handling and storage: It should be considered that the PT item includes both the sampling site and the samples taken during the sampling activity.&nbsp;</li><li>Packaging, labelling and distribution of PT items: The PT provider should provide clear instructions when specific labelling and packaging is required. Where there is a direct measurement the requirements of this section are not applicable.&nbsp;</li><li>Data analysis and records: Where the performance evaluation is based on a comparison to a reference procedure, this can be purely qualitative. Alternatively, observed deviations can be converted to numerical scores (e.g. 0 for negligible, 1 for minor, 2 for major) and some appropriate statistical data analysis performed.&nbsp;</li><li>Confidentiality and Collusion: When all the participants (or groups) are performing the sampling at the same time, then this must be made clear to the participants since confidentiality cannot be ensured and reasonable precautions need to be taken to prevent collusion given this situation.</li></ul><img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Proficiency_testing_schemes_for_sampling_2_d50329a4b4.jpg" alt="Eurachem: Proficiency testing schemes for sampling" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Proficiency_testing_schemes_for_sampling_2_d50329a4b4.jpg 210w," sizes="100vw" width="1099" height="816"><h4>Conclusion&nbsp;</h4>PT schemes for sampling play an important role in the improvement of sampling procedures and also in the development of participants from an educational point of view, especially if workshops with the participants are organised. PT schemes for sampling can also give an appreciation of the contribution of the sampling in relation to the overall quality of the measurement, including the associated contribution to the measurement uncertainty.&nbsp;<h4>More information and further reading&nbsp;</h4>Information about PT providers and schemes can be obtained from your national accreditation body, from the <a target="_blank" rel="noopener noreferrer" href="https://www.eptis.org/">EPTIS website</a> or from other national or international organizations.&nbsp;* <a target="_blank" rel="noopener noreferrer" href="https://pubs.rsc.org/en/content/articlelanding/2017/ay/c7ay90092a">Proficiency testing of sampling. AMC Technical Brief 78, 2017</a>Eurachem Guides/Leaflets:&nbsp;<ul><li>Selection, Use and Interpretation of Proficiency Testing Schemes by Laboratories, 2nd edition, 2011&nbsp;</li><li>Measurement uncertainty arising from sampling, 2nd edition, 2019&nbsp;</li><li>Leaflet on Pre- and post-analytical proficiency testing, 1st edition, 2009</li></ul>

Proficiency testing schemes for sampling

Test items are sometimes available from proficiency testing (PT) providers after the completion of a PT round. The purpose of this information leaflet is to advise laboratories on benefits and limitations of surplus PT items.&nbsp;<h4>Potential uses&nbsp;</h4>Surplus PT items have a number of potential uses including:&nbsp;<ul><li>Assessing new measurement procedures and verifying their correct implementation;&nbsp;</li><li>Training of new analysts;&nbsp;</li><li>Assessing the likely performance in a PT scheme;&nbsp;</li><li>Troubleshooting of measurement procedures and reassessing analytical performance following questionable or unacceptable results reported in a PT round;&nbsp;</li><li>As internal quality control (IQC) samples [1].&nbsp;</li></ul><h4>General considerations&nbsp;</h4>The following factors should be considered prior to use of surplus PT items:<ul><li>Check whether any relevant accompanying information is available and meets the end-user requirements, e.g. the assigned value and associated uncertainty of the measurand(s) of interest.&nbsp;</li><li>Assess their physical suitability and ensure that the matrix is appropriate. Some PT items have compositions close to that of routine samples, while others are synthetic or fortified (spiked). This may influence the suitability of the PT item for assessing different stages of the measurement procedure.&nbsp;</li><li>Check their availability. Surplus PT items may only be available in small numbers which will limit their usefulness for repeated testing over time, for example, as IQC samples.&nbsp;</li><li>Assess the stability information provided. PT items must be stable during the conduct of a round but there is no requirement for PT items to be assessed for stability following the completion of the round. The PT provider may be able to provide additional stability data or storage requirements, otherwise further evaluations by the end-user will be necessary.&nbsp;</li></ul><h4>Usefulness of accompanying PT data&nbsp;</h4>At the end of a PT round an assigned value is provided for each parameter, and participants’ performances are judged against a predefined performance acceptability criterion, for example, using a z score [2]. This information can support the use of surplus test items provided the following points are considered:&nbsp;<ul><li>If the assigned value is a consensus value end-users may need to reflect on the measurement procedure used to derive it;&nbsp;</li><li>Whether the metrological traceability of the assigned values makes them suitable for assessing measurement bias, for example if derived from a known spike value or a measurement made with a primary measurement procedure;</li><li>Whether the PT provider’s performance evaluation criteria agree with the fitness-for-purpose criteria required by the end-user. If so they could be used to set control limits in IQC charts. Otherwise own criteria have to be set [3].&nbsp;</li></ul><h4>Case study - Use of surplus PT items when implementing a standard method&nbsp;</h4>A laboratory wishes to adopt the standard method EN 15763 for the determination of the mass fraction of cadmium in foodstuffs by ICP-MS after microwave digestion. To this purpose, five PT items with different matrices and levels (mass fraction) were obtained from a PT provider and analysed in duplicate.&nbsp;The laboratory (i) verifies that the average of the replicates falls within the satisfactory performance range for the respective PT rounds (| z score | ≤ 2) and (ii) checks that the observed difference of the replicates does not exceed the repeatability limit (r) of the standard method.&nbsp;The data below confirms that the laboratory can operate the method correctly.<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_Use_of_surplus_proficiency_test_items_case_study_8e190bd7d7.jpg" alt="Eurachem: Case study - Use of surplus PT items when implementing a standard method " srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_Use_of_surplus_proficiency_test_items_case_study_8e190bd7d7.jpg 245w," sizes="100vw" width="1600" height="359">

Use of surplus proficiency test items

A laboratory will occasionally have a poor performance in a PT scheme. When this occurs, the laboratory should acknowledge it, carry out an investigation and document a review of possible causes, even if it decides not to take any specific action. The purpose of this leaflet is to advise laboratories on how to best address such events. A good management of the poor performance in a PT can save you time and money.&nbsp;<h4>Evaluation of poor performance&nbsp;</h4>Every unsatisfactory performance score indicates a problem which needs to be investigated. In addition the laboratory should set its own criteria for launching an investigation with regards to other indicators of possible poor performance, such as questionable performance scores or observed trends.<h4>Root cause investigation&nbsp;</h4>The depth of the investigation will depend on several factors, such as the importance of the analysis, the frequency of unsatisfactory results and evidence of trends.&nbsp;The laboratory should check whether the PT report gives an explanation for the unsatisfactory performance. If no reason is provided, a stepwise approach is preferable to maximize the chances of determining the root cause of the problem. This approach is described in Annex B of the Eurachem Guide [1].<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_How_to_investigate_poor_performance_in_proficiency_testing_Annex_B_be140d5b39.jpg" alt="Eurachem: How to investigate poor performance in proficiency testing - Annex B" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_How_to_investigate_poor_performance_in_proficiency_testing_Annex_B_be140d5b39.jpg 153w," sizes="100vw" width="726" height="740">The investigation should consist of the following steps and involve the personnel that performed the analysis and where approporiate, also involve the laboratory management:&nbsp;<ol><li>Analyse the raw data, internal quality control data, any trend from previous PT rounds and the overall performance of the participants in that round;&nbsp;</li><li>After investigation is complete, make a plan for corrective action and consider implications for previous test results;&nbsp;</li><li>Execute and record the corrective action(s);&nbsp;</li><li>Check whether the corrective action(s) was effective.&nbsp;</li></ol><h4>Causes for poor performance&nbsp;</h4>The causes of poor performance can be grouped into three categories:&nbsp;<ol><li>Clerical errors are not directly linked to the laboratory’s technical competence, but can underline that the laboratory may have a potential problem when reporting results to its customers. Clerical errors can include transcription errors, mislabelling, incorrect units and decimal point errors. Identifying them is an important first step of an investigation. If such errors are a regular cause of poor performance, the investigation should focus on the related aspects of the quality management system.&nbsp;</li><li>Technical problems can occur at any step of the analytical procedure. If the investigation does not enable the laboratory to identify the root cause, it may be necessary to review the method validation.&nbsp;</li><li>Problems related to the PT scheme. Poor performance could also be due to the PT scheme not being entirely appropriate. The Eurachem Guide [1] provides information on selecting an appropriate PT scheme. In other cases, a problem may have occurred with the PT items. The laboratory is encouraged to discuss its findings with the PT provider.&nbsp;</li></ol>The “Fishbone diagram” or “5 Whys” are useful tools to investigate the root cause of a problem.<h4>Case studies&nbsp;</h4><h5>Example 1</h5>A medical laboratory has provided results in a PT scheme for tumor markers in serum with a 20 % deviation from the assigned value, although the daily internal controls performed well. The laboratory investigates the process and discovers the mistake was due to an incorrect unit conversion factor being introduced for PT results when the IT system was last updated. Since patient results are reported in a different unit than the one used by the PT provider no patient results were affected.&nbsp;<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_How_to_investigate_poor_performance_in_proficiency_testing_0df20b3875.jpg" alt="Eurachem: How to investigate poor performance in proficiency testing" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_How_to_investigate_poor_performance_in_proficiency_testing_0df20b3875.jpg 245w," sizes="100vw" width="1539" height="815"><h5>Example 2</h5>A laboratory performing food and feed analyses obtained poor performance scores on Listeria monocytogenes enumeration, because the number of microorganisms found was underestimated. Laboratory investigations showed an issue in incubation time and temperature applied by the analyst. The laboratory found that the analyst used an out-of-date procedure.<img src="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/Eurachem_How_to_investigate_poor_performance_in_proficiency_testing_2_4e56101bbb.jpg" alt="Eurachem: How to investigate poor performance in proficiency testing" srcset="https://storage.googleapis.com/labrulez-bucket-strapi-h3hsga3/thumbnail_Eurachem_How_to_investigate_poor_performance_in_proficiency_testing_2_4e56101bbb.jpg 245w," sizes="100vw" width="1185" height="526">

How to investigate poor performance in proficiency testing

Article

Product

Events

Webinars

Application

Academy

Video

Poster

Scientists

Plastics

Environment

Food

Water

Health

Presentation

Scientific article

Interview

Toxicology

Popularization

Science and research

Software

Covid-19

Career

Laboratories

Various

Zaměření

Type of News

Focus

Eurachem is a European network promoting international traceability of chemical measurements and good analytical practice. It provides expert guidance, supports method validation, uncertainty evaluation, proficiency testing, and facilitates discussion on technical and policy issues in analytical quality.

Eurachem

Restek at ISC 2026

Thermo Fisher Scientific at ISC 2026

Waters at ISC 2026

Managing 4D-STEM data with smarter acquisition

Spatial Proteomics for Translational Research

Streamlining Endotoxin Testing: Automated Sample Preparation and Audit-Ready Workflows for QC Laboratories