Six SIGMA evaluation of 17 biochemistry parameters using bias calculated from internal quality control and external quality assurance data

Background Six Sigma is a popular quality management system that enables continuous monitoring and improvement of analytical performance in the clinical laboratory. We aimed to calculate sigma metrics and quality goal index (QGI) for 17 biochemical analytes and compare the use of bias from internal quality control (IQC) and external quality assurance (EQA) data in the calculation of sigma metrics. Methods This retrospective study was conducted in Marmara University Pendik E&R Hospital Biochemistry Laboratory. Sigma metrics calculation was performed as (TEa-bias)/CV). CV was calculated from IQC data from June 2018 - February 2019. EQA bias was calculated as the mean of % deviation from the peer group means in the last seven surveys, and IQC bias was calculated as (laboratory control result mean-manufacturer control mean)/ manufacturer control mean) x100. In parameters where sigma metrics were <5; QGI=bias/1.5 CV) score of <0.8 indicated imprecision, >1.2 pointed inaccuracy, and 0.8-1.2 showed both imprecision and inaccuracy. Results Creatine kinase (both levels), iron and magnesium (pathologic levels) showed an ideal performance with ≥6 sigma level for both bias determinations. Eight of the 17 parameters had different sigma levels when we compared sigma values calculated from EQA and IQC derived bias% while the rest were grouped at the same levels. Conclusions Sigma metrics is a good quality tool to assess a laboratory's analytical performance and facilitate the comparison of the assay performances in the same manner across multiple systems. However, we might need to design a tight internal quality control protocol for analytes showing poor assay performance.


Introduction
Clinical laboratories are constantly making great efforts to produce more efficient and reliable results.Quality control plays a vital role in the laboratory's ability to produce accurate test results, and it has been a component of laboratory medicine for more than 50 years.While the first quality control methods were designed to detect errors visually, in the 1930s, Shewhart and his colleagues pioneered statistical analysis in the control process.Then, Levey and Jennings (1, 2) charts, Westgard multiple rules, and Sigma metric are successfully added to quality control management.Using statistical data to monitor the process has enabled the laboratorians to reduce the possibility of producing inappropriate results.
More than 70% of clinical decisions are based on laboratory results, and most clinicians believe that laboratories produce test results with zero errors.However, there is no zero-fault process (3).Errors can occur at any time during the testing process while controlling and monitoring is the task of the laboratory specialists.It is crucial to minimise the error rate regarding patient safety (1,4).
Depending upon the time of presentation, laboratory errors are observed in preanalytical (46-68%), analytical (7-13%), and post-analytical (18.5-47%) phases, and performing quality studies for all phases is of great importance for the reliability of the results.In the analytical phase, quality indicators encompass the internal and external quality control results.While the quality indicator for the preanalytical phase is the rejected sample rate, in the post-analytic phase, they are the rate of results that were not given on time and the rate of unreported panic value.Another approach is to use the Six Sigma concept to evaluate preanalytical, analytical, and post-analytic phases (5,6).
Six Sigma is a quality management tool used for monitoring and improving the performance of analytical processes in the clinical laboratory.It is not just a tool to describe process performance but also detects errors and eliminates their causes as much as possible to reach the zero error goal (7,8).The Six Sigma scale ranges from 0 to 6, and as the Sigma value increases, the performance of the test increases.While a sigma value of 6 in a process indicates that the performance of the test is excellent, the lowest sigma value for acceptable performance is 3.When the analysis performance 5 sigma, the quality goal index (QGI) should be calculated to determine the cause of poor performance.The two most important concepts that drive analytic quality management are accuracy and precision.QGI determines the problems of inaccuracy and imprecision for the analytical processes concerning their quality goals (9).
In this study, we aimed to calculate sigma metrics and quality goal index (QGI) for 17 biochemical analytes and compare the use of bias from internal quality control (IQC) and external quality assurance (EQA) data in the calculation of sigma metrics.

Materials and Methods
The study was retrospectively conducted in Marmara University Pendik E&R Hospital Biochemistry Laboratory.A total of 17 parameters, namely albumin, alkaline phosphatase (ALP), alanine aminotransferase (ALT), amylase, aspartate aminotransferase (AST), creatine kinase (CK), iron, inorganic phosphate, glucose, calcium (Ca), creatinine, magnesium (Mg), total bilirubin, total cholesterol, total protein, triglycerides, and uric acid were enrolled.The analyses were done by an AU5800 biochemistry analyser (Beckman Coulter, USA), and data were collected over a 7-month period (01/June/2018-28/February/2019).According to our quality control schedule, we ran 2 levels of IQC samples (Bio-Rad, USA) every day and one level of EQA sample (RIQAS, UK) per month, either normal (N) or pathological (P) as defined by the external agency.The sigma metrics were estimated according to the formula below using bias, CV, and total allowable error (TEa) (3).Sigma = (TEa-bias)/CV.The coefficient of variation (CV) was calculated for each IQC level separately.

CV (%) = (SD/mean) x100
The bias % for each parameter was calculated from both IQC and EQA data.Bias (%) = [(our IQC or EQA result -Beckman Coulter IQC mean, or peer group mean of EQA)/ Beckman Coulter IQC mean, or peer group mean of Conclusions: Sigma metrics is a good quality tool to assess a laboratory's analytical performance and facilitate the comparison of the assay performances in the same manner across multiple systems.However, we might need to design a tight internal quality control protocol for analytes showing poor assay performance.

EQA] x100
Total allowable error (TEa) values were decided according to Clinical Laboratory Improvement Amendments Specifications (CLIA'88).
Statistical Package for the Social Sciences (SPSS Inc.; Chicago, IL, USA) software was used to analyse the sigma values obtained from internal and external quality control data.We checked for normality using the Kolmogorov-Smirnov test.To compare the mean difference between the two groups, we conducted a paired t-test.A group with less than three sigma values was considered unsuccessful, while more than three were deemed successful.The Mc Nemar test for paired samples was used to compare the sigma values.
Quality goal index (QGI) values were calculated to understand the reason behind lower performances according to sigma metrics (6).QGI = Bias /1.5 CV For analytes that fall short of Six Sigma quality, a QGI ratio of <0.8 indicates imprecision, a ratio >1.2 indicates inaccuracy, and a QGI ratio between 0.8-1.2indicates both imprecision and inaccuracy.CV, bias, Sigma and QGI values were calculated with Microsoft Excel Software.

Results
Average CV (%) values of 17 biochemistry parameters calculated from IQC samples over 7 months range between 3.05-5.87for level 1 and 2.4-4.72 for level 2. Bias (%) values were calculated from IQC and EQA for the same period (Table I).
The sigma and QGI values calculated using the bias obtained from EQA data (either N or P samples) were summarised in Table II.Among the 17 analytes, the performances of amylase (N and P), CK (N and P), iron (P), and magnesium (P) were excellent with >6 sigma level.ALP (P), ALT (P), magnesium (N), and uric acid (P) showed very good sigma performance, while the performances of ALP (N), AST (N and P), iron (N), triglyceride (N and P), uric acid (N) were good with <5-≥4 sigma level.On the other hand, the performances of ALT (N), calcium (P), total bilirubin (N), and total protein (P) were medium with <4-≥3, sigma level while albumin (N and P), inorganic phosphate (N and P), glucose (N and P), calcium (N), creatinine (N and P), total cholesterol (N and P), and total protein (N) had poor performances (sigma metrics was less than 3).QGI values were calculated for parameters which had analytic performance <5 sigma and the leading causes of poor performance were determined accordingly.The performance of 7 parameters (ALP, ALT, amylase, CK, iron, magnesium, and uric acid) at one or more EQA levels was very good or excellent.Twelve parameters (albumin, ALP, AST, iron, inorganic phosphate, glucose, calcium, total bilirubin, total cholesterol, total protein, triglyceride, and uric acid) exhibited precision problems at one or more EQA levels.In comparison, two parameters (ALT and creatinine) showed that there were accuracy and precision problems at one or more EQA levels (Table II).
The sigma and QGI values which were determined using the bias calculated from the IQC data analysed daily between June 2018 and February 2019, were given in Table III.
The QGI values parameters with analysis performance <5 sigma were also given in Table III.These values were calculated using 2 levels of IQC bias data, and the leading causes of poor performance were determined accordingly.The performance of seven parameters (ALP, ALT, amylase, CK, iron, magnesium, and uric acid) at one or more IQC levels was either very good or excellent.Eleven parameters (Albumin, ALP, ALT, AST, inorganic phosphate, glucose, calcium, total bilirubin, total cholesterol, total protein, triglyceride) exhibited precision problems, two parameters (AST and creatinine) showed that there were problems with accuracy and precision.In contrast, one parameter (amylase) showed accuracy problems at one or more IQC levels.The mean and standard deviation of quality control sigma values were 4.38 ±2.17, while external quality control sigma values had a mean and SD of 4.30±2.13,respectively.However, the paired ttest showed no statistically significant difference (P=0.547) between the two groups.The performance evaluation for sigma values showed no statisti-cally significant difference between the two paired groups (McNemar's test, P=1.000).
Westgard has determined the preference of IQC rules according to sigma levels (10).We have selected the rules according to sigma levels calculated using EQA and ICQ bias data (Table IV).Sigma methodology is one of the critical process improvement tools used in evaluating analytical performance, revealing errors in accuracy and precision together.It is based on calculations emphasising the use of quantitative techniques to measure the actual performance of the process (10).Detecting and eliminating errors increases the testing process's quality.Sigma methodology is used for quantitative comparison of laboratories and methods, allowing the laboratory to determine its quality control strategy.
In this study, we have evaluated the analytical performance of 17 biochemistry parameters, namely albumin, ALP, ALT, amylase, AST, CK, iron, inorganic phosphate, glucose, calcium, creatinine, magnesium, total bilirubin, total cholesterol, total protein, triglycerides, uric acid.We have calculated the sigma values of these tests using the bias values derived from IQC and EQA data and selected Westgard quality control rules accordingly.We have also calculated each parameter's quality goal index (QGI) to determine whether the problem in tests with a sigma value <6 was imprecision or inaccuracy (8,11,12).
We have calculated bias % using EQA or IQC values, which both had 2 levels, and we have obtained 68 sigma values for 17 biochemistry parameters.Eight parameters (albumin, ALT, amylase, iron, calcium, total bilirubin, total cholesterol, and uric acid) had different sigma levels when we compared sigma values calculated from EQA and IQC-derived bias %, while the rest of the parameters were grouped at same levels.IQC-derived sigma values of albumin (level 2), ALT (level 2), iron (level 1), total cholesterol (level 2), and uric acid (level 1) were higher than EQA-derived counterparts.On the other hand, amylase (levels 1 and 2), iron (level 1), Ca (level 2), and total bilirubin (level 2) had IQC-derived sigma values lower than their EQA-derived counterparts.
When QGI was calculated using EQA-derived bias, albumin (N and P), ALP (N), AST (N and P), iron (N), inorganic phosphate (N and P), glucose (N and P), Ca (N and P), total bilirubin (N and P), total cholesterol (N and P), total protein (N and P), triglycerides (N and P), and uric acid (N and P), parameters were below 0.8.We interpreted this as an error in imprecision, which is an expression of random error.Random errors cannot be predicted in magnitude and direction; therefore, they can be positive or negative and occur with repeated measurements.These errors occur due to unstable electrical supply, unstable temperature and incubation conditions, bubbles in reagents and reagent lines, individual operator variation in pipetting, timing, etc.Therefore, these conditions should be reviewed in tests with a QGI below 0.8.QGI values of ALT (level 1) and creatinine (level 1 and level 2) were calculated between 0.8 and 1.2.In these tests, the source of the problem was determined as an error in both precision and accuracy, which showed the presence of both random and systematic errors.Systematic errors are always in the same direction, either positive or negative.Therefore, to increase performance in these tests, situations that may cause both random and systematic errors should be reviewed and controlled (13).
Among 17 parameters, ALT (level 2), CK (level 1 and level 2), iron (level 2), and Mg (level 2) had a sigma value of six or above, and the daily IQC evaluation can be done with one level of IQC sample by using 1 3s Westgard rule alone.ALP (levels 1 and 2), AST (levels 1 and 2), amylase (level 2), Mg (level 1), iron (level 1), triglyceride (levels 1 and 2), uric acid (levels 1 and 2) with sigma values between 4 and 6 the performance of the tests can be evaluated with two levels of control once daily with 1 2.5s Westgard rule alone.Albumin (level 2), ALT (level 1), total bilirubin (level 1), total cholesterol (level 2), and total protein (level 2) parameters with sigma values between 3 and 4 the use of two levels of controls twice daily with 1 3s /2 2s /R 4s /4 1s /8x Westgard's multirule is necessary.
We have performed root cause analysis (RCA) for parameters with sigma values below 3 to review and control the causes that lead to poor performance.RCA is a systematic process to develop corrective actions by identifying the causes of undesirable events and the factors that cause changes in performance (14,15).Fishbone diagrams should be prepared and evaluated for possible error causes (16).In the RCA analysis of tests with low sigma values (<3), the number of operators, performances of devices and methods, and environmental factors must be questioned.The laboratory should consider using alternative methods (17).
Numerous studies were done, and several investigators reported different sigma metrics.As an example, Singh et al. (15) have calculated sigma values of 3.9 and 3.5 for total protein (levels 1 and 2), while the values were 3.0 and 3.2 in the study of Kumar et al. (6).Differences in sigma values between our research and other studies may arise due to the type of analysers, quality control materials, reagents, or differences in calibration.In addition, the different sources from which the total allowable error values are taken may cause the sigma values to differ between studies.
With Sigma methodology, quality control processes that need to be implemented are selected, contributing to improving analytical performance.When calculating sigma values, both IQC and EQA data are valuable in evaluating the analytical stage, and we have seen minimal differences between the sigma calculations based on them.We concluded that bias % could be calculated from IQC data when external quality assurance data were unavailable.We must prefer test-specific quality control rules since the sigma values are different for different tests.It should be remembered that Sigma evaluation processes significantly contribute to the laboratory in terms of total quality.

Table I
Average CV (%) values of level 1 and 2 IQC samples, bias (%) values calculated from IQC and EQA samples, and TEa values obtained from CLIA'88 for 17 biochemistry parameters.
46 Çevlik, Haklar et al.: Six Sigma Evaluation of 17 Biochemistry Parameters Table II Sigma metrics and QGI for parameters with Sigma <5 calculated using bias values derived from EQA data (*imprecision, **imprecision and inaccuracy).