Number of stool specimens

Several studies have now examined the influence of the number of samples used for testing on clinical sensitivity and specificity. Allison takes any positive result from 3 stool samples measured using FlexSure OBT as an indication for referral and shows higher sensitivity for cancer than studies using single stool samples (Allison et al. 2007). Unsurprisingly other studies show agreement with that conclusion (St John et al. 1993; Allison et al. 1996; Knaani & Samuel 1997; Nakama et al. 1999; Greenberg et al. 2000; Nakama, Zhang & Fattah 2000; Rozen, Wong et al. 2003). Nakama et al. using Monohaem, showed sensitivities of 89% for cancer with 3 stools compared with 56% for a single stool (Nakama et al. 1999).

Using Hem-SP, Morikawa showed low sensitivity for cancer using a single-day sample (Morikawa et al. 2005). Rozen et al. (2006) used 3 stools for the OC-Sensor which contrasts with 2-day samples used in Japan (Nakama, Zhang & Fattah 2000) and 1-day biennial testing performed in Italy (Castiglione et al. 2002). The relative insensitivity in the Italian study to lesions in the proximal bowel (16.3 vs 30.7%) raises further doubts about the use of a single-day sample. In a study using OC-Sensor in an at-risk population, Levi et al. (2007) took numeric measurements from three samples and used the highest concentration of the three as the discriminating factor. Recent studies have taken the average concentration from 2 stool measurements as the discriminating parameter, an approach that reduces the positivity rate. 

The use of different cut-off limits and different numbers of stool samples illustrates how programme algorithms can manipulate clinical sensitivities and specificities for the lesions of interest. Chen describes the use of a cost-effectiveness model as a method of determining the optimal cut-off concentration for an iFOBT (Chen et al. 2007). In the study by Levi et al. (2007) using an iFOBT OCMicro, a scatter plot of 2 consecutive samples showed that of those with cancer or adenomas, 21 of 91 had elevated or markedly elevated faecal blood in one sample but were normal in the other. This is further evidence of intermittent or variable bleeding, sample heterogeneity or poor sample technique that will reduce clinical sensitivity. Imperiale (2007) commenting on the paper by Levi in his editorial in Annals of Internal Medicine (Levi et al. 2007), speculated that concentration-related clinical sensitivity and specificity could be used to determine post-test risk. If risk was related to subject age or sex, this would provide more sophisticated criteria for colonoscopy referral than is currently used. 

Optimising clinical performance using test cut-off limits & algorithms

Cut-off limits 

Until recently it has not been possible to adjust the analytical sensitivity of FOBT tests. This is still not possible for existing gFOBTs, with the exception of the simple adjunct of hydrating the specimen prior to testing with Hemoccult SENSA. With Hemoccult SENSA, hydration increases test sensitivity at the expense of specificity, thereby increasing the false positive rate (Mandel et al. 1993; Ransohoff & Sandler 2002). Hemoccult and Hemoccult SENSA have been compared in two large studies (Mandel et al. 1993). As a result of rehydration, the rate of positive results increased more than fourfold, from 2.4 to 9.8%. Sensitivity increased from 80.8% to 92.2% while both specificity and PPV decreased (specificity: 90.4% rehydrated and 97.7% non-rehydrated. PPV: 2.2 rehydrated and 5.6 non-rehydrated). In the study by Levin, Hess & Johnson (1997) the positivity rates were 5% and 14.6% and PPV 14% and 7% respectively for the non-rehydrated and the rehydrated. Rehydration using Hemoccult SENSA increases clinical sensitivity and decreases specificity and positive predictive value. The high positivity rate of this approach renders it unsuitable for population screening.

With iFOBTs that provide a numeric result, it is possible to adjust the cut-off limit to obtain an acceptable compromise between clinical sensitivity and specificity. This manipulation can provide an adequate detection rate from an acceptable cohort of subjects invited for colonoscopy. Several recent papers have addressed the issue of modifying the faecal haemoglobin cut-off limit of iFOBTs including the following studies (Sieg et al. 1999; Castiglione et al. 2000; Nakama, Zhang & Zhang 2001; Castiglione et al. 2002; Launoy et al. 2005; Vilkin et al. 2005; Rozen et al. 2006; Chen et al. 2007; van Rossum et al. 2009). The data are summarised in Table 4.5. By increasing the positive cut-off limit, the test sensitivity and positivity rate decreases and specificity and positive predictive values for colorectal cancer detection increase. It must be appreciated that these studies used different commercial products with different analytical characteristics, and therefore simple comparisons can be misleading. 

Chen found an analytical cut-off limit range of 100–150 ng/mL faecal haemoglobin in an iFOBT to provide an acceptable balance between sensitivity and specificity (Nakama, Zhang & Zhang 2001; Chen et al. 2007). Increasing the cut-off limit to 300 ng/mL brought an increase in specificity that was small for the corresponding decrease in sensitivity and detection of cancers. A recent study by Rossum of 6157 50–75 year old Dutch participants and using a single OC-Sensor collection and OC-Micro analyser concluded that dropping from the standard 100 ng/mL cut-off to 75 ng/mL brought ‘optimal’ results and may be recommended for population screening in the Netherlands (van Rossum et al. 2009). This study also concluded that where colonoscopy capacity is insufficient, a cut-off up to 200 ng/mL would result in minimal false negatives for cancer although more for advanced adenoma. Policy makers are faced with an arbitrary decision based on the balance between missed cancers/advanced adenomas and the cost of colonoscopy 

European guidelines for quality assurance in colorectal cancer screening and diagnosis

Comparative clinical performance - gFOBT and iFOBT

In the USA, Allison et al. (2007) prospectively compared two types of FOBTs, a sensitive gFOBT (Hemoccult SENSA) and a manual iFOBT (Flexsure). A large number of patients (7394 subjects were eligible for the study) were requested to perform both tests. All patients positive for either FOBTs were invited to have a total colonoscopy, whereas all patients negative to FOBT were advised to have a sigmoidoscopy. All cancers occurring during the two years following the test were identified, so that it was possible to estimate the absolute sensitivity and specificity for detecting advanced neoplasms in the left colon within two years after the FOBT screening for the two tests administered separately and in combination. The sensitivity for detecting cancer was 81.8% (95% CI = 47.8% to 96.8%) for the iFOBT and 64.3% (95% CI = 35.6% to 86.0%) for the gFOBT. The sensitivity for detecting distal advanced adenomas was higher for gFOBT than for iFOBT 41.3% (95% CI = 32.7% to 50.4%) vs 29.5% (95% CI = 21.4% to 38.9%). PPV was much higher for iFOBT than for gFOBT for distal cancer (5.2% and 1.5% for iFOBT and gFOBT respectively) and for advanced adenomas (19.1 and 8.9% for iFOBT and gFOBT respectively). The authors concluded that iFOBT has high sensitivity and specificity for detecting left-sided colorectal cancer and that it may be a useful replacement for the gFOBT.

The study by Dancourt et al. (2008) compared the performance of a 3-day gFOBT and 2-day iFOBT in 17 215 subjects. For 1205 subjects who participated and had colonoscopy, the PPV for the guaiac and immunochemical test was respectively 5.9% v 5.2% for cancer and 27.2% and 17.5% for adenoma.  

The study by van Rossum et al. (2008) represents a milestone in the comparison of gFOBT with iFOBT, being the first randomised trial in a population based screening setting. A large number of people (20 623) aged 50–75 years were randomised to either gFOBT (Hemoccult II, Beckman Coulter Inc. Fullerton, CA, USA) or iFOBT (OC-Sensor). For iFOBT, the standard cut-off of 100 ng/mL was used. iFOBTs showed higher compliance than did gFOBTs (56.9% vs 46.9% respectively p<.01). The positivity rate was significantly higher in iFOBTs compared to gFOBTs (5.0% vs. 2.4% respectively, p<0.01). Cancer or advanced adenomas were found, respectively, in 11 and 46 of gFOBTs and in 24 and 121 of iFOBTs. The detection rate per 1000 examinations for cancer was 71% higher in iFOBT compared to gFOBT; the detection rate per 1000 examinations for advanced adenomas was 106% higher in iFOBT as compared to gFOBT. The number-to-scope to find 1 cancer or 1 adenoma was comparable between the tests, with the PPV not statistically different. In conclusion, iFOBT compared to gFOBT demonstrated a higher detection rate with a similar PPV.

The results of these five studies are consistent with data from the first European screening programmes. The UK Pilot study adopted Hema-screen, a conventional non-rehydrating gFOBT, using duplicate samples on 3 consecutive stools extended to 2 further sets of 3 stools if indicated. This UK pilot study gave a positivity rate during the first round of 1.9%. The Detection Rates (DR) for cancer and neoplasia (cancer and advanced or non-advanced adenoma) were 1.62 in 1000 and 6.91 in 1000 respectively. The PPV for neoplasia was 46.9% in England and 47.3% in Scotland (UK Colorectal Cancer Screening Pilot Group 2004). 

Comparative clinical performance - gFOBT and iFOBT

Many studies comparing iFOBT and gFOBT have been performed in the last 8 years, and several systematic reviews of the literature have been undertaken more recently. 

In 2007 Kerr published a systematic review by the Health Technology Assessment (NZHTA) of New Zealand which had the aim of identifying the international evidence for the clinical and cost effecttiveness of screening tests for colorectal cancer (Kerr et al. 2007). This review included all primary research published as full original reports and secondary research, systematic reviews and meta-analyses published since November 2004. It also included seven eligible primary research papers (Rozen, Knaani & Samuel 1997; Rozen, Knaani & Samuel 2000; Saito et al. 2000; Zappa et al. 2001; Cheng et al. 2002; Cole et al. 2003; Ko, Dominitz & Nguyen 2003) and five eligible secondary research papers; Australian Health Technology Advisory Committee (AHTAC) (1997), Conseil d'Évaluation des Technologies de la Santé du Quebec (2000), Canada, Craven UK (Craven 2001), Young World Health Organization and World Organization for Digestive Endoscopy (Young et al. 2002), Piper Blue Cross Blue Shield Association Technology Evaluation Center US

The review concluded that “there is limited definitive evidence regarding superior immunochemical FOBT performance over the guaiac tests. However, evidence from cross-sectional studies suggests that the immunochemical test HemeSelect, Beckman Coulter Inc. Fullerton, CA, USA… is comparable, or superior, to guaiac testing… The conclusions on this topic should be revisited if further reliable evidence on the comparative performance of screening FOBTs becomes available”.

A similar conclusion was reached in a systematic review commissioned by the UK NHS and undertaken by the Centre for Reviews and Dissemination of the University of York in 2007 (Burch et al. 2007) which examined the literature until 2004. The review included 59 studies 39 evaluated gFOBTs, 35 evaluated iFOBTs and one evaluated sequential FOBTs. It concluded that there was no clear evidence from direct or indirect comparisons to suggest that guaiac or immunochemical FOBTs performed better. However amongst iFOBTs, Immudia-HemSP (now Hem-SP) appeared to be the most sensitive and specific. 

In the four years since 2004, six studies comparing the performance of gFOBT and iFOBT have been published (Levi et al. 2006; Smith et al. 2006; Allison et al. 2007; Guittet et al. 2007; Dancourt et al. 2008; van Rossum et al. 2008). Some further studies have investigated the accuracy of iFOBTs which, although without a direct comparison with gFOBTs, confirmed the performance of iFOBTs which was reported in the following studies 

In Australia, Smith et al. (2006) performed a paired comparison of an iFOBT (InSure) with a sensitive gFOBT (Hemoccult SENSA); 2351 asymptomatic and 161 symptomatic subjects were requested to perform both FOBTs. iFOBT returned a true-positive result significantly more often in cancer (n = 24; 87.5% vs. 54.2%) and in significant adenomas (n = 61; 42.6% vs. 23.0%) while the false-positive rate for any neoplasia was marginally higher with the iFOBT than the gFOBT (3.4% vs. 2.5%; 95% CI of difference, 0–1.8%): the PPV for cancer and significant adenomas was slightly better for iFOBT (41.9% vs 40.4%). 

Description of terms used to describe test effectiveness

Where, a are true positive, b are false positive, c are false negative and d are true negative

Sensitivity = a/(a+c) 

Specificity = d/(b+d) 

PPV = a/(a+b) 

“True” in true positive, is an abstract concept because in practice a reference standard must be adopted. For colorectal cancer screening, true is usually defined by the outcome of total colonoscopy, the best practical diagnostic procedure we have though it does not have a sensitivity of 100%. In a clinical setting it is not always possible to perform a total colonoscopy on all subjects who have negative screening tests, so it is difficult to estimate the number of false negatives (c) and true negatives (d). The difficulty of estimating false negative has a great impact on sensitivity but much less so on specificity. In fact (c) is a number much lower than (d), so that the sum c+d (i.e. the number of negatives to the test) is a small overestimate of d.

For sensitivity, (c) is a significant proportion of (a+c), so that it is necessary to have a direct estimate of the number of false negatives. Very often this estimate is obtained by measurement of the interval cancers (i.e. the number of colorectal cancers that are diagnosed in subjects negative to the test during defined interval of time). Clearly the reliability of the estimated number of false negatives will depend on the time interval, and that will increase as time elapses. It is therefore important when comparing estimates of sensitivity obtained in this way to verify that the time interval used is the same.

The ideal theoretical approach to estimating cancer-screening performance would be to obtain the disease status using a “gold-standard” method that is independent of the screening method. For colorectal cancer, the disease status is usually determined from a histological examination of biopsy specimens of those with positive test results, because it is not ethically acceptable to collect biopsies from all individuals undertaking a screening test. The sensitivity and specificity of screening test are therefore usually estimated using interval cancers. As initially described by Day (1985) interval cancers will not include slow-growing cancers missed by the test and not evident between two screening events (therefore clinical sensitivity will be overestimated). Conversely, interval cancers will include fastgrowing cancers not present at the time of the screening test, but developing during the interval period (thus underestimating clinical sensitivity). This limitation is common to all screening procedure evaluations. 

Clinical performance

Description of terms used to describe test effectiveness 

gFOBT screening has been proven to be effective in reducing colorectal cancer mortality (Hewitson et al. 2007). In randomised trials the reduction in cause-specific mortality ranged from 15% (Hardcastle et al. 1996) to 33% (Mandel et al. 1993). Such a large variance in mortality can be explained by test differences, different numbers of faecal samples, different intervals between invitation cycles (one-year or two-year) and different responses to invitation associated with the characteristics and composition of the population screened. The sensitivity and specificity quoted for a test will therefore be influenced both by the test’s analytical characteristics and the context in which the test is used and evaluated.

gFOBTs come in two forms, the conventional form with normal sensitivity and the more sensitive variety, Hemoccult SENSA, in which the sample is hydrated before analysis. Hemoccult SENSA performs quite differently from the gFOBTs used in European trials (Hardcastle et al. 1996; Kronborg et al. 1996) and is both more sensitive and less specific. Comparison of the clinical performance of gFOBT and iFOBT is complex because iFOBTs can have different levels of specificity and sensitivity indeed they may have variable positive cut-off concentrations. Changes in cut-off concentrations result in different clinical performance characteristics.

Although only one population-based RCT has been described with iFOBT (van Rossum et al. 2008), the many published diagnostic accuracy studies provide information on the comparative ability of current tests to distinguish subjects with or without colorectal cancer and adenoma and can be considered an acceptable indication of the satisfactory performance of iFOBT in population screening (Burch et al. 2007). 

Quality Assurance

 Quality assurance of gFOBT testing  

Whilst an immunochemical test is recommended, programmes that adopt a traditional guaiac test need to apply additional laboratory quality procedures. To minimise variability and error associated with visual test reading, including manual results input, the following procedures should be considered

o Use of appropriate temperature for artificial lighting and neutral-coloured walls in the reading laboratory; 

o Use of a national laboratory training programme to prosper consistency of interpretation; 

o A blinded internal QC check each day for each analyst prior to commencing testing;

o Adoption of a monitoring programme to identify operator related analytical performance (e.g. positivity variability and bias); and 

o Double entry of test results 

 Quality assurance of iFOBT testing  

Consistency in analytical performance must be assured by the adoption and application of rigorous quality assurance procedures. Manufacturer’s Instructions for Use must be followed. Laboratories should perform daily checks of analytical accuracy and precision across the measurement range with particular emphasis at the selected cut-off limit. Rigorous procedures need to be agreed and adopted on how internal quality control data is interpreted and how the laboratory responds to unsatisfactory results. Performance data, both internal quality control and external quality assessment data, should be shared and reviewed by a Quality Assurance team working across the programme. Sufficient instrumentation should be available to avoid delays in analysis due to instrument failure or maintenance procedures

External quality assessment  

A European external quality assessment scheme should be developed to facilitate Europe-wide quality assurance of occult blood testing and enhance the reproducibility of testing within and between countries providing population screening

 Outcome monitoring  

All aspects of laboratory performance in respect of the screening test should be part of a rigorous quality assurance system. Uptake, undelivered mail, time from collection to analysis, analytical performance (internal QC and external QA), positivity rates, lost & spoilt kits and technical failure rate, technician performance variability and bias should each be subject to rigorous monitoring

Quality of information 

 The proportion of unacceptable tests received for measurement is influenced by the ease of use of the test kit and the quality of the instructions for use. This proportion should not exceed 3% of all kits received; less than 1% is desirable  

Faecal sampling/collection system

Many factors influence the uptake and reliability of sample collection. Inappropriate implementation can result in grossly misleading results. No single collection methodology is supported by the literature; however, the following factors should be considered when selecting a device for taking samples in population screening:

The distribution process should be reliable and reach all selected subjects.

 The laboratory should be able to unambiguously identify the subject ID on the test device perhaps using a suitable barcode. 

 The laboratory should be able to check the manufacturer’s device expiry date on each returned device. 

 The instructions for using the device must be simple and clear. 

 The device should to be simple and easy to use by the target population. 

 The device should leave minimal opportunity for collection error. 

 The device should facilitate consistency in the volume of sample collected. 

 The device/instructions should discourage inappropriate repeat sampling into/onto the sample device. Misuse of the device by participants should not cause loss of sample buffer.

 The system should not be susceptible to interference from toilet bowl disinfectants, etc.

 The screening participant should be able to record the date of sample collection to ensure the laboratory can verify receipt within an acceptable sample stability period.  

 The process used by the subject for returning the device should be simple, reliable, safe and, when appropriate, should meet EU postal regulations.

A local pilot study should be undertaken to ensure that the chosen device and associated distribution, sampling and labelling procedures are acceptable   

Laboratory organisation

 Number of laboratory sites

  Population screening necessitates the receipt, measurement and recording of thousands of tests each day. The samples should be analysed without delay to avoid further sample denaturation and avoid an increase in false negative results. Inter-laboratory analytical imprecision is well described and can be observed through established external quality assurance schemes. Improved consistency is achieved by adopting common analytical platforms, analytical and quality standards and shared staff training. The analysis needs to be reproducible across a screening population and therefore the number of analytical centres should be minimised with automated analytical systems utilised wherever possible and agreed common testing procedures adopted by each centre

Laboratory staff 

 All laboratories providing population screening should be led by a qualified clinical chemist who is trained and experienced in the techniques used for analysis and with clinical quality assurance procedures 

 Laboratory accreditation and quality monitoring 

 All laboratories providing screening services should be associated with a laboratory accredited to ISO 15189:2007 Medical laboratories - Particular requirements for quality and competence. The laboratories should perform Internal Quality Control (IQC) procedures and participate in an appropriate External Quality Assessment Scheme (EQAS) 

 Distribution of FOBT kits by mail 

 Distribution and receipt of FOBT kits using local postal services can be an effective means of reaching the designated population 

Screening algorithm

 Sample and test numbers 

Few studies have examined the number of stool specimens necessary to optimise the diagnostic performance of FOBT. Consideration should be given to using more than one specimen together with criteria for assigning positivity which together provide a referral rate that is clinically, logistically and financially appropriate to the screening programme. The clinical sensitivity and specificity of testing can be modified depending on how the test data are used. Guaiac-based tests typically use 3 stools, but an algorithm using additional tests can be used to adjust clinical sensitivity and specificity

Determining test positivity

The choice of a cut-off concentration to be used in an immunochemical test to discriminate between a positive and negative result will depend on the test device chosen, the number of samples used and the algorithm adopted to integrate the individual test results. Whilst an increasing number of studies are reporting the experience of different algorithms, local conditions, including the effect on sample stability of transport conditions, preclude a simple prescribed algorithm at this time. Adoption of a test device and the selection of a cut-off concentration should follow a local pilot study to ensure that the chosen test, test algorithm and transport arrangements work together to provide a positivity rate that is clinically, logistically and financially acceptable

Test interference:  

 Dietary restriction 

 Dietary constituents present potential interference in guaiac faecal occult blood tests. Dietary restriction has not been demonstrated to significantly increase screening specificity, and risks reducing participation rate. The potential for dietary interference is significantly less for immunochemical tests. With the qualification that a diet peculiar to a particular country or culture may not have been tested or reported, dietary restriction is not indicated for programmes using either guaiac-based or immunochemical tests 

 Drug restriction 

 Interference from bleeding associated with drugs such as aspirin, nonsteroidal anti-inflammatory drugs and anticoagulants (e.g. warfarin) present potential interference in both guaiac and immunochemical faecal occult blood tests. Although the literature carries some contradicting reports of the effect of anticoagulants on screening outcome, drug restriction is not recommended for population screening programmes using either guaiac-based or immunochemical tests 

FAECAL OCCULT BLOOD TESTING

Guaiac-based faecal occult blood tests 

 Guaiac-based faecal occult blood tests have proven characteristics that make them suitable for population screening. They lack the analytical specificity and sensitivity of immunochemical tests, their analysis cannot be automated and the concentration at which they turn from negative to positive cannot be adjusted by the user. For these reasons guaiac-based tests are not the preferred test for a modern population screening programme, although depending on local labour costs, the mechanism of kit distribution and collection and reduced sample stability in immunochemical testing, they might prove more practicable and affordable than immunochemical testing 

Immunochemical faecal occult blood tests

Immunochemical tests have improved test characteristics compared to conventional guaiac-based tests. They are analytically and clinically more sensitive and specific, their measurement can be automated and the user can adjust the concentration at which a positive result is reported. Immunochemical tests are currently the test of choice for population screening; however, individual device characteristics including, ease of use by the participant and laboratory, suitability for transport, sampling reproducibility and sample stability are all important when selecting the iFOBT most appropriate for an individual screening programme

DNA and other related new markers  

Only tests for blood in faeces have been demonstrated to have the necessary characteristics to be suitable for population screening. DNA and other related new markers are currently unsuitable for screening, either singly or as members of a panel of tests

Sample stability between collection and analysis  

Whilst a maximum period of 14 days between collection and analysis is quoted for many guaiac faecal occult blood tests, that quoted for immunochemical tests is significantly shorter. Until more stability data are published, screening programmes should adopt the conditions and period of storage described in manufacturer’s Instructions for Use having determined that they are appropriate for local conditions which might expose samples to high temperatures for long periods of time