Validity of MAST-CLA for diagnosis of arthropod allergy using receiver operating characteristic (ROC) analysis

The prevalence of allergic disease is rapidly increasing. Over the years, studies have consistently been indicating that sensitization to house dust mite allergens is closely associated with asthma in children and adults (Illi et al., 2001; Jaakkola et al., 2006), but no ideal treatment has yet been developed. The skin prick test, a typical in vivo test, is being extensively utilized for the diagnosis of allergic diseases, and is considered the gold standard, although it also has several disadvantages, including invasiveness or false positive reactions (Kiniker, 1989). The radioallergosorbent test (RAST), a typical in vitro test using patient sera, has been extensively utilized by allergists around the world, but it has an important handicap in that it requires the use of radioisotopes. Additionally, enzymatic immunoassay tools, such as the multiple allergen simultaneous test-chemiluminescent assay (MAST-CLA) (Brown et al., 1985; Miller et al., 1984) and the Pharmacia CAP System^™ (Van Durme and Stevens, 1989), are commonly used, and have evidenced good agreement rates of results with the skin prick test (Agata et al., 1993; Finnery et al., 1989; Orgino et al., 1993; Scolozzi et al., 1989; Scolozzi et al., 1992; Tang and Wu, 1989; Tang et al., 1994). MAST-CLA has the advantage that it can test for 35 common allergens simultaneously, and thus it is suitable as a screening test. This is why it is currently one of the most popular diagnostic tools used in the Republic of Korea (Hwang et al., 2001; Lee et al., 1995; Yang et al., 1998). However, a controversy rages among allergists as to the cut-off points for a positive result (Hwang et al., 2001; Lee et al., 1995; Yang et al., 1998), and this has ramifications with regard both to specificity and sensitivity. Therefore, the lack of a consistent cut-off standard significantly ameliorates the reliability of MAST-CLA.

The present study was conducted to determine the validity of MAST-CLA as compared to the skin prick test for arthropod allergens, i.e., Dermatophagoides farinae, Dermatophagoides pteronyssinus, and a cockroach mix, using agreement rates, and to determine the optimal individual cut-off points for improved reliable test results, using receiver operating characteristic (ROC) curves.

A total of 97 subjects (54 men and 43 women, all are same class volunteer students of a medical school) in an age range of 24.7 ± 2.7 (mean ± SD) years old, none of whom had ever been assessed for allergic diseases, were evaluated. Three allergens derived from D. farinae, D. pteronyssinus and a cockroach mix (Bencard, Worthing, West Sussex, UK) were utilized as follows in routine laboratory tests conducted at the Soonchunhyang Cheonan Hospital (Table 1). Histamine and normal saline solution were employed as positive and negative controls. All allergens and controls were applied at the medial side of the forearm, and then pricked using a 26-gauge needle. After 15 min of pricking, the size of each of the indurations was measured using the means of the longest and shortest diameters. In cases in which the size of the indurations was equal to or larger than the positive control (histamine), the results were considered to be positive. The Korean inhalant panel of the MAST-CLA (MAST-Immunosystems, Mountain view, California, USA) was used for comparison. The panel actually contains the profile of 35 allergens, but 3 allergens corresponding to the skin prick test were analyzed (Table 2). The MAST-CLA was conducted in accordance with the method described by Brown et al. (1985). All MAST-CLA results are graded into 6 classes (0, trace, I, II, III, and IV).

Table 1.

Results of MAST-CLA by classes about arthropod allergens

Table 2.

Consistency rates between MAST-CLA and skin prick test against arthropod allergens

The statistical analysis was conducted with the SPSS software (SPSS v. 12.0, SPSS Inc., Chicago, Illinois, USA). Percentage agreement and κ-values were calculated in order to evaluate the consistency of the skin prick test and the MAST-CLA (class I regarded as a positive cut-off point). The sensitivity and specificity of each MAST-CLA point (class of trace, I, II, III, and IV) was calculated against the skin prick test. The ROC analysis was utilized to evaluate the adequacy of MAST-CLA and to determine the optimal cut-off points. A P-value below 0.05 was considered to be statistically significant.

The positive rates for 3 different allergens were 49.5% for D. farinae, 50.5% for D. pteronyssinus and 30.9% for the cockroach mix from the skin prick test. Also, from the MAST-CLA, the positive rates of D. farinae, D. pteronyssinus and the cockroach mix were 47.4%, 42.3%, and 15.4%, respectively (Table 1). The overall percentage agreement rates and κ-values are provided in Table 2. All of the values were statistically significant (P < 0.05).

For the purpose of the ROC analysis, the sensitivities and specificities for each of the allergens were calculated by changing the cut-off points of MAST-CLA (Table 3). The optimal individual cut-off points for 3 allergens were obtained by ROC analysis (P < 0.05): class II for D. farinae, class I for D. pteronyssinus, and "trace" for the cockroach mix (Fig. 1).

Table 3.

Validity of MAST-CLA by different cut-off points compared to skin prick test

Fig. 1

ROC curves of MAST-CLA by different cut-off points compare to skin prick test about arthropod allergens. *: optimal cut-off point.

The skin prick test is generally considered to be a gold standard for use in the diagnosis of allergic diseases. MAST-CLA has been popularly applied as a diagnostic tool for simplicity, convenience, and safety. However, there have still been some arguments regarding the specificity and sensitivity of MAST-CLA due to an insufficient quantity of scientific studies (Yang et al., 1998; Lim et al., 2001; Nepper-Christensen et al., 2003). In fact, this problem has resulted in a controversy regarding the definition of a positive cut-off point, and many allergists have utilized a single unique cut-off point (Park et al., 1991; Agata et al., 1993; Lee et al., 1995; Yang et al., 1998; Hwang et al., 2001; Lim et al., 2001; Nepper-Christensen et al., 2003). In the Republic of Korea, a unique cut-off point for MAST-CLA, which was first proposed by Park et al. (1991) as class II, has been adopted by many Korean allergists. However, this has also caused a problem of low positive rates as compared with the results from other countries (Lee et al. 1995; Yang et al., 1998). Recently, Lim et al. (2001) reported that if a cut-off point of class I is adopted, the MAST-CLA positive rates will be increased significantly. The other problem is that there is only personal preference that currently guides allergists in their selection of a unique cut-off point for every allergen in the MAST-CLA, due to insufficient data regarding the proper points. In the present study, a ROC analysis has allowed us to suggest a meaningful and individual MAST-CLA cut-off point of 3 arthropod allergens, based on a scientific analysis.

ROC analysis can be utilized to determine the relationship between the sensitivity and specificity of a clinical test for allergy in accordance with different cut-off points. Unfortunately, changing the cut-off point to try and increase either sensitivity or specificity will usually result in a reduction in the other measures; thus, we need to determine the optimal cut-off point (Metz, 1978; Zweig and Campbell, 1993). The ROC curve is a plot of the sensitivity (proportion of individuals predicted to have a future adverse event among those who actually have an event) versus the 1-specificity (proportion of individuals predicted to have an event among those who remain event-free) for the range of cut-off points that can be utilized to predict whether or not a patient will experience an event. The area under the ROC curve is frequently utilized as an index of how well a scoring system is able to classify patients into 1 of the 2 medical alternatives (an area of 1 is perfect, 0.5 is non-informative) such as, death or survival, disease or non-disease (Streiner and Norman, 1995).

The ROC curve analysis allowed us to determine the optimal MAST-CLA cut-off points as compared to the skin prick test. The percentage agreements of the MAST-CLA in comparison with the skin prick test were significantly increased by these newly obtained individual optimal cut-off points. This finding can be considered to be, in part, a solution to the specificity and sensitivity problems of this test. We were able to acquire limited data only for 3 arthropod allergens in MAST-CLA. However, the panel contains 32 other allergens. Therefore, we suggest that further evaluations be conducted in order to determine the optimal individual cut-off points for each allergen in the MAST-CLA panel, via ROC analysis.