ANSI N42.45

Description / Abstract: This standard provides test methods for the evaluation of image quality of computed tomography (CT) security-screening systems. The quality of data for automated analysis is the primary concern. This standard does not address the system's ability to use this image data to automatically detect explosives or other threat materials, which is typically verified by an appropriate regulatory body.

Purpose

Security screening systems are generally used to scan parcels, including luggage, for the presence of illicit items such as explosives, drugs, or other contraband. Many of the screening systems currently used, particularly in transportation security applications, are based on CT imaging technology. Generally, as the parcel is transported through the system, the system collects a CT image of the parcel. These data are then subjected to automated analysis to determine whether a threat may be present or the parcel is considered clear. If the automated analysis determines a threat may be present, the image is often presented to a system operator who can override the automated decision, clearing the parcel, or referring it for further processing such as opening it and manually searching for threats.

Historically, government regulators have established evaluation procedures to determine whether a system's automated detection performance is adequate for use in applications within their borders. Typically, a vendor submits a copy of their product, including their software to the regulator's facility. The regulator runs a wide variety of parcels with threats inside through the system as well as parcels without threats that represent the typical stream of commerce. Detection and false alarm rates are determined and compared against performance criteria. If the criteria are met, the system is approved for use. This testing assures that the system is capable of meeting the required criteria, but how does one assure that all copies of the system meet the criteria? Normal manufacturing variability, quality control issues, or aging of the equipment may degrade performance versus what was observed on the article tested by the regulator. Replicating the original test on each machine in question is impractical. Transporting the regulator's threat set to a factory site or to locations where the machines are in use, presents significant security and in some cases safety concerns. This standard seeks to address this issue by specifying a suite of test methods that can be carried out on site without need for hazardous materials.

The performance testing carried out by the regulators essentially evaluates the combination of the system's ability to produce an image of the parcel along with its automatic analysis of that image data to reach a decision of threat or clear. The second part of this sequence, the analysis, is implemented through software. It should be noted that the regulators generally require that this software be designed so as to NOT evolve through use. The software used at all locations in the field must perform the same as the software did at the time of evaluation by the regulator. Configuration management of such software is a well known and straightforward art. Therefore, the real opportunity for performance variation comes from the imaging system that provides the data to the analysis software. If one can quantitatively validate that the quality of the image produced by the system in question is statistically equivalent to the image produced by the article evaluated by the regulator, one can be highly confident that the performance of the system in question is the same as what was approved by the regulator.

Purchasers of CT systems for security screening applications are generally not CT experts. Inconsistencies in methods for measuring seemingly standard image quality values (resolution, signal-to-noise, etc.) can confuse the potential user of such CT systems. Other standards exist for testing aspects of CT image quality, particularly in the medical field. This standard specifies a set of methods to apply in assessing CT image quality geared towards security screening. An application of this standard would be in the factory acceptance testing of equipment. The standard could be used to indicate whether the unit offered for sale produces the equivalent image quality as the unit that was tested by the cognizant regulatory agency. Since various image quality metrics can be traded off against one another and achieve similar levels of threat detection, it is generally not valid, in contrast to medical CT, to make model-to-model or manufacturer-tomanufacturer comparisons of individual test results for CT systems used for security-screening.

This standard does not address image quality presented to the operator. The image quality provided to the operator is not necessarily at the same level as that used by the automated analysis. The data may be degraded before presenting to the operator to decrease resources required for rendering the image on the screen. Conversely, the data used in the automated analysis may be intentionally degraded to control the computational loading of the analysis computer. The user of this standard may want to separately assess the quality of the images presented to the system's operator. A wide range of methods is available for this purpose including the use of visual line pair gauges and ASTM F792-08 [B1].<sup>1

1</sup> The numbers in brackets correspond to those of the bibliography in Annex A.