Example of Pre-Market Clinical Validation in MR

The primary dictionary definition of “validate” typically includes the idea of “to confirm”, or to ensure something is valid. Other entries emphasize the concept of an official sanction, confirmation, or approval of something. These suggest the recognition, establishment, or illustration of the worthiness or legitimacy of something. So, applying this term to clinical products prior to release (i.e., pre-market) might suggest that clinical validation means that the product has been clinically tested to confirm that it gives accurate and reliable results when used correctly. Clinical validation is valuable to ensure that a new feature such as Precise IQ Engine (PIQE) performs as expected when used as intended. But beyond that, validation provides a tool to identify missed opportunities and potential limitations.

The necessary testing in clinical validation for each individual new feature could vary greatly depending on that feature, its applications, and the expected claims about its performance. Some features may require minimal testing, while others, especially those relying on machine learning algorithms, may require complex, multi-site, multi-reader evaluations. As an illustrative example of clinical validation at Canon Medical Systems, the pre-market evaluation of PIQE in MRI is summarized here.

Precise IQ Engine: PIQE for MRI is a Deep Learning based technique that generates higher in-plane matrix images from images acquired with lower matrix while mitigating the ringing artifact associated with typical matrix expansion methods. PIQE has been validated, and is initially targeted, for brain and knee regions in Fast Spin Echo 2D imaging. As described below, bench testing and clinical evaluation studies performed in the validation for PIQE support indication for use in brain and knee imaging, however initial experiences with off label use of PIQE in other anatomies, at the medical professional’s discretion, have already shown great promise for clinical adoption.

Within the validation studies, PIQE underwent performance (bench) testing using computational analysis as well as a multi-center, randomized, blinded clinical image review.

Initial performance testing, based on images of the American College of Radiology (medium sized) resolution phantom, was performed using metrics of signal-to-noise ratio (SNR) and signal intensity profiles for ringing and sharpness. Results indicated that PIQE can generate higher in-plane matrix from lower matrix images and contributes to ringing artifact reduction and increase of sharpness. The performance testing also included evaluation in typical clinical images of brain and knee, based on the calculated metrics of Edge Slope Width (to evaluate image sharpness), Ringing Variable Mean (to evaluate ringing artifacts), SNR, and Contrast Change Ratio. This further bench testing confirmed that PIQE (as compared to other typical techniques) generates images with sharper edges while mitigating the smoothing and ringing effects and maintaining similar or better contrast and SNR, even when applied to tripling the matrix size in both dimensions (i.e., an overall factor of 9x).

Because PIQE is a fundamentally new technique, and because it relies on machine learning based algorithms, it was important to evaluate the performance more rigorously with a human observer study to quantify image quality as well as diagnostic quality. Within the clinical image review, images were anonymized, randomized, blinded (i.e., stripped of identifying information like reconstruction techniques), and distributed for detailed image evaluation and scoring to six US board-certified radiologists, three specializing in neuro imaging, and three in musculoskeletal imaging. The images were reconstructed with either the conventional method or PIQE at various matrix sizes and for various scan durations. For the image evaluation, characteristics such as ringing artifact, image sharpness, SNR, overall image quality (IQ) and feature conspicuity were scored by the radiologists using a modified 5-point Likert scale, where a score of three (3) or above is considered clinically acceptable. A total of 53 unique subjects, from multiple sites in USA, Europe, and Japan, were scanned at either 3T (Vantage Galan 3T) or 1.5T (Vantage Orian or Vantage Fortian) in brain or knee to provide the data sets comprising a total of 544 clinical scans, representing multiple orientations (axial, sagittal and coronal), and multiple contrast weightings (T1-/T2-/PD-weighted with/without Fat saturation) within the FSE2D family of pulse sequences. In particular, based on the claims of increasing matrix size, reducing scan time, and mitigating ringing, various comparisons between low matrix acquisition with high matrix reconstruction versus high matrix acquisition and standard filters were performed. The clinical image review, segregated and performed by field strength, resulted in all images scoring at, or above, the clinically acceptable level by the Radiologists. The reviewers exhibited a strong agreement at the “good” and “very good” level for all IQ metrics such as SNR, image sharpness, image ringing, overall IQ and feature conspicuity. Furthermore, the scores were statistically compared across the various combinations of acquired vs reconstructed matrix sizes (and PIQE vs traditional filters) to identify similarities and superiority.

In summary, the validation testing confirmed multiple indications for PIQE:

• PIQE generates higher spatial in-plane resolution images from lower resolution images, with the ability to triple the matrix dimensions in both in-plane directions.
• PIQE contributes to ringing artifact reduction, denoising, and increased sharpness.
• PIQE can accelerate scanning by reducing the acquisition matrix only, while maintaining clinical matrix size and image quality.
• PIQE benefits can be obtained on regular clinical protocols without requiring acquisition parameter adjustment.

Early clinical adoption and monitoring of PIQE’s clinical implementation has already begun. Please visit the other VISIONS articles to learn more.

While the PIQE feature provides a good example to observe the process of clinical validation, it is important to note that each new feature is considered on an individual basis when designing the validation studies for it. Some features may be fully validated using only phantoms and a few volunteers, and others may require many clinical subjects from multiple imaging sites. While some features may not require randomized, blinded physician image review, many do. Canon is fortunate to have a broad team of highly qualified reviewers – both regular Canon MR users and those who do not regularly read Canon MR images – to provide valuable feedback and comparisons. Canon is grateful for their time and effort spent helping to ensure the features provide accurate and reliable results as expected.