Lumos: Software for Multi-level Multi-reader Comparison of Cardiovascular Magnetic Resonance Late Gadolinium Enhancement Scar Quantification

Meier C., Eisenblätter M., Gielen S.

Cardiovascular magnetic resonance (CMR) has significantly revolutionized the comprehension and diagnosis of cardiac diseases, particularly through the utilization of late gadolinium enhancement (LGE) imaging for tissue characterization. LGE enables the visualization of expanded extracellular spaces in conditions such as fibrosis, fibrofatty tissue, or edema. The growing recognition of LGE’s prognostic capacity underscores its importance, evident in the increasing explicit recommendations within guidelines. Notably, the contemporary characterization of cardiomyopathies relies on LGE-based scar assessment by CMR to a large extent. This review describes the pattern and prognostic value of LGE in detail for various cardiac diseases. Despite its merits, establishing LGE as a reliable risk marker encounters challenges. Limitations arise from the fact that not all diseases show LGE, and it should always be analyzed in the context of all CMR sequences and the patient’s medical history. In summary, LGE stands as a robust indicator of adverse outcomes in diverse cardiovascular diseases. Its further integration into routine practice is desirable, necessitating widespread availability and application to accumulate both individual and scientific experience.

Hadler T., Ammann C., Wetzl J., Viezzer D., Gröschel J., Fenski M., Abazi E., Lange S., Hennemuth A., Schulz-Menger J.

Cardiovascular Magnetic Resonance (CMR) imaging is a growing field with increasing diagnostic utility in clinical routine. Quantitative diagnostic parameters are typically calculated based on contours or points provided by readers, e.g. natural intelligences (NI) such as clinicians or researchers, and artificial intelligences (AI). As clinical applications multiply, evaluating the precision and reproducibility of quantitative parameters becomes increasingly important. Although segmentation challenges for AIs and guidelines for clinicians provide quality assessments and regulation, the methods ought to be combined and streamlined for clinical applications. The goal of the developed software, Lazy Luna (LL), is to offer a flexible evaluation tool that is readily extendible to new sequences and scientific endeavours.An interface was designed for LL, which allows for comparing annotated CMR images. Geometric objects ensure precise calculations of metric values and clinical results regardless of whether annotations originate from AIs or NIs. A graphical user interface (GUI) is provided to make the software available to non-programmers. The GUI allows for an interactive inspection of image datasets as well as implementing tracing procedures, which follow statistical reader differences in clinical results to their origins in individual image contours. The backend software builds on a set of meta-classes, which can be extended to new imaging sequences and clinical parameters. Following an agile development procedure with clinical feedback allows for a quick implementation of new classes, figures and tables for evaluation.Two application cases present LL's extendibility to clinical evaluation and AI development contexts. The first concerns T1 parametric mapping images segmented by two expert readers. Quantitative result differences are traced to reveal typical segmentation dissimilarities from which these differences originate. The meta-classes are extended to this new application scenario. The second applies to the open source Late Gadolinium Enhancement (LGE) quantification challenge for AI developers "Emidec", which illustrates LL's usability as open source software.The presented software Lazy Luna allows for an automated multilevel comparison of readers as well as identifying qualitative reasons for statistical reader differences. The open source software LL can be extended to new application cases in the future.

Aquaro G.D., De Gori C., Faggioni L., Parisella M.L., Cioni D., Lencioni R., Neri E.

Abstract Late gadolinium enhancement (LGE) is the most relevant tool of cardiac magnetic resonance for tissue characterization, and it plays a pivotal role for diagnostic and prognostic assessment of cardiomyopathies. The pattern of presentation of LGE allows differential diagnosis between ischaemic and non-ischaemic heart disease with high diagnostic accuracy, and among different cardiomyopathies, specific presentation of LGE may help to make a diagnosis. Late gadolinium enhancement may be caused by conditions that significantly increase the interstitial space or, less frequently, that slow down Gd exit, like myocardial fibrosis. In chronic myocardial infarction, hypertrophic cardiomyopathies (HCM), dilated cardiomyopathy, Fabry disease, and other conditions, LGE is a marker of myocardial fibrosis, but also in patients with acute myocarditis where LGE may be also explained by the increase of interstitial space caused by interstitial oedema or by tissue infiltration of inflammatory cells. In cardiac amyloidosis, LGE represents myocardial fibrosis but the interstitial overload of amyloid proteins should also be considered as a potential cause of LGE. The identification of the pattern of presentation of LGE is also very important. In the ischaemic pattern, LGE always involves the subendocardial layer with more or less transmural extent, it is confluent, and every single scar should be located in the territory of one coronary artery. In the non-ischaemic pattern, LGE does not fulfil the previous criteria, being midwall, subepicardial, or mixed, not necessarily confluent or confined to a territory of one coronary artery. For cardiomyopathies, the exact pattern of non-ischaemic LGE is important. Quantitative analysis of LGE is required in some specific conditions as in HCM. Magnetic resonance imaging with LGE technique should be performed in every patient with suspect of cardiomyopathy. The lack of standardization of pulse sequence and mostly of quantification methods is the main limitation of LGE technique.

Navidi Z., Sun J., Chan R.H., Hanneman K., Al-Arnawoot A., Munim A., Rakowski H., Maron M.S., Woo A., Wang B., Tsang W.

Scar quantification on cardiovascular magnetic resonance (CMR) late gadolinium enhancement (LGE) images is important in risk stratifying patients with hypertrophic cardiomyopathy (HCM) due to the importance of scar burden in predicting clinical outcomes. We aimed to develop a machine learning (ML) model that contours left ventricular (LV) endo- and epicardial borders and quantifies CMR LGE images from HCM patients.We retrospectively studied 2557 unprocessed images from 307 HCM patients followed at the University Health Network (Canada) and Tufts Medical Center (USA). LGE images were manually segmented by two experts using two different software packages. Using 6SD LGE intensity cutoff as the gold standard, a 2-dimensional convolutional neural network (CNN) was trained on 80% and tested on the remaining 20% of the data. Model performance was evaluated using the Dice Similarity Coefficient (DSC), Bland-Altman, and Pearson’s correlation. The 6SD model DSC scores were good to excellent at 0.91 ± 0.04, 0.83 ± 0.03, and 0.64 ± 0.09 for the LV endocardium, epicardium, and scar segmentation, respectively. The bias and limits of agreement for the percentage of LGE to LV mass were low (-0.53 ± 2.71%), and correlation high (r = 0.92). This fully automated interpretable ML algorithm allows rapid and accurate scar quantification from CMR LGE images. This program does not require manual image pre-processing, and was trained with multiple experts and software, increasing its generalizability.

Heiberg E., Engblom H., Carlsson M., Erlinge D., Atar D., Aletras A.H., Arheden H.

The objective of the study was to investigate variability and agreement of the commonly used image processing method “n-SD from remote” and in particular for quantifying myocardial infarction by late gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR). LGE-CMR in tandem with the analysis method “n-SD from remote” represents the current reference standard for infarct quantification. This analytic method utilizes regions of interest (ROIs) and defines infarct as the tissue with a set number of standard deviations (SD) above the signal intensity of remote nulled myocardium. There is no consensus on what the set number of SD is supposed to be. Little is known about how size and location of ROIs and underlying signal properties in the LGE images affect results. Furthermore, the method is frequently used elsewhere in medical imaging often without careful validation. Therefore, the usage of the “n-SD” method warrants a thorough validation. Data from 214 patients from two multi-center cardioprotection trials were included. Infarct size from different remote ROI positions, ROI size, and number of standard deviations (“n-SD”) were compared with reference core lab delineations. Variability in infarct size caused by varying ROI position, ROI size, and “n-SD” was 47%, 48%, and 40%, respectively. The agreement between the “n-SD from remote” method and the reference infarct size by core lab delineations was low. Optimal “n-SD” threshold computed on a slice-by-slice basis showed high variability, n = 5.3 ± 2.2. The “n-SD from remote” method is unreliable for infarct quantification due to high variability which depends on different placement and size of remote ROI, number “n-SD”, and image signal properties related to the CMR-scanner and sequence used. Therefore, the “n-SD from remote” method should not be used, instead methods validated against an independent standard are recommended.

Lalande A., Chen Z., Pommier T., Decourselle T., Qayyum A., Salomon M., Ginhac D., Skandarani Y., Boucher A., Brahim K., de Bruijne M., Camarasa R., Correia T.M., Feng X., Girum K.B., et. al.

A key factor for assessing the state of the heart after myocardial infarction (MI) is to measure whether the myocardium segment is viable after reperfusion or revascularization therapy. Delayed enhancement-MRI or DE-MRI, which is performed 10 min after injection of the contrast agent, provides high contrast between viable and nonviable myocardium and is therefore a method of choice to evaluate the extent of MI. To automatically assess myocardial status, the results of the EMIDEC challenge that focused on this task are presented in this paper. The challenge's main objectives were twofold. First, to evaluate if deep learning methods can distinguish between non-infarct and pathological exams, i.e. exams with or without hyperenhanced area. Second, to automatically calculate the extent of myocardial infarction. The publicly available database consists of 150 exams divided into 50 cases without any hyperenhanced area after injection of a contrast agent and 100 cases with myocardial infarction (and then with a hyperenhanced area on DE-MRI), whatever their inclusion in the cardiac emergency department. Along with MRI, clinical characteristics are also provided. The obtained results issued from several works show that the automatic classification of an exam is a reachable task (the best method providing an accuracy of 0.92), and the automatic segmentation of the myocardium is possible. However, the segmentation of the diseased area needs to be improved, mainly due to the small size of these areas and the lack of contrast with the surrounding structures.

Hadler T., Wetzl J., Lange S., Geppert C., Fenski M., Abazi E., Gröschel J., Ammann C., Wenson F., Töpper A., Däuber S., Schulz-Menger J.

Cardiovascular magnetic resonance imaging is the gold standard for cardiac function assessment. Quantification of clinical results (CR) requires precise segmentation. Clinicians statistically compare CRs to ensure reproducibility. Convolutional Neural Network developers compare their results via metrics. Aim: Introducing software capable of automatic multilevel comparison. A multilevel analysis covering segmentations and CRs builds on a generic software backend. Metrics and CRs are calculated with geometric accuracy. Segmentations and CRs are connected to track errors and their effects. An interactive GUI makes the software accessible to different users. The software’s multilevel comparison was tested on a use case based on cardiac function assessment. The software shows good reader agreement in CRs and segmentation metrics (Dice > 90%). Decomposing differences by cardiac position revealed excellent agreement in midventricular slices: > 90% but poorer segmentations in apical (> 71%) and basal slices (> 74%). Further decomposition by contour type locates the largest millilitre differences in the basal right cavity (> 3 ml). Visual inspection shows these differences being caused by different basal slice choices. The software illuminated reader differences on several levels. Producing spreadsheets and figures concerning metric values and CR differences was automated. A multilevel reader comparison is feasible and extendable to other cardiac structures in the future.

Holtackers R.J., Emrich T., Botnar R.M., Kooi M.E., Wildberger J.E., Kreitner K.-.

Background Late gadolinium enhancement (LGE) is a widely used cardiac magnetic resonance imaging (MRI) technique to diagnose a broad range of ischemic and non-ischemic cardiomyopathies. Since its development and validation against histology already more than two decades ago, the clinical utility of LGE and its span of applications have increased considerably. Methods In this review we will present the basic concepts of LGE imaging and its diagnostic and prognostic value, elaborate on recent developments and emerging methods, and finally discuss future prospects. Results Continuous developments in 3 D imaging methods, motion correction techniques, water/fat-separated imaging, dark-blood methods, and scar quantification improved the performance and further expanded the clinical utility of LGE imaging. Conclusion LGE imaging is the current noninvasive reference standard for the assessment of myocardial viability. Improvements in spatial resolution, scar-to-blood contrast, and water/fat-separated imaging further strengthened its position. Key Points:  Citation Format

van der Velde N., Hassing H.C., Bakker B.J., Wielopolski P.A., Lebel R.M., Janich M.A., Kardys I., Budde R.P., Hirsch A.

The aim of this study was to assess the effect of a deep learning (DL)–based reconstruction algorithm on late gadolinium enhancement (LGE) image quality and to evaluate its influence on scar quantification. Sixty patients (46 ± 17 years, 50% male) with suspected or known cardiomyopathy underwent CMR. Short-axis LGE images were reconstructed using the conventional reconstruction and a DL network (DLRecon) with tunable noise reduction (NR) levels from 0 to 100%. Image quality of standard LGE images and DLRecon images with 75% NR was scored using a 5-point scale (poor to excellent). In 30 patients with LGE, scar size was quantified using thresholding techniques with different standard deviations (SD) above remote myocardium, and using full width at half maximum (FWHM) technique in images with varying NR levels. DLRecon images were of higher quality than standard LGE images (subjective quality score 3.3 ± 0.5 vs. 3.6 ± 0.7, p < 0.001). Scar size increased with increasing NR levels using the SD methods. With 100% NR level, scar size increased 36%, 87%, and 138% using 2SD, 4SD, and 6SD quantification method, respectively, compared to standard LGE images (all p values < 0.001). However, with the FWHM method, no differences in scar size were found (p = 0.06). LGE image quality improved significantly using a DL-based reconstruction algorithm. However, this algorithm has an important impact on scar quantification depending on which quantification technique is used. The FWHM method is preferred because of its independency of NR. Clinicians should be aware of this impact on scar quantification, as DL-based reconstruction algorithms are being used. • The image quality based on (subjective) visual assessment and image sharpness of late gadolinium enhancement images improved significantly using a deep learning–based reconstruction algorithm that aims to reconstruct high signal-to-noise images using a denoising technique. • Special care should be taken when scar size is quantified using thresholding techniques with different standard deviations above remote myocardium because of the large impact of these advanced image enhancement algorithms. • The full width at half maximum method is recommended to quantify scar size when deep learning algorithms based on noise reduction are used, as this method is the least sensitive to the level of noise and showed the best agreement with visual late gadolinium enhancement assessment.

Lalande A., Chen Z., Decourselle T., Qayyum A., Pommier T., Lorgis L., de la Rosa E., Cochet A., Cottin Y., Ginhac D., Salomon M., Couturier R., Meriaudeau F.

One crucial parameter to evaluate the state of the heart after myocardial infarction (MI) is the viability of the myocardial segment, i.e., if the segment recovers its functionality upon revascularization. MRI performed several minutes after the injection of a contrast agent (delayed enhancement-MRI or DE-MRI) is a method of choice to evaluate the extent of MI, and by extension, to assess viable tissues after an injury. The Emidec dataset is composed of a series of exams with DE-MR images in short axis orientation covering the left ventricle from normal cases or patients with myocardial infarction, with the contouring of the myocardium and diseased areas (if present) from experts in the domains. Moreover, classical available clinical parameters when the patient is managed by an emergency department are provided for each case. To the best of our knowledge, the Emidec dataset is the first one where annotated DE-MRI are combined with clinical characteristics of the patient, allowing the development of methodologies for exam classification as for exam quantification.

Schulz-Menger J., Bluemke D.A., Bremerich J., Flamm S.D., Fogel M.A., Friedrich M.G., Kim R.J., von Knobelsdorff-Brenkenhoff F., Kramer C.M., Pennell D.J., Plein S., Nagel E.

With mounting data on its accuracy and prognostic value, cardiovascular magnetic resonance (CMR) is becoming an increasingly important diagnostic tool with growing utility in clinical routine. Given its versatility and wide range of quantitative parameters, however, agreement on specific standards for the interpretation and post-processing of CMR studies is required to ensure consistent quality and reproducibility of CMR reports. This document addresses this need by providing consensus recommendations developed by the Task Force for Post-Processing of the Society for Cardiovascular Magnetic Resonance (SCMR). The aim of the Task Force is to recommend requirements and standards for image interpretation and post-processing enabling qualitative and quantitative evaluation of CMR images. Furthermore, pitfalls of CMR image analysis are discussed where appropriate. It is an update of the original recommendations published 2013.

Kramer C.M., Barkhausen J., Bucciarelli-Ducci C., Flamm S.D., Kim R.J., Nagel E.

This document is an update to the 2013 publication of the Society for Cardiovascular Magnetic Resonance (SCMR) Board of Trustees Task Force on Standardized Protocols. Concurrent with this publication, 3 additional task forces will publish documents that should be referred to in conjunction with the present document. The first is a document on the Clinical Indications for CMR, an update of the 2004 document. The second task force will be updating the document on Reporting published by that SCMR Task Force in 2010. The 3rd task force will be updating the 2013 document on Post-Processing. All protocols relative to congenital heart disease are covered in a separate document. The section on general principles and techniques has been expanded as more of the techniques common to CMR have been standardized. A section on imaging in patients with devices has been added as this is increasingly seen in day-to-day clinical practice. The authors hope that this document continues to standardize and simplify the patient-based approach to clinical CMR. It will be updated at regular intervals as the field of CMR advances.

Gräni C., Eichhorn C., Bière L., Kaneko K., Murthy V.L., Agarwal V., Aghayev A., Steigner M., Blankstein R., Jerosch-Herold M., Kwong R.Y.

Although the presence of late gadolinium enhancement (LGE) using cardiovascular magnetic resonance imaging (CMR) is a significant discriminator of events in patients with suspected myocarditis, no data are available on the optimal LGE quantification method. Six hundred seventy consecutive patients (48 ± 16 years, 59% male) with suspected myocarditis were enrolled between 2002 and 2015. We performed LGE quantitation using seven different signal intensity thresholding methods based either on 2, 3, 4, 5, 6, 7 standard deviations (SD) above remote myocardium or full width at half maximum (FWHM). In addition, a LGE visual presence score (LGE-VPS) (LGE present/absent in each segment) was assessed. For each of these methods, the strength of association of LGE results with major adverse cardiac events (MACE) was determined. Inter-and intra-rater variability using intraclass-correlation coefficient (ICC) was performed for all methods. Ninety-eight (15%) patients experienced a MACE at a medium follow-up of 4.7 years. LGE quantification by FWHM, 2- and 3-SD demonstrated univariable association with MACE (hazard ratio [HR] 1.05, 95% confidence interval [CI]:1.02–1.08, p = 0.001; HR 1.02, 95%CI:1.00–1.04; p = 0.001; HR 1.02, 95%CI: 1.00–1.05, p = 0.035, respectively), whereas 4-SD through 7-SD methods did not reach significant association. LGE-VPS also demonstrated association with MACE (HR 1.09, 95%CI: 1.04–1.15, p < 0.001). In the multivariable model, FWHM, 2-SD methods, and LGE-VPS each demonstrated significant association with MACE adjusted to age, sex, BMI and LVEF (adjusted HR of 1.04, 1.02, and 1.07; p = 0.009, p = 0.035; and p = 0.005, respectively). In these, FWHM and LGE-VPS had the highest degrees of inter and intra-rater reproducibility based on their high ICC values. FWHM is the optimal semi-automated quantification method in risk-stratifying patients with suspected myocarditis, demonstrating the strongest association with MACE and the highest technical consistency. Visual LGE scoring is a reliable alternative method and is associated with a comparable association with MACE and reproducibility in these patients. NCT03470571 . Registered 13th March 2018. Retrospectively registered.

Bernard O., Lalande A., Zotti C., Cervenansky F., Yang X., Heng P., Cetin I., Lekadir K., Camara O., Gonzalez Ballester M.A., Sanroma G., Napel S., Petersen S., Tziritas G., Grinias E., et. al.

Delineation of the left ventricular cavity, myocardium, and right ventricle from cardiac magnetic resonance images (multi-slice 2-D cine MRI) is a common clinical task to establish diagnosis. The automation of the corresponding tasks has thus been the subject of intense research over the past decades. In this paper, we introduce the “Automatic Cardiac Diagnosis Challenge” dataset (ACDC), the largest publicly available and fully annotated dataset for the purpose of cardiac MRI (CMR) assessment. The dataset contains data from 150 multi-equipments CMRI recordings with reference measurements and classification from two medical experts. The overarching objective of this paper is to measure how far state-of-the-art deep learning methods can go at assessing CMRI, i.e., segmenting the myocardium and the two ventricles as well as classifying pathologies. In the wake of the 2017 MICCAI-ACDC challenge, we report results from deep learning methods provided by nine research groups for the segmentation task and four groups for the classification task. Results show that the best methods faithfully reproduce the expert analysis, leading to a mean value of 0.97 correlation score for the automatic extraction of clinical indices and an accuracy of 0.96 for automatic diagnosis. These results clearly open the door to highly accurate and fully automatic analysis of cardiac CMRI. We also identify scenarios for which deep learning methods are still failing. Both the dataset and detailed results are publicly available online, while the platform will remain open for new submissions.

Karim R., Bhagirath P., Claus P., James Housden R., Chen Z., Karimaghaloo Z., Sohn H., Lara Rodríguez L., Vera S., Albà X., Hennemuth A., Peitgen H., Arbel T., Gonzàlez Ballester M.A., Frangi A.F., et. al.

Studies have demonstrated the feasibility of late Gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) imaging for guiding the management of patients with sequelae to myocardial infarction, such as ventricular tachycardia and heart failure. Clinical implementation of these developments necessitates a reproducible and reliable segmentation of the infarcted regions. It is challenging to compare new algorithms for infarct segmentation in the left ventricle (LV) with existing algorithms. Benchmarking datasets with evaluation strategies are much needed to facilitate comparison. This manuscript presents a benchmarking evaluation framework for future algorithms that segment infarct from LGE CMR of the LV. The image database consists of 30 LGE CMR images of both humans and pigs that were acquired from two separate imaging centres. A consensus ground truth was obtained for all data using maximum likelihood estimation. Six widely-used fixed-thresholding methods and five recently developed algorithms are tested on the benchmarking framework. Results demonstrate that the algorithms have better overlap with the consensus ground truth than most of the n-SD fixed-thresholding methods, with the exception of the Full-Width-at-Half-Maximum (FWHM) fixed-thresholding method. Some of the pitfalls of fixed thresholding methods are demonstrated in this work. The benchmarking evaluation framework, which is a contribution of this work, can be used to test and benchmark future algorithms that detect and quantify infarct in LGE CMR images of the LV. The datasets, ground truth and evaluation code have been made publicly available through the website: https://www.cardiacatlas.org/web/guest/challenges.