Reproducibility Made Easy: A Tool for Transparent and Standardized Reporting of Magnetic Resonance Spectroscopy Methods

Authors & Affiliations

Antonia Susnjar [1], Antonia Kaiser [2], Gianna Nossa [3], Dunja Simicic [4,5], Aaron Gudmundson [4,5]

[1] Athinoula A. Martinos Center for Biomedical Imaging, Department of Radiology Massachusetts General Hospital and Harvard Medical School, Boston MA, USA; [2] Animal Imaging and Technology core, CIBM Center for Biomedical Imaging, École polytechnique fédérale de Lausanne (EPFL), Lausanne, Switzerland; [3] School of Health Sciences, Purdue University, West Lafayette IN, USA; [4] Russell H. Morgan Department of Radiology and Radiological Science, The Johns Hopkins University School of Medicine, Baltimore MD, USA; [5] F. M. Kirby Research Center for Functional Brain Imaging, Kennedy Krieger Institute, Baltimore MD, USA

Abstract

Going beyond traditional imaging, Magnetic Resonance Spectroscopy (MRS) is a non-invasive method that assesses tissue metabolic profiles and it is widely used in research and increasingly applied in the clinical setting. Öz at el. (2) demonstrate that MRS is accessible on all standard MRI scanners. Through effective analysis, MRS offers vital insights into tissue composition, eliminating the need for traditional surgical biopsies. However, inconsistent analysis have impeded MRS validation and limited its clinical application. Decades of progress in MRS data processing and quantification development have led to numerous consensus papers, including the Minimum Reporting Standards in MRS (MRSinMRS) by Lin et al. (1). Despite the existence of this standardized reporting criteria, its adoption remains limited, affecting research rigor and reproducibility. A significant challenge lies in locating parameters within raw files due to nomenclature variations among MRI vendors. MRSinMRS consensus paper, published in 2021, received notable attention with 102 citations. However, only 43 references incorporated the MRSinMRS table, revealing a gap in implementation. To address this, the proposed solution is to develop an open-source toolbox "Reproducibility Made Easy." By utilizing a single MRS raw data file, this toolbox aims to automatically identify experimental parameters required for the MRSinMRS table and generate a methods section suitable for publications. "Reproducibility Made Easy" will be a Python-based application operating through an intuitive graphical user interface (GUI) built with Tkinter, requiring no coding experience. The application's user-friendly interface and adherence to the MRSinMRS consensus guidelines aim to empower scientists and clinicians, facilitating the easy sharing of methodologies and enhancing accessibility of research findings within the scientific community.

Project Updates

Preprint Submission: We have successfully submitted a preprint of our current research progress. Our paper is now accessible on Arxiv for you to read and share with interested peers. You can view the preprint here: Arxiv Preprint. 

Journal Submission: Our paper has been submitted to the Magnetic Resonance in Medicine Journal. This is a significant step towards peer-reviewed recognition of our research efforts. For more details about the journal, you can visit their website here: Magnetic Resonance in Medicine.

Upcoming Conference Presentation: We are gearing up to present our research at the International Society for Magnetic Resonance in Medicine (ISMRM) in Singapore from May 4-9. During this event, we will be engaging in a poster presentation where we look forward to gaining insightful feedback from the global research community. This feedback will be invaluable for further refining our project and considering potential add-ons for our REMY system based on expert suggestions.

Introduction

Motivation

The MRSinMRS consensus paper, published in 2021, has gained immediate traction, receiving 102 citations to date, even though only 43 references incorporated the MRSinMRS table, while the remaining 59 citations only acknowledged the paper (Figure 1).Nonetheless, the integration of MRSinMRS within the field has encountered a setback, primarily due to vendor-specific nomenclature of specific acquisition parameters. To overcome this obstacle, we have developed the Reproducibility Made Easy (RMY) toolbox, a simple solution that automates the population of the MRSinMRS table and generates the methods section.

Figure 1. Citation and Inclusion of MRSinMRS. Since its publication in 2021, the MRSinMRS consensus has been cited 102 times. However, only 43 of these citations have incorporated the reporting table, while the remaining references have solely cited the consensus itself.

Impact/Significance

The collective advancements in technical innovations within the field of Magnetic Resonance Spectroscopy (MRS), coupled with the consensus achieved among experts, aim to enhance the adoption of MRS by clinical researchers, psychiatrists, and non-specialized scientists. Despite this progress, challenges persist, with a significant concern being the limited implementation of standardized reporting criteria for MRS studies. It is crucial to comprehensively report MRS study methodology because the output data, referenced to total Creatine (tCr) in institutional units (IU) or in Millimolar/Millimolal (mM), is influenced by factors like voxel size, position, number of data points, averages, etc. While most journals in the MRI field require study overview checklists for various acquisition strategies and clinical trials (e.g., STARD17, CONSORT18, PRISMA19, and STROBE20), these checklists do not address reporting standards specifically for MRS. Consequently, novice researchers lack sufficient guidance, leading to difficulties in comparing studies and interpreting results accurately. This situation may contribute to the slow translation of MRS from the research setting to clinical applications due to the potential for inaccurate and inconsistent conclusions.

Hypotheses/Aims

Aim 1. Initially, our aim was to develop a standalone application using Tkinter, but this necessitates certain system dependencies like Python3. With the support of funding, we can leverage app development software to create a user-friendly interface, allowing for a simplified experience through drag-and-drop file functionality

Aim 2. Most important part of the application development is testing using actual MRS datasets (dicom, p file, rda, spar, sdat, nifti). Unfortunately, current IRB restrictions hinder our ability to procure testing data. Our proposed solution is to either license datasets from diverse vendors or research sites, or obtain datasets from multiple scanner vendors such as Phillips, GE, Siemens, and Bruker. The acquired data will be deidentified, open source, and made freely accessible for comprehensive testing within the research community.

Aim 3. The rapid increase in development of available processing MRS software presents an opportunity for considerable advancement within the MRS field. As mentioned before, a drawback resides in the need for conclusive reporting of methods. We are actively strategizing to extend our outreach to software developers working on the development of MRS processing tools. The essence of our plan involves integrating our open-source application into existing tools, creating a partnership that simplifies reporting methods for the MRS scientists and eliminates an aspect of what can be perceived as “black box” processing. Although many MRS processing tools have taken steps to ensure transparency and logical checkpoints throughout their processing pipelines, the inclusion of MRSinMRS outputs and the completion of the third section in the table, pertaining to data analysis methods and outcomes, will further enhance the credibility of published research. 

Aim 4. Our abstract has been selected for an oral presentation during “Reproducible Research” session and poster presentation during the “Magnetic Resonance Spectroscopy” session at the International Society for Magnetic Resonance in Medicine 2024. We are also actively working on publication that would be posted at Biorxiv.

Materials/Methods

RMY will be designed to be a standalone application for the creation of the MRSinMRS table and an MRS methods section that can be used for publications. The Python-based software (v. 3.11) will be used as an open-source and it will compiled into executables for different operating systems, so that it requires no experience by operating through an intuitive graphical user interface (GUI) built using Tkinter (3). The application will be operating-system (OS) agnostic, meaning it will operate uniformly across platforms. As an open-source application, RMY will be transparent, granting visibility into its underlying algorithms. One MRS raw data file will be required to read in acquisition parameters (e.g. repetition time (TR), echo time (TE)) and exports a comma separated values (.csv) file that provides the MRSinMRS table as well as a text document (Latex, word, and PDF format) with a completed and referenced Methods section as shown in Figure 2. Advantages of the application are evident in easy data input (supporting formats such as p, .ima, .rda, .dat, .spar, .sdat, fid, ser for GE, Siemens, Phillips, and Bruker MRS data formats), intuitive graphical user interface (GUI), and independence from any proprietary dependencies.

Figure 2. MRSinMRS application tool adapts to the input dataset. Leveraging spec2nii, the software self-adjusts automatically, configuring the optimal workflow for the given input dataset (4). Thus, no manual intervention is required to locate the inputs. One MRS raw data file is required to read acquisition parameters (such as the repetition time (TR), echo time (TE)) and export excel file that provides MRSinMRS table as well as a word document with completed and referenced Methods section.

Study type

• Observational Study - Information is gathered from healthy volunteers participating in the study to assess the functionality of newly developed application software.

Existing data

• Registration prior to creation of data: As of the date of submission of this research plan for preregistration, the data have not yet been collected, created, or realized.
• Registration prior to accessing the data: As of the date of submission, the data exist, but have not been accessed by you or your collaborators. Commonly, this includes data that has been collected by another researcher or institution.

Explanation of existing data

• The most important phase in the application development process involves rigorous testing with authentic human-subject MRS datasets, including various formats like dicom, p file, rda, spar, sdat, and nifti. Unfortunately, existing Institutional Review Board (IRB) restrictions present challenges in obtaining the necessary testing data. To overcome this hurdle, our proposed strategy is to either license datasets from a diverse range of vendors or research sites such as Mayo Clinic, MGH, and Krieger Institute or alternatively, obtain datasets from multiple scanner vendors at our institutions, including Phillips, GE, Siemens, and Bruker. It is crucial to note that all acquired data will undergo a meticulous deidentification process, ensuring compliance with ethical standards. Furthermore, we are committed to making these deidentified datasets open source and freely accessible on GitHub. This initiative aims to foster comprehensive testing opportunities within the research community, facilitating the refinement and optimization of our application.

Data collection procedures

• To validate the performance of our software, we intend to secure data through two primary avenues. Ideally, if budget constraints permit, we aim to expedite the process by obtaining licensed datasets from esteemed research sites like Mayo Clinic and MGH. It is worth noting, however, that the cost of licensing data can vary significantly among institutions and research sites. As a safer and more financially viable alternative, we plan to collect data from a group of healthy volunteers. This data will be solicited through an IRB-approved flyer, ensuring ethical and regulatory compliance. Our recruitment will be  disseminated at research institutions, including Purdue University, Johns Hopkins University, and Harvard University.

Sample size

• Within total of three hours of scanning, our target sample size is twelve (12) healthy participants that are not claustrophobic and ideally have had an MRI scan in the past. Each selected research site features a different MRI scanner vendor—Johns Hopkins employs Phillips, Purdue University utilizes GE, and Harvard uses Siemens. During each 30-minute scanning session, we plan to acquire MRS data through various sequences, including PRESS, sLASER, MEGA PRESS, and HERMES. This diverse set of sequences is strategically chosen to facilitate robust testing of our application. Each sequence necessitates distinct scanning parameters, resulting in diverse MRSinMRS table outputs.

Variables (optional)

• MRSinMRS table describes all the variables that will differ from each sequence (PRESS, sLASER, MEGA PRESS, and HERMES) resulting in a different population of the table.
• 

Budget

Costs

Proposed budget for the Reproducibility Made Easy toolbox. Majority of the funds would be used for the appropriate app development and data acquisition. Portion of the money would be used for graduate students to test all the acquired data and fix any potential bugs before the app is launched. Lastly, a portion of the funds would be used for the travel to the main conference in the MRI field in order to complete Aims 3 and 4. 

Acknowledgments

Our gratitude extends to the organizers and hosts of the inaugural MRS Hackathon 2023, who afforded us both space and the chance to interact with a diverse group of spectroscopists. This collaborative environment enabled us to conceive and actualize the RMY application. Their support was instrumental in converting our concept into a concrete solution, addressing the gap of reporting standards in the MRS field, which emerged during discussions at the MRS Workshop 2022 in Lausanne, Switzerland.

References

1. Lin A, Andronesi O, Bogner W, Choi I, Coello E, Cudalbu C, et al. Minimum Reporting Standards for in vivo Magnetic Resonance Spectroscopy (MRSinMRS): Experts’ consensus recommendations. NMR in Biomedicine. 2021 Feb 9;34(5).
2. Öz G, Deelchand DK, Wijnen JP, Mlynárik V, Xin L, Mekle R, et al. Advanced single voxel 1 H magnetic resonance spectroscopy techniques in humans: Experts’ consensus recommendations. NMR in Biomedicine. 2020 Jan 10
3. Moore AD. Python GUI Programming with Tkinter: Develop Responsive and Powerful GUI Applications with Tkinter. Accessed January 22, 2024. https://www.amazon.com/Python-GUI-Programming-Tkinter-applications-ebook/dp/B079HZPS7K
4. Clarke WT, Bell TK, Emir UE, et al. NIfTI-MRS: A standard data format for magnetic resonance spectroscopy. Magn Reson Med. 2022;88(6):2358-2370. doi:10.1002/MRM.29418