QSM Reconstruction Challenge 2.0

-d

Winners of QSM reconstruction challenge 2.0

• Winner of robustness category: mcTFI algorithm, Yan Wen  from Cornell University
• Winner of best NRMSE category: wL1L2 algorithm, Carlos Milovic from Pontificia Universidad Catolica de Chile 
• Honorable mention to best RMSE category: TVmpnl algorithm, Carlos Milovic from Pontificia Universidad Catolica de Chile
• Honorable mention to best relative NMRSE improvement category: FINE algorithm, Jinwei Zhang from University of Cornell
• Honorable mention to highest absolute RMSE improvement category: QSMInvNet algorithm, Juan Liu from the Medical College of Wisconsin

The QSM Challenge 2.0 is both a “deployment” challenge and an “insight” challenge. We aim to determine the best algorithms for specific clinical or research applications (deployment) as well as we aim to gain broader insight on the current state-of-the-art and remaining issues of the field.

–

Two-stage challenge design

The challenge will consist of two stages.

–

In the first stage, we will provide two MRI datasets and no ground truth. This stage assesses the reconstruction performance of the algorithms in a realistic setting where a ground truth is not available. Participants will have to determine optimal algorithmic parameters based on numerical considerations.

The main difference between both datasets (Sim1 and Sim2) is the presence of an intra-hemispheric calcification, and different levels of susceptibility contrast between tissues.  Both datasets contain measurement noise with a strength typical for clinical scans. Processing pipelines for the two datasets should be identical (including regularization parameters or regularization optimization).

Submissions for stage 1 (devoid of ground truth) will be accepted until the 27th of July, 2nd of August, 2019, 23:59 EDT.

–

In the second stage, we will provide data of the two models at different SNR levels (low SNR, high SNR) along with the ground truth susceptibility distribution. In addition, we will provide the evaluation metrics code and tissue segmentation information. The goal of this stage is to determine the optimal reconstruction performance that can be achieved with a certain algorithm. While the quantitative performance in the first stage may be reduced by a sub-optimal parameter setting, the second stage will not be affected by such a limitation. Participants may use the ground truth to optimize their algorithmic parameters.

Stage 2 submissions (with ground truth and metrics code) will be accepted until the 27th of August, 2019, 23:59 EDT.

–

Data provided

–

Download the data here:

http://bit.ly/2019-qsm-challenge-stage1-download

–

–

The datasets provided in this challenge result from forward-simulations based on in vivo T1 and T2* brain maps acquired at 7 Tesla and a synthetically constructed magnetic susceptibility map at a resolution of 0.64mm isotropic (1).

–

The steady-state signal equation was used to compute the signal magnitude at 0.64mm isotropic resolution. The phase was computed assuming the field induced by the synthetic susceptibility distributions at 7T. Susceptibility values outside a predefined brain mask were set to zero (thus, no background field removal is required) and the values inside the mask were demeaned to zero. Last but not the least, prior to computing the field generated the susceptibility distribution, this was zero padded to avoid residual background fields..

–

The sequence parameters used for the simulation were: TR/TEs=50/4/12/20/28ms, flip angle = 15 degrees. Acquisition at a 1mm isotropic resolution was simulated by k-space cropping. For more information on the procedure used to create this phantom refer to the ISMRM abstract (2).

–

Each provided model folder contains the following data information:

– – 4D magnitude signal – Magnitude.nii.gz

– – 4D phase signal – Phase.nii.gz

– – Brain Mask –MaskBrainExtracted.nii.gz

– –Frequency map (Hz) computed using spatial unwrapping and echo difference combination (3) – Frequency.nii.gz (if you decide to start from this map but your algorithm requires data to be passed in radians at a given echo time, it can be converted using the following equation: frequency x TE x 2 π)

–

Evaluation criteria

–

Submitted susceptibility maps will be evaluated completely blinded with respect to several application-specific criteria:

–

Overall metric

Demean detrended RMSE, ddRMSE, over the whole brain (detrending aims at compensating for potential systematic underestimation and demeaning global shifts in susceptibility within the region considered)

–

Cortical gray and white matter analysis

• Overall precision: Detrended and demeaned root mean squared error (“ddRMSE”) with respect to the ground truth for all voxels in the cortical gray and white matter. This metric should highlight subtle susceptibility tissue variations

–

Deep gray matter analysis

• Overall precision: ddRMSE with respect to the ground truth for all voxels in the deep gray matter.

• Overall quantitative accuracy: Slope using all voxels in the deep gray matter

–

Venous blood analysis

• Overall precision: ddRMSE with respect to the ground truth for all voxels corresponding to veins and their immediate vicinity.

–

Calcification

• Magnetic moment: the magnetic moment of the calcification (pixel) will be compared to the magnetic moment of the ground truth (extent of calcification will be defined by a reconstruction specific thresholding)

• Streaking artifact level (standard deviation in a  region surrounding the calcification (after removing ground truth variance, divided by mean susceptibility within the calcification region)

–

All metrics above (except for those associated to the calcification) will be averaged across the two datasets prior to perform the ranking.

–

Submission Format

–

Please read the following section carefully. The filenames of your submitted solutions must follow the format described here. Submissions with a different format will not be analyzed!

–

Computed susceptibility maps must be in ppm units (Freq Ⓧ DipoleKernel / (γB0)), where γB0=42.7747892×7 [MHz], must have the same pixel dimensions and matrix size as the provided field maps, and must be submitted in nii format.

–

Participants will submit their reconstructed maps for each stage separately through a dedicated submission form as a single ZIP file. The filename of the zip file will be a unique 10-character identifier that you will define when filling out the submission form. For example, the zip-file name could be jwgt5l4gbd.zip.Here, jwgt5l4gbd is a unique identifier.

–

For stage 1, the ZIP file needs to contain two files (Sim1 and Sim2). The naming of the files must follow the following format:

–jwgt5l4gbd_Sim1_Step1.nii

–jwgt5l4gbd_Sim2_Step1.nii

Replace jwgt5l4gbd by your own unique identifier identical to the zip-file name.

–

For stage 2, the ZIP file needs to contain four files:

–jwgt5l4gbd_Sim1Snr1_Step2.nii

–jwgt5l4gbd_Sim2Snr1_Step2.nii

–jwgt5l4gbd_Sim1Snr2_Step2.nii

–jwgt5l4gbd_Sim2Snr2_Step2.nii

–

Do not include any identifying information in the zip file or filenames. We will analyze the submissions completely blinded.

–

Submission

Please submit your data here:

http://bit.ly/2019-qsm-challenge-stage1-submission

–

–

References

1- MWA Caan, et al, MP2RAGEME: T1, T2*, and QSM mapping in one sequence at 7 Tesla, Human Brain Mapping 40 (6), 1786-1798, 2019

1

2- JP Marques, et al, Towards QSM Challenge 2.0: Creation and Evaluation of a Realistic Magnetic Susceptibility Phantom, Proc. 27th Annual Meeting of the ISMRM, Montreal, Canada, 2019

1

3- Robinson, S. D. et al. An illustrated comparison of processing methods for MR phase imaging and QSM: combining array coil signals and phase unwrapping. NMR Biomed. 30, e3601 (2017).

1field removal is added