“Multiple Sclerosis Spinal Cord Lesions Segmentation from MultiSequence MRIs”

 

An uncommon but relevant task: spinal cord lesions segmentation

Multisequences: 4 sequences potentially available with various resolutions

Missing modalities: all four sequences not available for each patient at training and at testing

 

Challenge Rationale

Lesions segmentation: Let’s not forget the Spinal Cord

The identification of Multiple Sclerosis (MS) lesions on Magnetic Resonance Images (MRI) is a complex and mentally demanding task that often leads to an underestimation of disease activity, even for most experienced radiologists. There is thus a need for automated tools that can provide clinicians an aid for accurate and robust identification and quantification of MS lesions.

To date, the medical imaging community concentrated its efforts toward the segmentation of the lesions in brain MRI. For this purpose, over the past years, several challenges have been organized to assess the ability of automated methods to detect multiple sclerosis (MS) lesions as compared to manual delineation. These have allowed the community to explore innovative directions.

However, clinically, the presence of lesions in the spinal cord has a major prognostic value compared to brain lesions. Also their detection represents a hard task for radiologists. Indeed, MS lesion detection/segmentation in spinal cord MRI is a complex task due to specific characteristics (e.g. lack of sharp contrasts between healthy and pathological tissue, high occurrence of significant artifacts). As a result, despite its clinical importance, spinal cord MRI is currently under-exploited in patients with MS. Providing clinicians with tools capable of reliably identifying these spinal cord lesions would therefore be a major added-value.

A Multisequence “Missing-Modality” Setting

Spinal cord lesion segmentation raises a specific methodological challenge. Indeed, in clinical practice, it is highly recommended to acquire at least two sequences among a set of available sequences, without specific guidelines to date. In practice, depending on the center and context, any combination of existing MR sequences can be provided. In this challenge, that represents a concrete complex case of multisequence datasets, we focus on four commonly used sequences: the sagittal T2 (that is always provided in the challenge and will be considered as the reference to segment), the sagittal STIR, the sagittal PSIR and the 3D MP2RAGE that will be provided in different combinations.

In practice, the training set will consist of the following pairs of acquisitions: 50 pairs of T2 and STIR data, 25 pairs of T2 and PSIR data and 25 pairs of T2 and MP2RAGE data. This data originate from various 1.5T and 3T scanners from two different brands. The testing set (not provided to challengers) will consist of the following set: 40 pairs of T2 and PSIR data, 20 pairs of T2 and PSIR data, 20 pairs of T2 and MP2RAGE data and 20 triplets of T2, STIR and MP2RAGE data. Moreover, some of its data will originate from a third scanner brand.

Toward a detection-based evaluation

Only fully automated methods will be allowed. The challenge evaluation will thus be oriented toward detection-based metrics. In particular, two separate target metrics will be used independently in this challenge:

• mean case-wise F1-score, i.e. the harmonic mean between sensitivity and precision, averaged among cases.
• mean case-wise lesion-wise sensitivity while maintaining a false positive rate under a desired level of 2 false positive per case.

The code for the evaluation will be provided with the first release of the training set. The proposed methods will be asked to output a result for each of the two scores to optimize (but are allowed to provide the same value for the two outputs).

Important Dates & Submission Process

Important dates:

• Challenge Website open: July 2, 2024
• Registration period: November 2024 – May 2025
• Training data first release: November 1, 2024
• Training data updated version release: February 1, 2025
• Short paper submission deadline: June 15, 2025
• Docker submission deadline: June 15, 2025
• Announcement of the results: end of September 2025

Submission process:

The detailed instructions for the docker interface and submission will be provided soon. In a nutshell, the four following steps will be required :

1. Build a Docker or Singularity image containing the method,
2. Create a Boutiques descriptor of the tool,
3. Make the image and descriptor available to the VIP team,
4. Validate its integration with the VIP team.

Submitted pipelines will be integrated in the Virtual Imaging Platform (VIP), allowing
for their execution and evaluation on the test set (unrealeased to the challengers).

 

 

We are thankful to our institutions, partners and sponsors to have made this challenge possible.