I have a fairly basic mathematical and implementational understanding of ML algorithms and CNNs, and I am trying to think of an approach for this task: https://www.kaggle.com/c/rsna-miccai-brain-tumor-radiogenomic-classification/data?select=test.

The "data" section explains the task and also gives a preview of the dataset.

Doubt on general Implementation approach:

From what I understand, we have 4 input parameters: FLAIR , T1W, T1Gd, and T2W. Based on these 4 parameters, we have to compute the "MGMT status"(Presence of MGMT), which is binary, i,e takes on values ($0/1$).

We can use a CNN architecture that uses a sigmoid activation function it's last layer (to get the output in $(0,1)$).

Now, I am aware that images are fed to neural networks as inputs, eg. in object detection programs, however in those examples a single image is fed as an input, and features are subsequently extracted by the network.

How should I approach my particular case, where I have multiple images as input parameters?

Also, what exactly are the top level folders 00000,00002..etc in the train dataset corresponding to? I initially thought they are behaving as "patient no.s" (i.e training examples), but then shouldn't the 4 subfolders in each top level folder have only one image each? One FLAIR image, one T1W image..etc. However there are multiple FLAIR (~200) , T1W,..etc images corresponding to each top level folder.


After Serali's answer , I have some insight as to how we should approach the preprocessing. The top level folders 00000,00002,.. etc are indeed functioning as "patient no.s", and this is what the file train_labels.csv refers to as "BraTS21ID".

Now for example we pick the folder 00000. There are 4 sub folders in it, and each sub folder contains various images: for eg., the FLAIR subfolder contains 200 images. These 200 images represent slices of the FLAIR MRI, and I suppose they are to be stacked together to give the full "FLAIR MRI".

Similarly, if we do this for all the images in the remaining subfolders, we will finally get 4 images per top-level folder, corresponding to FLAIR,T1W,T1gd,T2W. I have found a script that seems to do this:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
import os
import json
import glob
import random
import collections
import cv2
import pydicom
from pydicom.pixel_data_handlers.util import apply_voi_lut

from matplotlib import animation, rc
rc('animation', html='jshtml')

train_df = pd.read_csv("../input/rsna-miccai-brain-tumor-radiogenomic-classification/train_labels.csv")
def load_dicom(path):
    dicom = pydicom.read_file(path)
    data = dicom.pixel_array
    data = data - np.min(data)
    if np.max(data) != 0:
        data = data / np.max(data)
    data = (data * 255).astype(np.uint8)
    return data

def visualize_sample(
    types=("FLAIR", "T1w", "T1wCE", "T2w")
    plt.figure(figsize=(10, 3))
    patient_path = os.path.join(
    for i, t in enumerate(types, 1):
        t_paths = sorted(
            glob.glob(os.path.join(patient_path, t, "*")), 
            key=lambda x: int(x[:-4].split("-")[-1]),
        data = load_dicom(t_paths[int(len(t_paths) * slice_i)])
        plt.subplot(1, 4, i)
        plt.imshow(data, cmap="gray")
        plt.title(f"{t}", fontsize=10)
_brats21id = train_df.iloc[0]["BraTS21ID"] #patient ID 0 

This script displays the 4 final images for the patient ID 00000. Output:

enter image description here

Now, I am a bit confused as to what further I should do with these 4 images. Combine them again into one by "3-D stacking"? I have a piece of code that seems to finally obtain 1 image only, for each top level folder:

https://www.kaggle.com/ammarnassanalhajali/brain-tumor-3d-training (the "Functions to load images" section)

Main Question

I have some vague ideas about what the 2 scripts are doing, but I want to understand them concretely, since I wish to write my own code for this purpose.

Script 1

  1. The load_dicom() function is first reading the .dcm images using pydicom.read(), but whats going on with the manipulation of the data variable? Like data=data-np.min(data), data= (data*255).astype(np.uint8)
  2. What is going on in the function visualise_sample()?

Script 2

  1. What exactly is the function load_dicom_image() doing? I get the resizing part, but I don't understand the rotate and voi_lut parameters.

  2. The main function seems to be load_dicom_images_3d. I have no idea what this is doing. I think the glob.glob() is being used to iterate over all the files in the train section, but I don't understand the use of lambda subsequently, and I have no clue what the approach is with defining p1,p2 middle, etc.

  3. Can the first script be modified to achieve what the second one was doing?


1 Answer 1


Detailed information on the dataset is available in the "Data Description" text. Each number is indeed a separate case or patient if you will. Think of the numbers given as the patient ID numbers. So patients numbered 0, 2, 3, 5 etc are in the training dataset, patients numbered 1, 13, 15, 27 etc are in the public test - or validation dataset. Whatever number left is in the private test dataset that is used for grading.

For each patient there are four different types of MRI images for each case, that is what Flair, T1W, T1Gd, T2 are. Each MRI is made of several images called "slices", which are the several images in each of these folders (FLAIR, T1W, T1Gd, T2). What I mean is the collection of images of a FLAIR folder is what makes a FLAIR MRI.

"BraTS21ID" looks like a reference to the study itself that the publication was named after: "The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification". Competition is probably based on this paper.

As per implementation, I am not sure what is allowed or what is required in this particular Kaggle competition, but one simple approach is probably using 3D convolutions. I did not check the details of each folder but usually an MRI consist of several "slices", which at the end of the day are put together (I mean literally placed side by side, like this) to get a single image. You can try to do this for all images, and at the end of the day, place each final image created on top of each other to obtain a 3D tensor.

Let me be a little more specific about the process: My main goal is to obtain a final image of this form. If all folders contain 200 images, it is probably a good idea to lower this number. At the end of the day, we want to put all images side by side to get a single NxM dimensional image, where N,M represent number of small images (slices) in each row, column. So, for example we can get a 5X5 image - an image that represents one of the MRI types - for example FLAIR - that contains 25 small images within. In order to lower the number of images this way, one can simply take the average of each pixel value in every 8 images - So images numbered 1-8 would form a single image, 9-16 another and so on.

We can go ahead and apply the same procedure to all 4 types of MRIs. So, we end up with 4 final images for each MRI type, each of which has 5 rows and 5 columns of smaller images - the slices we took average of. We can now stack those on top of each other to get a 3 Dimensional tensor, which can be used as an input to 3D Convolutions.

  • $\begingroup$ But then shouldn't the structure be like this: there are top level folders such as 00000,00001... etc, inside each top level folder there are 4 folders, FLAIR,T1W,.. and inside each folders there is only 1 image..? $\endgroup$
    – satan 29
    Sep 26, 2021 at 8:34
  • $\begingroup$ I don't understand. Top level folders are titled with numbers in order. They seem to be spread amongst train, validation and test - latest kept separate for grading. Each numbered folder has 4 folders with names FLAIR etc and each of those folders have several images. Each image corresponds to a slice. Each MRI is made of several such slices. What seems to be the problem? I am editing my answer to give more details. $\endgroup$
    – serali
    Sep 26, 2021 at 8:57
  • $\begingroup$ for instance, the folder 00001 has a folder "FLAIR", which contains 203 images. Are all the 203 images "slices" for the FLAIR parameter of the MRI of patient -1 ? $\endgroup$
    – satan 29
    Sep 26, 2021 at 9:06
  • $\begingroup$ Yes. But FLAIR is a type of MRI. T1W, T1Gd, T2 are all separate types of MRI. So you have 4 different MRIs for each patient. $\endgroup$
    – serali
    Sep 26, 2021 at 9:13
  • $\begingroup$ So the "input image" for patient 1 will be all the 203 images of the FLAIR part+ all the images of the T1W part.....etc combined? $\endgroup$
    – satan 29
    Sep 26, 2021 at 9:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.