open-discussion > RE: Reproducibility analysis mICA
Apr 4, 2018  12:04 AM | Will Khan - King's College London
RE: Reproducibility analysis mICA
Hi Tawfik, 

Thanks very much for your help with this - much appreciated. 

Please find a point-by-point response to your previous email: 

First to your question, yes you have to run mica for all subfolders generated by split-half (number of mica commands = 2 x number of dimensionalities). One can use a bash script to schedule them all using fsl_sub if your fsl setup uses parallel processing. Or call mica one by one for each folder.

What do you mean by run mica for all subfolders generated by split-half?

Doesn't the mica script in the bin directory already do this when you supply the folder generated by split-half as the input? Could you provide an example? Thanks! 

To provide some clarity, I have outlined all the steps I have run below according to your suggestions in a previous email:

STEP 1: Generate all the relevant subfolders using py/splithalf.py script: 

  ./splithalf.py $working_dir/input_list 50 $working_dir/results/out_50

input_list is a text file of absolute paths to each .nii file like so: 

/Volumes/ImagingData_WillKhan/HCP_Data2018/raw_data/100307/MNINonLinear/Results/rfMRI_REST1_LR/rfMRI_REST1_LR_hp2000_clean.nii.gz
/Volumes/ImagingData_WillKhan/HCP_Data2018/raw_data/100307/MNINonLinear/Results/rfMRI_REST1_RL/rfMRI_REST1_RL_hp2000_clean.nii.gz
/Volumes/ImagingData_WillKhan/HCP_Data2018/raw_data/100307/MNINonLinear/Results/rfMRI_REST2_LR/rfMRI_REST2_LR_hp2000_clean.nii.gz
/Volumes/ImagingData_WillKhan/HCP_Data2018/raw_data/100307/MNINonLinear/Results/rfMRI_REST2_RL/rfMRI_REST2_RL_hp2000_clean.nii.gz

STEP 2: Run group ICA on PCC mask using bin/mica script:

export MICADIR=/Users/wasimkhan/Documents/mICA_Toolbox

./mica $working_dir/input_list $working_dir/results/out_50 -mask $working_dir/masks/PCC_and_Precuneus_mask_thr20_Harvard.nii.gz -dim 2 -extra -a concat -- sep_vn --nobet --bgthreshold=10 --tr=1 --report --mmthresh=0.5 --Oall -o $working_dir/results/mica_test_2comp

Note this command was run a total of 5 times for 2, 4, 6, 8 and 10 components.

Therefore, in the $working_dir/results directory I have the following directory structure: 

melodic_test_2comp....melodic_test_4comp....melodic_test_6comp and so on.

STEP 3: calculate a correlation using py/ic_corr.py script. Here I changed the relevant lines of code from your previous email and renamed the script py/ic_corr_updated.py. 

Usage: ic_corr.r in_prefix samples dims [just_read]
in_prefix path of the folder that contains samples folders
samples number of samplings
dims dimensionality range
[just_read] 0 is default. 1 will not perform fsl_cc and just reads any previously calculated cross-correlation file

./ic_corr_updated.py $working_dir/results/out_50 50 2-10 0

in_prefix:  I supply the folder generated by py/splithalf.py
samples: is this the number of permutations run in splithalf? I provide 50
dims: I supply 2-10 as I have already produced 5 different group ICA's (2,4,6,8,10) using the mica script  .

I get this error: 

File "./ic_corr_updated.py", line 102, in
if int(proc1.stdout.read().rstrip('\n')) != 1 or int(proc2.stdout.read().rstrip('\n')) != 1 or int(n1.stdout.read().rstrip('\n')) != int(dim) or int(n2.stdout.read().rstrip('\n')) != int(dim):
TypeError: a bytes-like object is required, not 'str'

line 102 reads like so: 

if int(proc1.stdout.read().rstrip('\n')) != 1 or int(proc2.stdout.read().rstrip('\n')) != 1 or int(n1.stdout.read().rstrip('\n'))!= int(dim) or int(n2.stdout.read().rstrip('\n')) != int(dim):



Could you please advise if all the following steps are correct or if I have made a mistake somewhere? 

Apologies for the rather long email - I wanted to be as detailed as possible so that it is easier in the debugging process :) 


Thanks so much!

Cheers, 

Will



Originally posted by Tawfik Moher Alsady:
Hi Will,

First to your question, yes you have to run mica for all subfolders generated by split-half (number of mica commands = 2 x number of dimensionalities). One can use a bash script to schedule them all using fsl_sub if your fsl setup uses parallel processing. Or call mica one by one for each folder.

About ic_corr.r, I guess you are using python 3. A small change has to be made to the script to make it compatible with python 3:
Line 72: change the following
dim=range(int(minmax[0]),int(minmax[1])+1)
to
dim=list(range(int(minmax[0]),int(minmax[1])+1))

Change line 178 and 180:
xtics=range(0,len(dims))
else:
xtics=range(0,len(dims), int(math.ceil(len(dims)/15.0)))

to:
xtics=list(range(0,len(dims)))
else:
xtics=list(range(0,len(dims), int(math.ceil(len(dims)/15.0))))

I hope it works now. If it works I will post an updated version with those changes.

Threaded View

TitleAuthorDate
Will Khan Mar 21, 2018
Florian Beissner Mar 22, 2018
Will Khan Mar 26, 2018
Tawfik Moher Alsady Mar 27, 2018
Will Khan Mar 28, 2018
Will Khan Mar 28, 2018
Tawfik Moher Alsady Mar 29, 2018
RE: Reproducibility analysis mICA
Will Khan Apr 4, 2018
Tawfik Moher Alsady Apr 9, 2018
Will Khan Apr 9, 2018