Dear Alfonso,

When trying to run my analysis (5 participants, each with 6 resting state sessions) on a slurm computing cluster, the job does not continue past "Step 5/7: Importing ROI data".

Regarding the submission of the job, I think I have all the right settings: (I also ran the test, which finished correctly).

profile name: Slurm computer cluster

command used to submit a job:
sbatch --job-name=JOBLABEL --error=STDERR --output=STDOUT OPTS SCRIPT

check status: squeue --jobs=JOBID

I have not added any additional commands or submit settings. When I submit the job to the cluster, I start 5 parallel jobs, as suggested by the conn toolbox.


The ROI's I specified are subject and session specific grey matter, white matter and CSF images (generated by the segmentation during the preprocessing which I ran outside of conn using SPM12). This seems to work fine, as the segmentation step is skipped in step 2, so conn seems to find and use these images correctly. Additionally, I also specified 8 ROI's based on previous studies (I created these ROI's using MarsBar, by selecting a sphere of 4mm around peak voxel coordinates and writing the images to the same space / dimensions and voxel size as the normalized functional images, so I did not explicitly coregister them, but I assume that's not needed as they are both in the same MNI space). These 8 ROI's are the same for each participant.

I also selected the option to have the data set 1 (the functional images minus the 's' prefix).

When I submit the job, it starts running correctly through steps 1 - 4. When I check on the job status on the cluster, I see that each of the 5 jobs has about 9 minutes of run time to complete the set-up steps. After these 9 minutes, the job is still at step 5, importing the ROI data and then just runs out of time.

So I guess I have two questions, 1) Is this 9 minute window something that CONN determines and passes onto the cluster, or is this something our slurm cluster decides? And if conn determines this, can I extend this?

2) did I specify something wrong regarding the ROI's causing the job to be unable to get beyond step 5?

I'll attach the output / log files I have.

Kind regards,

Martijn Teuchies

Dear Martijn,

The 9-minute limit must be the default settings in your own cluster (CONN does not specify any time-limit on its own unless you explicitly include that info in the cluster configuration options). In a slurm cluster you can explicitly specify your requested job time-limit using the flag "--time=HH:MM:SS", so I would suggest simply editing in CONN's 'cluster settings' GUI your slurm profile to add in the field named 'additional submit settings . in-line ' the string

--time=02:00:00

and that should do the trick

Hope this helps
Alfonso


Originally posted by Martijn Teuchies:

Dear Alfonso,

Yes, that sounds like it should work, I will add that to the settings for submitting the job to the cluster.

Thank you very much!

Martijn

Dear Alfonso,

I also confronted this problem recently when I used CONN 18.a.  I have used the completely same settings in CONN 16 before, and it works well. The error information appears at the step 5/7: Importing ROI data. Strangely, this error is persistent even I did not import any ROI files. Could you help me to figure out the reasons.

The error information is as follows:

ERROR DESCRIPTION:

Error using horzcat
Dimensions of matrices being concatenated are not consistent.
Error in rex>rex_do (line 725)
cov1=[dataM,detrend(params.covariates,'constant')];
Error in rex (line 179)
[params.ROIdata,params.ROInames,params.ROIinfo.basis,params.ROIinfo.voxels,params.ROIinfo.files,params.ROIinfo.select,params.ROIinfo.trans]=rex_do(params,1);
Error in conn_rex (line 8)
[varargout{1:nargout}]=rex(varargin{:});
Error in conn_process (line 805)
[data{nroi1},namesroi{nroi},params]=conn_rex(Vsourcethis,Vmask{nroi}{min(nses,nsesstemp)},'summary_measure','eigenvariate','dims',CONN_x.Setup.rois.dimensions{nroi},'conju
Error in conn_process (line 16)
case 'setup', disp(['CONN: RUNNING SETUP STEP']); conn_process([0:4,4.5,5]);
Error in conn (line 4447)
else conn_process('setup'); ispending=false;
Error in conn_menumanager (line 120)
feval(CONN_MM.MENU{n0}.callback{n1}{1},CONN_MM.MENU{n0}.callback{n1}{2:end});

Thank you very much in advance!
Yang Yang