Greetings Alfonso and Karl!

I am using CONN for preprocessing of some rs-fMRI data using batch scripting in MATLAB. During the preprocessing of around 200 subjects, the process stopped due to an external interruption. Now in the gui, i am familiar that if i load the existing project, it will resume execution. How to ensure this in the batch script? I tried batch.Setup.isnew=0, along with setting overwrite and done to 0. However, the preprocessing steps began again for all subjects. 

Where am I going wrong? Kindly suggest a solution to this. Thanks!

Originally posted by Alfonso Nieto-Castanon:

Hi Karl, Sorry about that, and thanks for reporting this issue, that was a limitation/bug in the code where the batch fields Setup.subjects.groups and Setup.subjects.effects were not compatible with the Setup.add option. I have fixed that in the attached patch (this patch is for release 20b, to install it copy the attached file to your conn distribution folder overwriting the file with the same name there). Please let me know if you still run into any issues Best Alfonso Originally posted by Karl Lerud:

Hi Alfonso, I wanted to follow up on this. I am trying to follow your advice about adding more subjects, and only preprocessing the new subjects. I have done what you suggest, also trying to follow the instructions for Setup.add in the batch documentation. In addition to what it says there, I have also set Setup.RT to only have the values for the new subjects, and also changed Setup.nsubjects to be the number of new subjects (as opposed to all in the project), hopefully that is all correct so far. It is giving me the following error: Index exceeds the number of array elements (6). Error in conn_batch (line 1196) CONN_x.Setup.l2covariates.values{nsub}{nl2covariates}=(batch.Setup.subjects.groups(nsub)==ngroup); Error in multielectrodeCONNsetup_paper (line 331) conn_batch(conn_x) In this case, I am adding 6 new subjects, and that brings the total number in this project to 109. I checked, and internally to conn_batch, CONN_x.Setup.nsubjects is 109, not 6. I am not sure if that is right, or what to expect. I can provide more information or lines of code if that is necessary, hopefully this is enough information to diagnose what is going on. Thanks, Karl Originally posted by Alfonso Nieto-Castanon:

Hi Karl, Sorry about that, the batch.Setup.overwrite field only applies to the Setup step, similarly for the batch.Denoising.overwrite and batch.Analysis.overwrite fields. Unfortunately there is no equivalent 'overwrite' option for preprocessing, but you could either use the batch.Setup.add field to explicitly add/preprocess new subjects to an existing project, or manually select which subjects you want to preprocess/analyze using the general batch.subjects field. In your case I believe the simplest way would have been to modify your script to only include the information about your new 4 subjects (e.g. change the contents of Setup.functionals, Setup.structurals, etc.), and then use Setup.isnew=0 and Setup.add=1 so that CONN adds this new information to your project and preprocesses only those new subjects (and then for the Setup/Denoising/First-level steps, the overwrite option would suffice so there is no need to change anything else in the scripts there)  Hope this helps Alfonso Originally posted by Karl Lerud:

Thanks for the reply Alfonso. I have a project with ~95 subjects, and the only preprocessing I am using CONN for is functional_art. All other preprocessing is done before import to CONN. So I executed my script, and everything seems to run correctly to completion. Then I added 4 more subjects to the project, and simply changed Setup.isnew, Setup.overwrite, Denoising.overwrite, and Analysis.overwrite all to 0 (which were all 1 the first time around), and ran the script. This seems like the correct way to go according to what you said and the documentation.  The first thing that happened is that it ran the art preprocessing on all 99 subjects, overwriting and redoing this for the first 95. It also correctly did the art preprocessing for the 4 new subjects, but of course I didn't need to spend all that time redoing the first 95. Then it did the rest fairly quickly, so I think telling it not to overwrite the rest of the analysis worked. Is there a way I can tell if it only processed the 4 recent subjects, or if it re-processed everything? And at the very least, it seems like it shouldn't have to redo the art preprocessing for everyone. Is there anything I am doing wrong, and is there a way to make this work as expected? Thanks, Karl Originally posted by Alfonso Nieto-Castanon:

Hi Karl, The options are batch.Setup.overwrite, batch.Denoising.overwrite, and batch.Analysis.overwrite (set those fields to 0 when running the Setup, Denoising, or first-level analysis steps, respectively, in order to skip already-processed subject/ROIs; see all conn_batch field details in https://web.conn-toolbox.org/resources/c...) Best Alfonso Originally posted by Karl Lerud:

Could I bump this? My question is basically just how to skip already-processed subjects/ROIs in a CONN batch script. Thanks, Karl