help
help > RE: New User and Batch Scripting
Dec 10, 2015 01:12 PM | Jalmar Teeuw
RE: New User and Batch Scripting
Hi Nathaniel,
We use CONN batch scripting to run our dataset in batches (also for parallel processing) and different stages separately (to check the intermediate results). To continue with the next stage you'll only need to define the parameters of that stage. Below is an example for performing the Second-Level analysis stage after the project has been processed all the way up to and including the First-Level analysis stage.
The idea is that for each stage we clear the batch variable (see first line), otherwise CONN will repeat the previously defined stages. If there is a gap between running the different stages (e.g. you quit Matlab between stages) you need to load the CONN_x variable stored in your project file as global variable (see second and third line), otherwise you'll see error message such as 'no subject defined' or 'out of bound indexing' since the project is not automatically loaded by CONN batch script. After that, we define the batch structure, point it to our existing project file, define the parameters for the next stage (in this case Second-Level analysis in batch.Results.*) and call conn_batch to perform the next stage. The parameters we do not define are filled in by CONN with default values. In our case we currently leave most options at default since we have a very simple setup of resting-state data of only healthy subjects. Also, note we only perform ROI-to-ROI and seed-to-voxel, and no voxel-to-voxel analysis, so the second part is commented out.
One thing to point out is that you may need to merge you individual projects into a single project for the second-level analysis. At the moment we perform this step manually in CONN: open CONN, go the Project menu and select 'Merge'; set the number of projects to merge, select 'Create new project' and add all the project you would like to merge. After you click on Done (lower left corner) CONN will ask you were to create the new project and copy all the necessary files to the next project folder. It should be possible to automate this step; check the conn_merge function in conn_merge.m. For the second-level analysis, with all your subjects in a single project, you may no longer need to use CONN batch script above but can instead run the analysis from the CONN GUI; open the merged project file in CONN and go to the Second-Level Results tab.
Hope this helps,
Regards,
Jalmar
We use CONN batch scripting to run our dataset in batches (also for parallel processing) and different stages separately (to check the intermediate results). To continue with the next stage you'll only need to define the parameters of that stage. Below is an example for performing the Second-Level analysis stage after the project has been processed all the way up to and including the First-Level analysis stage.
clear batch;
global CONN_x;
load(path_to_conn_project_file, 'CONN_x');
%% open single batch file
batch.filename = path_to_conn_project_file;
batch.parallel.N = 0; % run locally = 0
%% BATCH Results (Second-Level ROI-to-ROI and seed-to-voxel)
batch.Results.done = 1;
batch.Results.overwrite = 1;
batch.Results.analysis_number = [1]; % sequential indexes inditify each set of independent analysis, alternative a string identifying the analysis name
%batch.Results.foldername = ''; % folder to store the results
%batch.Results.between_subjects.effect_names = {}; % cell array of second-level effect names
%batch.Results.between_subjects.contract = []; % constrast vector, same size as effect_names
%batch.Results.between_conditions.effect_names = {}; % cell array of condition names (as in Setup.conditions.names)
%batch.Results.between_conditions.contract = []; % constrast vector, same size as effect_names
%batch.Results.between_sources.effect_names = {}; % cell array of source names (as in Analysis.regressors, typically appended wiht _N_M, see documentation)
%batch.Results.between_sources.contract = []; % constrast vector, same size as effect_names
%% BATCH Results (Second-Level voxel-to-voxel)
%batch.Results.done = 0;
%batch.Results.overwrite = 1;
%batch.Results.foldername = ''; % folder to store the results
%batch.Results.between_subjects.effect_names = {}; % cell array of second-level effect names
%batch.Results.between_subjects.contract = []; % constrast vector, same size as effect_names
%batch.Results.between_conditions.effect_names = {}; % cell array of condition names (as in Setup.conditions.names)
%batch.Results.between_conditions.contract = []; % constrast vector, same size as effect_names
%batch.Results.between_measures.effect_names = {}; % cell array of source names (as in Analysis.measures)
%batch.Results.between_measures.contract = []; % constrast vector, same size as effect_names
%% run Analysis processing step
tic;
conn_batch(batch);
disp(['CONN batch running Results took ', num2str(round(toc / 60)), ' minutes']);
global CONN_x;
load(path_to_conn_project_file, 'CONN_x');
%% open single batch file
batch.filename = path_to_conn_project_file;
batch.parallel.N = 0; % run locally = 0
%% BATCH Results (Second-Level ROI-to-ROI and seed-to-voxel)
batch.Results.done = 1;
batch.Results.overwrite = 1;
batch.Results.analysis_number = [1]; % sequential indexes inditify each set of independent analysis, alternative a string identifying the analysis name
%batch.Results.foldername = ''; % folder to store the results
%batch.Results.between_subjects.effect_names = {}; % cell array of second-level effect names
%batch.Results.between_subjects.contract = []; % constrast vector, same size as effect_names
%batch.Results.between_conditions.effect_names = {}; % cell array of condition names (as in Setup.conditions.names)
%batch.Results.between_conditions.contract = []; % constrast vector, same size as effect_names
%batch.Results.between_sources.effect_names = {}; % cell array of source names (as in Analysis.regressors, typically appended wiht _N_M, see documentation)
%batch.Results.between_sources.contract = []; % constrast vector, same size as effect_names
%% BATCH Results (Second-Level voxel-to-voxel)
%batch.Results.done = 0;
%batch.Results.overwrite = 1;
%batch.Results.foldername = ''; % folder to store the results
%batch.Results.between_subjects.effect_names = {}; % cell array of second-level effect names
%batch.Results.between_subjects.contract = []; % constrast vector, same size as effect_names
%batch.Results.between_conditions.effect_names = {}; % cell array of condition names (as in Setup.conditions.names)
%batch.Results.between_conditions.contract = []; % constrast vector, same size as effect_names
%batch.Results.between_measures.effect_names = {}; % cell array of source names (as in Analysis.measures)
%batch.Results.between_measures.contract = []; % constrast vector, same size as effect_names
%% run Analysis processing step
tic;
conn_batch(batch);
disp(['CONN batch running Results took ', num2str(round(toc / 60)), ' minutes']);
The idea is that for each stage we clear the batch variable (see first line), otherwise CONN will repeat the previously defined stages. If there is a gap between running the different stages (e.g. you quit Matlab between stages) you need to load the CONN_x variable stored in your project file as global variable (see second and third line), otherwise you'll see error message such as 'no subject defined' or 'out of bound indexing' since the project is not automatically loaded by CONN batch script. After that, we define the batch structure, point it to our existing project file, define the parameters for the next stage (in this case Second-Level analysis in batch.Results.*) and call conn_batch to perform the next stage. The parameters we do not define are filled in by CONN with default values. In our case we currently leave most options at default since we have a very simple setup of resting-state data of only healthy subjects. Also, note we only perform ROI-to-ROI and seed-to-voxel, and no voxel-to-voxel analysis, so the second part is commented out.
One thing to point out is that you may need to merge you individual projects into a single project for the second-level analysis. At the moment we perform this step manually in CONN: open CONN, go the Project menu and select 'Merge'; set the number of projects to merge, select 'Create new project' and add all the project you would like to merge. After you click on Done (lower left corner) CONN will ask you were to create the new project and copy all the necessary files to the next project folder. It should be possible to automate this step; check the conn_merge function in conn_merge.m. For the second-level analysis, with all your subjects in a single project, you may no longer need to use CONN batch script above but can instead run the analysis from the CONN GUI; open the merged project file in CONN and go to the Second-Level Results tab.
Hope this helps,
Regards,
Jalmar
Threaded View
| Title | Author | Date |
|---|---|---|
| Nathaniel Haines | Dec 8, 2015 | |
| Jalmar Teeuw | Dec 10, 2015 | |
| santiago velasco | May 18, 2017 | |
| Jalmar Teeuw | Dec 10, 2015 | |
| Nathaniel Haines | Dec 10, 2015 | |