help > RE: How to make it faster?
Jan 10, 2019  12:01 PM | Davide Fedeli
RE: How to make it faster?
Dear Alfonso, 
thank you for keeping the CONN toolbox updated and helping the CONN community everyday!

I'm using a windows PC for our CONN analyses. Would option C (from your answer to Haleh) work also for a windows computer with multiple cores?

Thanks for your help and support
Davide

Originally posted by Alfonso Nieto-Castanon:
Dear Haleh,

While usb3 is relatively fast, depending on the length of your data, your are still probably looking at something around 1 hour per subject (you can easily test that simply running a single-subject first, which is not a bad idea because that will help you debug if there are any issues before committing to waiting for a week for your results, and that will give you a good estimate on how long will it take to run all subjects). I do not believe neither storing the conn project locally nor running preprocessing separately on SPM would significantly reduce computation time. The main thing that does in my experience significantly speed things up is parallelizing your analyses. Some options would be:

a) HPC / cluster. Many institutions offer simple access to cluster computing resources, and CONN works with many standard cluster configurations right out of the box, so that might be worth investigating (see http://www.conn-toolbox.org/resources/cl... for additional info). If you follow this route, you would connect to your institution network remotely, run CONN and then simply select the parallelization option that reads "distributed processing" when running your analyses, and CONN will automatically distribute the analyses across your choice of nodes in the network (e.g as much as one node per subject)

b) if you have a shared remote storage at home/office (e.g. a network drive) and several computers all connected to the same shared storage, you may also use this setup as a sort of DIY cluster. If you follow this route, then you would simply select in CONN the parallelization option that reads "queue / script it (save as scripts to be run later)" to break down your processing pipeline into N blocks (here N is the number of different computers you have set up). That will create N Matlab scripts (or command-line shell scripts) that you can then manually run one on each of your computers. After all are finished, simply opening your CONN project from the GUI will merge the results from all of these different computers

and c) if you do not have access to multiple computers but your computer has a reasonable number of cores, you may also simply parallelize your analyses across the different cores. if you follow this route, you would first go to "Tools. HPC options. Configuration" and make sure there that the default profile (e.g. named "background process (Unix, Mac)" if you are on a Mac computer) works fine (e.g. just select 'Test profile'), and then, as in option (a) above, simply select in CONN the parallelization option that reads "distributed processing" when running your analyses to have CONN automatically distribute the analyses across your choice of cores (e.g. one process per core)

Hope this helps and good luck!
Alfonso

Threaded View

TitleAuthorDate
Haleh Karbasforoushan Dec 20, 2018
Alfonso Nieto-Castanon Dec 21, 2018
zj_lee Mar 15, 2023
Alfonso Nieto-Castanon Mar 19, 2023
RE: How to make it faster?
Davide Fedeli Jan 10, 2019
Pravesh Parekh Jan 10, 2019
Davide Fedeli Jan 11, 2019
Pravesh Parekh Jan 12, 2019