help
6 Subscribers 
help > How long does it take to process a dataset
Showing 1-2 of 2 posts
Sep 15, 2011 07:09 PM | Qingyang Li
How long does it take to process a dataset
The first 3 scripts run very fast, but but it took a very long time for the "run_irfret.sh". In fact, I started it the script 2 hours ago, and it still stays at "adjusting and getting IRFs - scond round". Is it normal?Thanks,Below is the scirpts I ran. 1 run_pestica_ica.sh2 run_pestica_est.sh3 filter_pestica_est.sh4 run_irfret.sh
Sep 19, 2011 09:09 PM | Erik Beall
RE: How long does it take to process a dataset
Hi Yang,
the fourth script "run_irfret.sh" is actually running two algorithms: one is my IRF-RETROICOR (an improved version of RETROICOR, see http://www.ncbi.nlm.nih.gov/pubmed/20096... for more details) algorithm, and the other is an algorithm to improve the quality of cardphase.dat.
The problem with cardphase is that the filter_pestica_est.sh script results in dithering of the true cardiac peaks, so the true location of the starts of each cardiac cycle are no longer accurately located. Its been suggested that I switch to wavelet filtering to reduce this, but in lieu of that, I have this script that is called by run_irfret.sh that adjusts each peak to optimize the correlation with ICA components (I used a similar procedure in my 2007 paper on PESTICA when I was having trouble with the pulse oximeter data). It is this second script that takes so much time, typically a few hours per study. It is optional, but in general it does improve teh number of peak_indices reported.
By the way, peak_indices refers to the number of voxels that are really highly correlated with the PESTICA cardphase and respphase outputs (I set a stringent threshold). In general, if you get less than 1000 voxels highly correlated for either cardphase or respphase, then that estimator might not be good. However, this will depend on field strength, voxel size and the level (possibly rate) of physiologic noise present in that subject.
***** Please note that I have seen respiratory noise disappearing at certain fixed breathing rates, and I've also seen very low cardiac coupling even when using a pulse oximeter with good data, and the same result in those subjects when using PESTICA). ****
So peak_indices is not always a slam dunk, and across subjects I have found it to be highly variable even with pulse ox and respiratory bellows signals (in those cases I do NOT use my fit_each_peak script to correct for dither, as there should be no dither present). You should look at the spatial maps and look for middle cerebral artery and other arterial/venous signatures.
Erik
the fourth script "run_irfret.sh" is actually running two algorithms: one is my IRF-RETROICOR (an improved version of RETROICOR, see http://www.ncbi.nlm.nih.gov/pubmed/20096... for more details) algorithm, and the other is an algorithm to improve the quality of cardphase.dat.
The problem with cardphase is that the filter_pestica_est.sh script results in dithering of the true cardiac peaks, so the true location of the starts of each cardiac cycle are no longer accurately located. Its been suggested that I switch to wavelet filtering to reduce this, but in lieu of that, I have this script that is called by run_irfret.sh that adjusts each peak to optimize the correlation with ICA components (I used a similar procedure in my 2007 paper on PESTICA when I was having trouble with the pulse oximeter data). It is this second script that takes so much time, typically a few hours per study. It is optional, but in general it does improve teh number of peak_indices reported.
By the way, peak_indices refers to the number of voxels that are really highly correlated with the PESTICA cardphase and respphase outputs (I set a stringent threshold). In general, if you get less than 1000 voxels highly correlated for either cardphase or respphase, then that estimator might not be good. However, this will depend on field strength, voxel size and the level (possibly rate) of physiologic noise present in that subject.
***** Please note that I have seen respiratory noise disappearing at certain fixed breathing rates, and I've also seen very low cardiac coupling even when using a pulse oximeter with good data, and the same result in those subjects when using PESTICA). ****
So peak_indices is not always a slam dunk, and across subjects I have found it to be highly variable even with pulse ox and respiratory bellows signals (in those cases I do NOT use my fit_each_peak script to correct for dither, as there should be no dither present). You should look at the spatial maps and look for middle cerebral artery and other arterial/venous signatures.
Erik