general-discussion
general-discussion > No progress and no error
Nov 20, 2013 03:11 PM | Debra Dawson
No progress and no error
Hi, I wrote a post several months ago with no response as a
reply to the "NIAK running really slowly" topic, but didn't get any
response, so I am trying a new topic.
I have run my script file, and all of the output directories are created along with all of the log files. There are 4 files which are identified as being submitted in the status log.
After the above progress, which occurred within the first few minutes, there has been no change at all in two days, and no error messages related to the running of niak processes. The only error I get is from my matlab closing after about a day: "Aborted (core dumped)". My impression is that my matlab closing isn't a issue because once the job has been submitted, it is run elsewhere.
Is this correct?
Why is nothing happening?
Below is my matlab script file which I ran on william.bic.mni.mcgill.ca, username ddawson.
%% Running NIAK
% If you are not seated in front of a BIC terminal use:
% 1)Connect to a BIC cluster terminal by ssh (e.g. ssh username@computer_name)
% 2)Run matlab on this terminal (matlab -nojvm -nodisplay)
% 3)Run this example below in matlab.
% path_niak = '/data/aces/aces1/pbellec/public/niak-edge/';
path_niak = '/data/aces/aces1/pbellec/public/niak-0.6.3/';
P = genpath(path_niak);
addpath(P);
%files_in=niak_files_in_rest_2;
rawdatapath='/export02/data/debbie/niakstuff/raw/';
niakoutpath='/export02/data/debbie/niakstuff/niak_out_Nov2013/';
storagepath='/export02/data/debbie/niakstuff/niak_out_storage/';
%mkdir(storagepath);
Subject(1).info={'AMB','beaulne_anne_marie', 'beaulne_anne_marie_20100719_093834', 2, [3:11], 'TRUE'};
Subject(2).info={'RG','graham_ross', 'graham_ross_20100924_095129', 3, [7:15], 'TRUE'};
Subject(3).info={'HH','hardie_heather', 'hardie_heather_20100830_145905', 3, [4:11], 'TRUE'};
Subject(4).info={'LM','mccabe_lorraine', 'mccabe_lorraine_20100830_124751', 3, [4:11], 'TRUE'};
Subject(5).info={'HM','monroe_hollis', 'monroe_hollis_20100719_115142', 2, [3:11], 'TRUE'};
Subject(6).info={'AS','stanhope_alexis', 'stanhope_alexis_20100726_133710', 2, [4:11], 'TRUE'};
Subject(7).info={'ZY','yao_zeshan', 'yao_zeshan_20101015_145702', 2, [6:14], 'TRUE'};
infoList=cat(1,Subject(:).info);
%nb_subject = sum(strcmp('TRUE',cat(1,infoList(:,6))));
subjIdxList = find(strcmp('TRUE',cat(1,infoList(:,6))));
nSubject=length(subjIdxList);
for iS=[1 4 5 6 7]%:nSubject
files_in=[];
subjIdx=subjIdxList(iS);
subjFieldName=sprintf('subject%d',subjIdx);
subjDataPath=[rawdatapath '/' Subject(subjIdx).info{3} '/'];
subjOutPath= [niakoutpath '/subject' num2str(iS) '/'];
subjStoragePath = [storagepath '/subject' num2str(iS) '/'];
% anatomy file
anatFileIdx=Subject(subjIdx).info{4};
files_in.(subjFieldName).anat = ...
[subjDataPath '/' Subject(subjIdx).info{3} '_' num2str(anatFileIdx) '_mri.mnc.gz'];
% functional files
boldFileIdxList=Subject(subjIdx).info{5};
nRuns=length(boldFileIdxList);
for iR=1:nRuns
boldFileIdx=boldFileIdxList(iR);
files_in.(subjFieldName).fmri.session1{iR} = ...
[subjDataPath '/' Subject(subjIdx).info{3} '_' num2str(boldFileIdx) '_mri.mnc.gz'];
end
%% Building the optional inputs
opt.style = 'standard-native'; %% The fMRI data will be analyzed in the standard-stereotaxic space
%opt.folder_out = [path_data 'fmri_preprocess' filesep];
% opt.folder_out = '/export02/data/kuwook/resting_long/niak/';%'/data/shmuel/shmuel1/kuwook_new/niak_mendola_normal/';
opt.folder_out = subjOutPath;
opt.size_output = 'quality_control';
opt.flag_corsica = 1;
%%%%%%%%%%%%%%%%%%%%
%% Bricks options %%
%%%%%%%%%%%%%%%%%%%%
%% 1. Linear and non-linear fit of the anatomical image in the stereotaxic
%% space (niak_brick_civet)
opt.bricks.civet.n3_distance = 25; % Parameter for non-uniformity correction.
%200 is a suggested value for 1.5T images, 25 for 3T images.
%If you find that this stage did not work well, this parameter is usually critical to improve the results.
%% 2. Motion correction (niak_brick_motion_correction)
opt.bricks.motion_correction.suppress_slice = 1; % Set the first and last slice to zero after interpolation
opt.bricks.motion_correction.suppress_vol = 3; % Remove the first three dummy scans
opt.bricks.motion_correction.vol_ref = 'median'; % Use the median of the run as a reference
opt.bricks.motion_correction.flag_session = 0; % Correct for both within and between sessions motion
opt.bricks.motion_correction.flag_run = true; % Consider each run as a separate session
opt.bricks.mask_brain.thresh = 0.5;
%% 3. Coregistration between T1 and T2 (niak_brick_coregister)
%% 4. Concatenation of the T2-to-T1 and T1-to-stereotaxic-nl
%% transformations. (niak_brick_concat_transf)
%% 5. Slice timing correction (niak_brick_slice_timing)
TR = 2; % Repetition time in seconds
nb_slices = 30; % Number of slices in a volume
opt.bricks.slice_timing.slice_order = [2:2:nb_slices 1:2:nb_slices];
opt.bricks.slice_timing.timing(1)=TR/nb_slices; % Time beetween slices
opt.bricks.slice_timing.timing(2)=TR/nb_slices; % Time between the last slice of a volume and the first slice of next volume (here there is no delay in TR)
%% 6. Temporal filetring (niak_brick_time_filter)
opt.bricks.time_filter.hp = 0.01; % Apply a high-pass filter at cut-off frequency 0.01Hz (slow time drifts)
%opt.bricks.time_filter.lp = 0.1; % Do not apply low-pass filter. Low-pass filter induce a big loss in degrees of freedom without sgnificantly improving the SNR.
%% 7. Correction of physiological noise (niak_pipeline_corsica)
opt.bricks.sica.nb_comp = 50;
opt.bricks.component_supp.threshold = 0.15;
%% 8. Resampling in the stereotaxic space (niak_brick_resample_vol)
%opt.bricks.resample_vol.voxel_size = -1;
%% 9. Spatial smoothing (niak_brick_smooth_vol)
opt.bricks.smooth_vol.fwhm = 0; % Apply an isotropic 6 mm gaussian smoothing.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Generation of the pipeline %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%opt.flag_test = 1;
% opt.path_folder = opt.folder_out;
% opt.path_logs = [opt.folder_out,'logs/'];
% opt.psom.mode = 'qsub'; %qsub
% opt.psom.mode_pipeline_manager = 'batch';
% opt.psom.max_queued = 4;
pipeline = niak_pipeline_fmri_preprocess(files_in,opt);
%%%
% move file
mkdir(subjStoragePath);
fprintf(1,'Copying files (subj #%d)...\n',iS);
bSuccess=copyfile([subjOutPath '/*'],subjStoragePath);
if bSuccess
fprintf(1,'\tDeleting original files...');
bSuccess=rmdir([subjOutPath],'s');
if ~bSuccess;error('Orignial files not removed');end
end
fprintf(1,'Done.\n');
end
I have run my script file, and all of the output directories are created along with all of the log files. There are 4 files which are identified as being submitted in the status log.
After the above progress, which occurred within the first few minutes, there has been no change at all in two days, and no error messages related to the running of niak processes. The only error I get is from my matlab closing after about a day: "Aborted (core dumped)". My impression is that my matlab closing isn't a issue because once the job has been submitted, it is run elsewhere.
Is this correct?
Why is nothing happening?
Below is my matlab script file which I ran on william.bic.mni.mcgill.ca, username ddawson.
%% Running NIAK
% If you are not seated in front of a BIC terminal use:
% 1)Connect to a BIC cluster terminal by ssh (e.g. ssh username@computer_name)
% 2)Run matlab on this terminal (matlab -nojvm -nodisplay)
% 3)Run this example below in matlab.
% path_niak = '/data/aces/aces1/pbellec/public/niak-edge/';
path_niak = '/data/aces/aces1/pbellec/public/niak-0.6.3/';
P = genpath(path_niak);
addpath(P);
%files_in=niak_files_in_rest_2;
rawdatapath='/export02/data/debbie/niakstuff/raw/';
niakoutpath='/export02/data/debbie/niakstuff/niak_out_Nov2013/';
storagepath='/export02/data/debbie/niakstuff/niak_out_storage/';
%mkdir(storagepath);
Subject(1).info={'AMB','beaulne_anne_marie', 'beaulne_anne_marie_20100719_093834', 2, [3:11], 'TRUE'};
Subject(2).info={'RG','graham_ross', 'graham_ross_20100924_095129', 3, [7:15], 'TRUE'};
Subject(3).info={'HH','hardie_heather', 'hardie_heather_20100830_145905', 3, [4:11], 'TRUE'};
Subject(4).info={'LM','mccabe_lorraine', 'mccabe_lorraine_20100830_124751', 3, [4:11], 'TRUE'};
Subject(5).info={'HM','monroe_hollis', 'monroe_hollis_20100719_115142', 2, [3:11], 'TRUE'};
Subject(6).info={'AS','stanhope_alexis', 'stanhope_alexis_20100726_133710', 2, [4:11], 'TRUE'};
Subject(7).info={'ZY','yao_zeshan', 'yao_zeshan_20101015_145702', 2, [6:14], 'TRUE'};
infoList=cat(1,Subject(:).info);
%nb_subject = sum(strcmp('TRUE',cat(1,infoList(:,6))));
subjIdxList = find(strcmp('TRUE',cat(1,infoList(:,6))));
nSubject=length(subjIdxList);
for iS=[1 4 5 6 7]%:nSubject
files_in=[];
subjIdx=subjIdxList(iS);
subjFieldName=sprintf('subject%d',subjIdx);
subjDataPath=[rawdatapath '/' Subject(subjIdx).info{3} '/'];
subjOutPath= [niakoutpath '/subject' num2str(iS) '/'];
subjStoragePath = [storagepath '/subject' num2str(iS) '/'];
% anatomy file
anatFileIdx=Subject(subjIdx).info{4};
files_in.(subjFieldName).anat = ...
[subjDataPath '/' Subject(subjIdx).info{3} '_' num2str(anatFileIdx) '_mri.mnc.gz'];
% functional files
boldFileIdxList=Subject(subjIdx).info{5};
nRuns=length(boldFileIdxList);
for iR=1:nRuns
boldFileIdx=boldFileIdxList(iR);
files_in.(subjFieldName).fmri.session1{iR} = ...
[subjDataPath '/' Subject(subjIdx).info{3} '_' num2str(boldFileIdx) '_mri.mnc.gz'];
end
%% Building the optional inputs
opt.style = 'standard-native'; %% The fMRI data will be analyzed in the standard-stereotaxic space
%opt.folder_out = [path_data 'fmri_preprocess' filesep];
% opt.folder_out = '/export02/data/kuwook/resting_long/niak/';%'/data/shmuel/shmuel1/kuwook_new/niak_mendola_normal/';
opt.folder_out = subjOutPath;
opt.size_output = 'quality_control';
opt.flag_corsica = 1;
%%%%%%%%%%%%%%%%%%%%
%% Bricks options %%
%%%%%%%%%%%%%%%%%%%%
%% 1. Linear and non-linear fit of the anatomical image in the stereotaxic
%% space (niak_brick_civet)
opt.bricks.civet.n3_distance = 25; % Parameter for non-uniformity correction.
%200 is a suggested value for 1.5T images, 25 for 3T images.
%If you find that this stage did not work well, this parameter is usually critical to improve the results.
%% 2. Motion correction (niak_brick_motion_correction)
opt.bricks.motion_correction.suppress_slice = 1; % Set the first and last slice to zero after interpolation
opt.bricks.motion_correction.suppress_vol = 3; % Remove the first three dummy scans
opt.bricks.motion_correction.vol_ref = 'median'; % Use the median of the run as a reference
opt.bricks.motion_correction.flag_session = 0; % Correct for both within and between sessions motion
opt.bricks.motion_correction.flag_run = true; % Consider each run as a separate session
opt.bricks.mask_brain.thresh = 0.5;
%% 3. Coregistration between T1 and T2 (niak_brick_coregister)
%% 4. Concatenation of the T2-to-T1 and T1-to-stereotaxic-nl
%% transformations. (niak_brick_concat_transf)
%% 5. Slice timing correction (niak_brick_slice_timing)
TR = 2; % Repetition time in seconds
nb_slices = 30; % Number of slices in a volume
opt.bricks.slice_timing.slice_order = [2:2:nb_slices 1:2:nb_slices];
opt.bricks.slice_timing.timing(1)=TR/nb_slices; % Time beetween slices
opt.bricks.slice_timing.timing(2)=TR/nb_slices; % Time between the last slice of a volume and the first slice of next volume (here there is no delay in TR)
%% 6. Temporal filetring (niak_brick_time_filter)
opt.bricks.time_filter.hp = 0.01; % Apply a high-pass filter at cut-off frequency 0.01Hz (slow time drifts)
%opt.bricks.time_filter.lp = 0.1; % Do not apply low-pass filter. Low-pass filter induce a big loss in degrees of freedom without sgnificantly improving the SNR.
%% 7. Correction of physiological noise (niak_pipeline_corsica)
opt.bricks.sica.nb_comp = 50;
opt.bricks.component_supp.threshold = 0.15;
%% 8. Resampling in the stereotaxic space (niak_brick_resample_vol)
%opt.bricks.resample_vol.voxel_size = -1;
%% 9. Spatial smoothing (niak_brick_smooth_vol)
opt.bricks.smooth_vol.fwhm = 0; % Apply an isotropic 6 mm gaussian smoothing.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Generation of the pipeline %%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%opt.flag_test = 1;
% opt.path_folder = opt.folder_out;
% opt.path_logs = [opt.folder_out,'logs/'];
% opt.psom.mode = 'qsub'; %qsub
% opt.psom.mode_pipeline_manager = 'batch';
% opt.psom.max_queued = 4;
pipeline = niak_pipeline_fmri_preprocess(files_in,opt);
%%%
% move file
mkdir(subjStoragePath);
fprintf(1,'Copying files (subj #%d)...\n',iS);
bSuccess=copyfile([subjOutPath '/*'],subjStoragePath);
if bSuccess
fprintf(1,'\tDeleting original files...');
bSuccess=rmdir([subjOutPath],'s');
if ~bSuccess;error('Orignial files not removed');end
end
fprintf(1,'Done.\n');
end
Threaded View
| Title | Author | Date |
|---|---|---|
| Debra Dawson | Nov 20, 2013 | |
| Debra Dawson | Nov 22, 2013 | |
| Sebastian Urchs | Nov 21, 2013 | |
| Mike Ferreira | Nov 20, 2013 | |
