


Function to automatically test PRoNTo's integrity
The goal is to have PRoNTo run through typical analysis and check if the
calculations proceed smoothly.
This relies on pre-specified
- organisation of data in subdirectories
- batches with all the operations, in a .mat file with known location
Data sets considered, in this *specific order*:
1. "Haxby" - Haxby data, single subject, fmri
2. "IXI" - IXI data, multi subject, divergence & momentum maps
3. "Faces" - SPM's famous-vs-nonfamous faces data, multi subject.
See the subfunctions for a detailed description of the tests performed.
FORMAT ok = prt_check(list_check,dir_root)
INPUT
list_check - list of data sets to use, [1 2 3] by default
dir_root - root directory of data sets (you'd better set this for
your own HD organization!)
OUTPUT:
ok - vector of output (1='ok', 0='failed', -1='not tested')
NOTE:
- For a more automatic testing on your own system, then up date the
default 'dir_root' variable with the path to the 'PRoNTo_data'
directory on your system.
- This will close all Matlab windows before relaunching PRoNTo and the
matlabbatch system.
WARNING:
This version was developped for and is running on **SPM12**
_______________________________________________________________________
Copyright (C) 2012 Machine Learning & Neuroimaging Laboratory


0001 function ok = prt_check(list_check,dir_root) 0002 % Function to automatically test PRoNTo's integrity 0003 % 0004 % The goal is to have PRoNTo run through typical analysis and check if the 0005 % calculations proceed smoothly. 0006 % This relies on pre-specified 0007 % - organisation of data in subdirectories 0008 % - batches with all the operations, in a .mat file with known location 0009 % 0010 % Data sets considered, in this *specific order*: 0011 % 1. "Haxby" - Haxby data, single subject, fmri 0012 % 2. "IXI" - IXI data, multi subject, divergence & momentum maps 0013 % 3. "Faces" - SPM's famous-vs-nonfamous faces data, multi subject. 0014 % 0015 % See the subfunctions for a detailed description of the tests performed. 0016 % 0017 % FORMAT ok = prt_check(list_check,dir_root) 0018 % 0019 % INPUT 0020 % list_check - list of data sets to use, [1 2 3] by default 0021 % dir_root - root directory of data sets (you'd better set this for 0022 % your own HD organization!) 0023 % 0024 % OUTPUT: 0025 % ok - vector of output (1='ok', 0='failed', -1='not tested') 0026 % 0027 % NOTE: 0028 % - For a more automatic testing on your own system, then up date the 0029 % default 'dir_root' variable with the path to the 'PRoNTo_data' 0030 % directory on your system. 0031 % - This will close all Matlab windows before relaunching PRoNTo and the 0032 % matlabbatch system. 0033 % 0034 % WARNING: 0035 % This version was developped for and is running on **SPM12** 0036 % 0037 %_______________________________________________________________________ 0038 % Copyright (C) 2012 Machine Learning & Neuroimaging Laboratory 0039 0040 % Written by Christophe Phillips, CRC, ULg, Belgium. 0041 % $Id$ 0042 0043 % Defining the check parameters. 0044 % data sets, list to check and root directory 0045 %-------------------------------------------- 0046 % list here the ones you want to check, e.g. 'list_check = 1:3;' for all 0047 if nargin<1, list_check = [1 2 3]; end 0048 if nargin<2 0049 % adjust with your own data set 0050 dir_root = '/Users/chrisp/Documents/MATLAB/3_Data/PRoNTo/PRoNTo_data'; 0051 end 0052 while isempty(dir_root) || ~exist(dir_root,'dir') 0053 % or select the root directories manually 0054 dir_root = spm_select([1 1],'dir','Select root dir for data sets'); 0055 end 0056 dat_name = {'Haxby' , 'IXI ' , 'Faces'}; 0057 Ndat = numel(dat_name); 0058 data_dir = cell(Ndat,1); 0059 for ii=1:Ndat 0060 data_dir{ii} = fullfile(dir_root,deblank(dat_name{ii})); 0061 end 0062 0063 % Clearing Matlab then setting up PRONTO and the batch system 0064 %------------------------------------------------------------ 0065 close all 0066 prt_batch 0067 0068 % Going through the various tests 0069 %-------------------------------- 0070 ok = zeros(Ndat,1)-1; 0071 for ii=list_check 0072 switch ii 0073 case 1 % Haxby data 0074 ok(ii) = check_Haxby(data_dir{ii}); 0075 case 2 % IXI data 0076 ok(ii) = check_IXI(data_dir{ii}); 0077 case 3 % famous-vs-nonfamous data 0078 ok(ii) = check_FvsNF(data_dir{ii}); 0079 otherwise 0080 fprintf(1,'\nUNKNOWN DATA SET TO CHECK!\n') %#ok<*PRTCAL> 0081 beep 0082 end 0083 end 0084 0085 % Printing out the results 0086 %------------------------- 0087 fprintf('\nTesting on data sets:\n') 0088 for ii=1:Ndat 0089 switch ok(ii) 0090 case -1, msg = 'not tested'; 0091 case 0 , msg = 'failed'; 0092 case 1 , msg = 'passed'; 0093 otherwise, msg = 'unknown output flag'; 0094 end 0095 fprintf('\t%s\t: %s\n',dat_name{ii},msg); 0096 end 0097 0098 end 0099 0100 %========================================================================== 0101 %% INDIVIDUAL DATA SET CHECKING ROUTINES 0102 %========================================================================== 0103 0104 %% HAXBY data set 0105 function ok = check_Haxby(rdata_dir) 0106 % 0107 % This batch will go through the following modules, as saved in the 0108 % batch_test_HaxbyData.mat file: 0109 % - 'File selector' for (1) images, (2) SPM.mat, (3) mask 1st level, 0110 % (4) mask 2nd level, and (5) atlas for ROI 0111 % - 'Directory selector' for the root of the data directory 0112 % - 'Make directory', create 'test_results' directory at the root of the 0113 % data directory 0114 % - 'Data & design' as in manual example with whole brain mask 0115 % - 'Feature set', as in manual example, no 2nd level mask 0116 % - 'Specify model', use MKL with separate kernel for each ROI, as defined 0117 % in the AAL atlas. And a k-folds CV on blocks (k=4) 0118 % - 'Run model' with 1000 permutations 0119 % - 'Compute weights' -> create 'mkl_weights' image 0120 % - 'Feature set', DCT detrending and a 2nd level mask (fusiform gyrus) 0121 % - 'Specify model', multi-GPC Faces vs Houses vs Shoes, LOBO CV 0122 % - 'Run model' without any permutation 0123 % - 'Compute weights' -> create 'mgpc_weights' image 0124 0125 % select images, SPM.mat and mask(s) 0126 d_dir = fullfile(rdata_dir,'fMRI'); 0127 [img_files] = spm_select('FPList',d_dir,'^w.*\.nii$'); 0128 s_dir = fullfile(rdata_dir,'design'); 0129 [spm_file] = spm_select('FPList',s_dir,'^SPM\.mat$'); 0130 m_dir = fullfile(rdata_dir,'masks'); 0131 [msk_file] = spm_select('FPList',m_dir,'^.*\.img$'); 0132 a_dir = fullfile(prt('dir'),'atlas'); 0133 [atlas_file] = spm_select('FPList',a_dir,'^aal.*\.img$'); 0134 0135 % get batch file 0136 job = fullfile(prt('dir'),'_unitTests','batch_test_HaxbyData.mat'); 0137 0138 inputs = cell(6,1); 0139 inputs{1} = cellstr(img_files); % fMRI data 0140 inputs{2} = cellstr(spm_file); % SPM.mat file 0141 inputs{3} = cellstr(msk_file(2,:)); % 1st level mask 0142 inputs{4} = cellstr(msk_file(1,:)); % 2nd level mask 0143 inputs{5} = cellstr(atlas_file); % ROI atlas 0144 inputs{6} = {rdata_dir}; % root directory 0145 0146 ok = 1; 0147 0148 try 0149 job_id = cfg_util('initjob', job); 0150 sts = cfg_util('filljob', job_id, inputs{:}); 0151 if sts 0152 cfg_util('run', job_id); 0153 else 0154 disp('Job status problem.') 0155 ok = 0; 0156 end 0157 cfg_util('deljob', job_id); 0158 catch ME 0159 cfg_util('deljob', job_id); 0160 disp(ME.message) 0161 ok = 0; 0162 return; 0163 end 0164 0165 end 0166 0167 %========================================================================== 0168 %% IXI data set 0169 function ok = check_IXI(rdata_dir) 0170 % 0171 % NOTE: 0172 % The data set only includes 5 subjects! 0173 % 0174 % This batch will go through the following modules, as saved in the 0175 % batch_test_IXIdata.mat file: 0176 % - 'Load Variables from .mat File' 3 times the regression targets for the 0177 % Guys/Hammers/IOP data 0178 % - 'File selector' for the 6 sets of images: 0179 % Guys-divergence (g1m1), Guys-momentum (g1m2), 0180 % HammersH-divergence (g2m1), HammersH-momentum (g2m2) 0181 % IOP-divergence (g3m1), IOP-momentum (g3m2) 0182 % - 'File selector' for 2 masks: momentum, divergence (could be the same 0183 % file actually and it will be defined so when filling the batch) 0184 % - 'Directory selector' for the root of the data directory 0185 % - 'Make directory', create 'test_results' directory at the root of the 0186 % data directory 0187 % - 'Data & design' with 3 groups (Guys/HammersH/IOP) and 2 modalities each 0188 % ('momentum'/'divergence') + regression target (age). 0189 % - 'Feature set', only the 'divergence' data 0190 % - 'Specify model', svm Guys-vs-(HammersH+IOP), on divergence data, 0191 % leave-1s/gr-out CV 0192 % - 'Run model' with 1000 permutations 0193 % - 'Compute weights' -> create 'svm_GvsHI' image 0194 % - 'Feature set', only the 'momentum' data 0195 % - 'Specify model', KRR for age of all 3 groups of scans 0196 % - 'Run model' with 1000 permutations 0197 % - 'Feature set', pool 'momentum' and 'divergence' data together 0198 % - 'Specify model', GPC Guys-vs-HammersH-vs-IOP, on 'momentum+divergence' 0199 % data leave-1s/gr-out 0200 % - 'Run model' without permutation 0201 % - 'Specify model', RVR for age of all 3 groups of scans (using momentum 0202 % feature set) 0203 % - 'Run model' with 1000 permutations 0204 % - 'Specify model', GPR for age of all 3 groups of scans (using momentum 0205 % feature set) 0206 % - 'Run model' without permutation 0207 % - 'Feature set', both 'divergence' and 'momentum', one kernel modality 0208 % - 'Feature set', both 'divergence' and 'momentum', one kernel modality + 0209 % one kenel per ROI 0210 % - 'Specify model', MKL with 1 kernel/modality, Guys-vs-(HammersH+IOP), 0211 % leave-1s/gr-out CV 0212 % - 'Run model' without permutation 0213 % - 'Compute weights' -> create 'MKLmm_weights' image 0214 % - 'Specify model', MKL with 1 kernel/modality and /ROI, 0215 % Guys-vs-(HammersH+IOP), leave-1s/gr-out CV 0216 % - 'Run model' without permutation 0217 % - 'Compute weights' -> create 'MKLmmroi_weights' image, with ROI specific 0218 % weights 0219 0220 % set images directories and select mask(s) 0221 d_dir{1} = fullfile(rdata_dir,'divergences'); 0222 d_dir{2} = fullfile(rdata_dir,'momentum'); 0223 m_dir = fullfile(prt('dir'),'masks'); 0224 [msk_file] = spm_select('FPList',m_dir,'^.*\.img$'); 0225 a_dir = fullfile(prt('dir'),'atlas'); 0226 [atlas_file] = spm_select('FPList',a_dir,'^aal.*\.img$'); 0227 0228 % get batch file 0229 job = fullfile(prt('dir'),'_unitTests','batch_test_IXIdata.mat'); 0230 0231 inputs = cell(13,1); 0232 % set files: regression target variables 0233 inputs{1} = cellstr(fullfile(rdata_dir,'reg_targets','rt_Guys.mat')); 0234 inputs{2} = cellstr(fullfile(rdata_dir,'reg_targets','rt_HammerH.mat')); 0235 inputs{3} = cellstr(fullfile(rdata_dir,'reg_targets','rt_IOP.mat')); 0236 % set files: images into 6 sets (g1m1-g1m2-g2m1-g2m2-g3m1-g3m2) 0237 inputs{4} = cellstr(... 0238 spm_select('FPList',fullfile(d_dir{1},'Guys'),'^sdv.*\.nii$')); 0239 inputs{5} = cellstr(... 0240 spm_select('FPList',fullfile(d_dir{2},'Guys'),'^sa.*\.nii$')); 0241 inputs{6} = cellstr(... 0242 spm_select('FPList',fullfile(d_dir{1},'HammerH'),'^sdv.*\.nii$')); 0243 inputs{7} = cellstr(... 0244 spm_select('FPList',fullfile(d_dir{2},'HammerH'),'^sa.*\.nii$')); 0245 inputs{8} = cellstr(... 0246 spm_select('FPList',fullfile(d_dir{1},'IOP'),'^sdv.*\.nii$')); 0247 inputs{9} = cellstr(... 0248 spm_select('FPList',fullfile(d_dir{2},'IOP'),'^sa.*\.nii$')); 0249 % set files: mask (1st level) for both modalities 0250 inputs{10} = cellstr(msk_file); 0251 inputs{11} = cellstr(msk_file); 0252 % set files: atlas for ROI 0253 inputs{12} = cellstr(atlas_file); 0254 % set directory: where result directory is created = "root data directory" 0255 inputs{13} = {rdata_dir}; 0256 0257 ok = 1; 0258 try 0259 job_id = cfg_util('initjob', job); 0260 sts = cfg_util('filljob', job_id, inputs{:}); 0261 if sts 0262 cfg_util('run', job_id); 0263 else 0264 disp('Job status problem.') 0265 ok = 0; 0266 end 0267 cfg_util('deljob', job_id); 0268 catch ME 0269 cfg_util('deljob', job_id); 0270 disp(ME.message) 0271 ok = 0; 0272 return; 0273 end 0274 0275 end 0276 0277 %========================================================================== 0278 %% Faces Famous-vs-NonFamous data set 0279 function ok = check_FvsNF(rdata_dir) 0280 0281 % This batch will go through the following modules, as saved in the 0282 % batch_test_FacesData.mat file: 0283 % - 'File selector' for 0284 % (1/2) the 2 sets of images: 'Famous' and 'NonFamous' beta's 0285 % (3) the whole brain mask, and (4) the customCV.mat file 0286 % - 'Directory selector' for the root of the data directory 0287 % - 'Make directory', create 'test_results' directory at the root of the 0288 % data directory 0289 % - 'Data & design', defining the 2 groups Famous/NonFamous and data 0290 % - 'Feature set', using the beta images 0291 % - 'Specify model', svm F-vs-NF, leave-1s-out CV with hyper-parameter 0292 % estimation in nested CV 0293 % - 'Run model' without permutations 0294 % - 'Compute weights' -> create 'svm_FvsNF' image 0295 % - 'Specify model', gpc F-vs-NF, custom CV (training on 1st 20 images from 0296 % each group and testing last 6) 0297 % - 'Run model' without permutations 0298 % - 'Compute weights' -> create 'gpc_FvsNF' image 0299 0300 % select images, SPM.mat and mask(s) 0301 d_dir = fullfile(rdata_dir,'Famous'); 0302 [imgF_files] = spm_select('FPList',d_dir,'^beta.*\.img$'); 0303 d_dir = fullfile(rdata_dir,'NonFamous'); 0304 [imgNF_files] = spm_select('FPList',d_dir,'^beta.*\.img$'); 0305 m_dir = fullfile(prt('dir'),'masks'); 0306 [msk_file] = spm_select('FPList',m_dir,'^.*\.img$'); 0307 CV_file = spm_select('FPList',rdata_dir,'^customCV\.mat$'); 0308 0309 % get batch file 0310 job = fullfile(prt('dir'),'_unitTests','batch_test_FacesData.mat'); 0311 0312 inputs = cell(5,1); 0313 inputs{1} = cellstr(imgF_files); 0314 inputs{2} = cellstr(imgNF_files); 0315 inputs{3} = cellstr(msk_file); 0316 inputs{4} = cellstr(CV_file); 0317 % set directory where result directory is created = "root data directory" 0318 inputs{5} = {rdata_dir}; 0319 0320 ok = 1; 0321 try 0322 job_id = cfg_util('initjob', job); 0323 sts = cfg_util('filljob', job_id, inputs{:}); 0324 if sts 0325 cfg_util('run', job_id); 0326 else 0327 disp('Job status problem.') 0328 ok = 0; 0329 end 0330 cfg_util('deljob', job_id); 0331 catch ME 0332 cfg_util('deljob', job_id); 0333 disp(ME.message) 0334 ok = 0; 0335 return; 0336 end 0337 0338 end 0339