open-discussion > Too many gradients excluded by "default"
Showing 1-10 of 10 posts
Display:
Results per page:
Apr 19, 2017  02:04 AM | minchulusa
Too many gradients excluded by "default"
Hello ,
My name is MC kim, and I need some help.
The DTIPrep version I got is 1.2.4.
I try to preprocess my DWI images with DTIPrep, and I got some questions in doing it. 

1) I initially tried with 'default' protocol, and DTIPrep excluded 17 out for 31 gradients from a DWI data set from a subject.
the reason was due to interlace artifact. When I visual checked the DWI data, I couldn't agree that much artifact. 
I attached the captured QC result file.

I tried preprocessing again, and this time I changed the protocol, instead of using 'default'.
By default, DENOISING_bCheck, JOINDENOISING_bCheck, BRAINMAST_bCheck, and DOMINANTDIRECTION_bCheck at setted "No", and I changed settings into "YES"
As I tried again with all "YES" all of the directions were included in QCed DWI.
Thus I though by changing protocols to all "YES", the protocol became more tolerant to artifact.

My question is is it okay to run with the protocol setting of all "YES"? Is there any potential problem?

2) Even though the "default"protocol excluded more than half of DWI gradient directions, DTIPrep still generated DTI images such as FA, MD, etc.
Can I use this DTI image to analysis or tract generation? or should I prepration again with different protocol setting.

3)I read that DTIPrep has two nice functions, those are "vibration artifact checking" and "simulation based bias analysis".
I want to know which parameter on protocol setting represents those functions.
My guess is "DOMINANTDIRECTION_bCheck" maybe vibration artifact correction, am I right?

4) When I click on "Default" protocol, I get the error message saying "The following programs have not been found. Please enter the path manually if necessary> DiffusionWeightedVolumeMasking"
Can you tell me where can I get this program or how can I solve the problem?

Thanks in advance and I look forward to hear from you.

MC Kim
Attachment: error.jpg
Apr 19, 2017  03:04 PM | Martin Styner
RE: Too many gradients excluded by "default"
Hi Minchul
Re 1) the first denoising/smoothing option DENOISING_bCheck smooth the DWI data prior to any check which will smooth out any minor artifacts, which is why the previously rejected DWIs are now considered fine. It's "okay" to use that option, though we usually do not (I prefer un-smoothed data, but other researchers prefer smoothed data). This would definitely be something that you would need to report in any papers resulting from your work. The options JOINDENOISING_bCheck, BRAINMAST_bCheck, and DOMINANTDIRECTION_bCheck have no effect on the DWI cleaning/rejections.

What we usually do when too many DWI get rejected despite appearing appropriate is to change the thresholds for SLICE and INTERLACE checks. The report txt and xml files give you information which step rejects the most DWI and why. 

Re 2) That depends on your settings. You can tell DTIPrep not to create the final QCed DWI and DTI images when the resulting DWI has more than a certain percentage of rejected DWIs (tolerance threshold). At the most permissive (when data is very noisy), we usually do not go beyond 40% rejection, while our usual studies use 20-30% rejection tolerance. 50% seems to high to me and I would NOT use that data for further analysis.

re 3)DOMINANTDIRECTION_bCheck = vibration artifact check. This is not commonly needed unless you have earlier Tim Trio data or suspect presence of vibration artifacts. The issue here is that you need to train the classifier by running the check on sample data that looks good (the xml contains the entropy results, see our papers on this topic) and then use the mean & stdev from that sample data in the protocol to identify the presence or absence of such an artifact.

re 3) simulation based bias analysis: unfortunately that is not in DTIPrep yet. We simply don't have the manpower to put it in (there is no grant that support DTIPrep and we are currently only maintaining it and do not add any major functionality). This would be great as it would allow to reject/accept DTI data based on the expected error rather than a simple tolerance threshold.

re 4) DTIPrep offers support for multiple brain masking options. If you want to use that option you need to point DTIPrep to it (it's part of Slicer). We commonly suggest the use of FSL's bet/bet2 with IDWI (double click on the protocol option and DTIPrep will allow you to select which masking option to choose)

Martin
Originally posted by Minchul Kim:
Hello ,
My name is MC kim, and I need some help.
The DTIPrep version I got is 1.2.4.
I try to preprocess my DWI images with DTIPrep, and I got some questions in doing it. 

1) I initially tried with 'default' protocol, and DTIPrep excluded 17 out for 31 gradients from a DWI data set from a subject.
the reason was due to interlace artifact. When I visual checked the DWI data, I couldn't agree that much artifact. 
I attached the captured QC result file.

I tried preprocessing again, and this time I changed the protocol, instead of using 'default'.
By default, DENOISING_bCheck, JOINDENOISING_bCheck, BRAINMAST_bCheck, and DOMINANTDIRECTION_bCheck at setted "No", and I changed settings into "YES"
As I tried again with all "YES" all of the directions were included in QCed DWI.
Thus I though by changing protocols to all "YES", the protocol became more tolerant to artifact.

My question is is it okay to run with the protocol setting of all "YES"? Is there any potential problem?

2) Even though the "default"protocol excluded more than half of DWI gradient directions, DTIPrep still generated DTI images such as FA, MD, etc.
Can I use this DTI image to analysis or tract generation? or should I prepration again with different protocol setting.

3)I read that DTIPrep has two nice functions, those are "vibration artifact checking" and "simulation based bias analysis".
I want to know which parameter on protocol setting represents those functions.
My guess is "DOMINANTDIRECTION_bCheck" maybe vibration artifact correction, am I right?

4) When I click on "Default" protocol, I get the error message saying "The following programs have not been found. Please enter the path manually if necessary> DiffusionWeightedVolumeMasking"
Can you tell me where can I get this program or how can I solve the problem?

Thanks in advance and I look forward to hear from you.

MC Kim
Apr 20, 2017  02:04 AM | minchulusa
RE: Too many gradients excluded by "default"
Hi Martin
Thank you a lot for your kind advice, it was helpful.
I cameout with additional questions on your answer.

1) Upon your answer "Re 2)" , I want to set the tolerance threshold to whether create or not created final QCed DWI and DTIs.
I looked up at media wiki, but I cannot be sure which Parameter is for tolerance setting.
I have two guesses and marked "1" and "2" in attached image.
Please let me know the function and meanings of those two parameters.

2) Another question is that for "Re 3)", Am I supposed to train DTIPrep with my own artifact free data? Because I though DTIPrep has 
kind of built-in artifact free samples and compare with my data.

Thanks in advance and look forward to listen.

Minchul 
Originally posted by Martin Styner:
Hi Minchul
Re 1) the first denoising/smoothing option DENOISING_bCheck smooth the DWI data prior to any check which will smooth out any minor artifacts, which is why the previously rejected DWIs are now considered fine. It's "okay" to use that option, though we usually do not (I prefer un-smoothed data, but other researchers prefer smoothed data). This would definitely be something that you would need to report in any papers resulting from your work. The options JOINDENOISING_bCheck, BRAINMAST_bCheck, and DOMINANTDIRECTION_bCheck have no effect on the DWI cleaning/rejections.

What we usually do when too many DWI get rejected despite appearing appropriate is to change the thresholds for SLICE and INTERLACE checks. The report txt and xml files give you information which step rejects the most DWI and why. 

Re 2) That depends on your settings. You can tell DTIPrep not to create the final QCed DWI and DTI images when the resulting DWI has more than a certain percentage of rejected DWIs (tolerance threshold). At the most permissive (when data is very noisy), we usually do not go beyond 40% rejection, while our usual studies use 20-30% rejection tolerance. 50% seems to high to me and I would NOT use that data for further analysis.

re 3)DOMINANTDIRECTION_bCheck = vibration artifact check. This is not commonly needed unless you have earlier Tim Trio data or suspect presence of vibration artifacts. The issue here is that you need to train the classifier by running the check on sample data that looks good (the xml contains the entropy results, see our papers on this topic) and then use the mean & stdev from that sample data in the protocol to identify the presence or absence of such an artifact.

re 3) simulation based bias analysis: unfortunately that is not in DTIPrep yet. We simply don't have the manpower to put it in (there is no grant that support DTIPrep and we are currently only maintaining it and do not add any major functionality). This would be great as it would allow to reject/accept DTI data based on the expected error rather than a simple tolerance threshold.

re 4) DTIPrep offers support for multiple brain masking options. If you want to use that option you need to point DTIPrep to it (it's part of Slicer). We commonly suggest the use of FSL's bet/bet2 with IDWI (double click on the protocol option and DTIPrep will allow you to select which masking option to choose)

Martin
Originally posted by Minchul Kim:
Hello ,
My name is MC kim, and I need some help.
The DTIPrep version I got is 1.2.4.
I try to preprocess my DWI images with DTIPrep, and I got some questions in doing it. 

1) I initially tried with 'default' protocol, and DTIPrep excluded 17 out for 31 gradients from a DWI data set from a subject.
the reason was due to interlace artifact. When I visual checked the DWI data, I couldn't agree that much artifact. 
I attached the captured QC result file.

I tried preprocessing again, and this time I changed the protocol, instead of using 'default'.
By default, DENOISING_bCheck, JOINDENOISING_bCheck, BRAINMAST_bCheck, and DOMINANTDIRECTION_bCheck at setted "No", and I changed settings into "YES"
As I tried again with all "YES" all of the directions were included in QCed DWI.
Thus I though by changing protocols to all "YES", the protocol became more tolerant to artifact.

My question is is it okay to run with the protocol setting of all "YES"? Is there any potential problem?

2) Even though the "default"protocol excluded more than half of DWI gradient directions, DTIPrep still generated DTI images such as FA, MD, etc.
Can I use this DTI image to analysis or tract generation? or should I prepration again with different protocol setting.

3)I read that DTIPrep has two nice functions, those are "vibration artifact checking" and "simulation based bias analysis".
I want to know which parameter on protocol setting represents those functions.
My guess is "DOMINANTDIRECTION_bCheck" maybe vibration artifact correction, am I right?

4) When I click on "Default" protocol, I get the error message saying "The following programs have not been found. Please enter the path manually if necessary> DiffusionWeightedVolumeMasking"
Can you tell me where can I get this program or how can I solve the problem?

Thanks in advance and I look forward to hear from you.

MC Kim
Attachment: tolerance.jpg
Apr 20, 2017  02:04 AM | Martin Styner
RE: Too many gradients excluded by "default"
Hi Minchul
Re 1) it's the one called badGradientPercentageTolerance
Re 2) the defaults are for a specific setting (pediatric human data) and a given diffusion acquisition protocol. If you have a different protocol, then the values would be different and need to be retrained
Martin
Originally posted by Minchul Kim:
Hi Martin
Thank you a lot for your kind advice, it was helpful.
I cameout with additional questions on your answer.

1) Upon your answer "Re 2)" , I want to set the tolerance threshold to whether create or not created final QCed DWI and DTIs.
I looked up at media wiki, but I cannot be sure which Parameter is for tolerance setting.
I have two guesses and marked "1" and "2" in attached image.
Please let me know the function and meanings of those two parameters.

2) Another question is that for "Re 3)", Am I supposed to train DTIPrep with my own artifact free data? Because I though DTIPrep has 
kind of built-in artifact free samples and compare with my data.

Thanks in advance and look forward to listen.

Minchul 
Originally posted by Martin Styner:
Hi Minchul
Re 1) the first denoising/smoothing option DENOISING_bCheck smooth the DWI data prior to any check which will smooth out any minor artifacts, which is why the previously rejected DWIs are now considered fine. It's "okay" to use that option, though we usually do not (I prefer un-smoothed data, but other researchers prefer smoothed data). This would definitely be something that you would need to report in any papers resulting from your work. The options JOINDENOISING_bCheck, BRAINMAST_bCheck, and DOMINANTDIRECTION_bCheck have no effect on the DWI cleaning/rejections.

What we usually do when too many DWI get rejected despite appearing appropriate is to change the thresholds for SLICE and INTERLACE checks. The report txt and xml files give you information which step rejects the most DWI and why. 

Re 2) That depends on your settings. You can tell DTIPrep not to create the final QCed DWI and DTI images when the resulting DWI has more than a certain percentage of rejected DWIs (tolerance threshold). At the most permissive (when data is very noisy), we usually do not go beyond 40% rejection, while our usual studies use 20-30% rejection tolerance. 50% seems to high to me and I would NOT use that data for further analysis.

re 3)DOMINANTDIRECTION_bCheck = vibration artifact check. This is not commonly needed unless you have earlier Tim Trio data or suspect presence of vibration artifacts. The issue here is that you need to train the classifier by running the check on sample data that looks good (the xml contains the entropy results, see our papers on this topic) and then use the mean & stdev from that sample data in the protocol to identify the presence or absence of such an artifact.

re 3) simulation based bias analysis: unfortunately that is not in DTIPrep yet. We simply don't have the manpower to put it in (there is no grant that support DTIPrep and we are currently only maintaining it and do not add any major functionality). This would be great as it would allow to reject/accept DTI data based on the expected error rather than a simple tolerance threshold.

re 4) DTIPrep offers support for multiple brain masking options. If you want to use that option you need to point DTIPrep to it (it's part of Slicer). We commonly suggest the use of FSL's bet/bet2 with IDWI (double click on the protocol option and DTIPrep will allow you to select which masking option to choose)

Martin
Originally posted by Minchul Kim:
Hello ,
My name is MC kim, and I need some help.
The DTIPrep version I got is 1.2.4.
I try to preprocess my DWI images with DTIPrep, and I got some questions in doing it. 

1) I initially tried with 'default' protocol, and DTIPrep excluded 17 out for 31 gradients from a DWI data set from a subject.
the reason was due to interlace artifact. When I visual checked the DWI data, I couldn't agree that much artifact. 
I attached the captured QC result file.

I tried preprocessing again, and this time I changed the protocol, instead of using 'default'.
By default, DENOISING_bCheck, JOINDENOISING_bCheck, BRAINMAST_bCheck, and DOMINANTDIRECTION_bCheck at setted "No", and I changed settings into "YES"
As I tried again with all "YES" all of the directions were included in QCed DWI.
Thus I though by changing protocols to all "YES", the protocol became more tolerant to artifact.

My question is is it okay to run with the protocol setting of all "YES"? Is there any potential problem?

2) Even though the "default"protocol excluded more than half of DWI gradient directions, DTIPrep still generated DTI images such as FA, MD, etc.
Can I use this DTI image to analysis or tract generation? or should I prepration again with different protocol setting.

3)I read that DTIPrep has two nice functions, those are "vibration artifact checking" and "simulation based bias analysis".
I want to know which parameter on protocol setting represents those functions.
My guess is "DOMINANTDIRECTION_bCheck" maybe vibration artifact correction, am I right?

4) When I click on "Default" protocol, I get the error message saying "The following programs have not been found. Please enter the path manually if necessary> DiffusionWeightedVolumeMasking"
Can you tell me where can I get this program or how can I solve the problem?

Thanks in advance and I look forward to hear from you.

MC Kim
Apr 20, 2017  02:04 AM | minchulusa
RE: Too many gradients excluded by "default"
Hi Martin

If I set badGradientPercentage Tolerance "0.2" ,run DTIPrep, and get QCed DWI and DTI created,
Can I assume that the percentage of excluded data is below 20%?

Minchul
Apr 20, 2017  02:04 AM | Martin Styner
RE: Too many gradients excluded by "default"
Yes
Martin
May 11, 2017  02:05 AM | minchulusa
RE: Too many gradients excluded by "default"
Hello , Martin

I got into another problem of DTIPrep excluding the initial B0 gradient dataset. 
And I think it's due to "DWIConvert" module.
My DWI data is in nifti format and I tried to convert the data into NRRD format.
I used "DWIConvert" I could convert my data, but faced a problem of unwanted change of b-values.

I used "FSLtoNRRD" command and gave input b-value text file with one B0 and B1000, 1500 and 2000 values.
The given txt. file is " 0 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 "

After converting, I re-converted NRRD file into Nifti format to check everything is fine.(because DTIPrep kept on excluding initial B0 gradient).
But the B-value cameout with Nifti file is not same as I gave as input.
The numbers were ramdomly changed.
The b-value text file cameout is
"569.14677169328775
1210.4240156296401
2394.278657453659
2064.6115111473659
2436.8009231442984
1705.7749517509521
1516.4484019703827
1514.1840056875694
1719.9054732742741
2824.3546439594234
2039.8874449352959
1211.1570127685525
2394.0072468217677
2064.9339802438881
2437.7675224831487
1705.874096430357
1516.5357335934659
1514.0713333670778
1719.814571234273
2824.4174183001828
2040.1616777009938
1210.592447597332
2394.3981996158004
2064.7939236817956
2437.6351466971118
1706.1506126118095
1516.6813482354526
1514.5959658969289
1719.3471588181069
2824.4694227411933
2214.1862132865399
1818.5253904163474
2379.8818229158201
1534.0663695825635
2377.2198780491276
1531.6037871520705
2376.2974797197626
1533.6721350225935
3030.4495159553903
1497.819497263901
1139.6441598187059
1817.9113032772889
2379.4458786272944
1533.8076305638847
2377.341528007285
1532.092919034837
2376.8870089482484
1533.7763507339953
3030.39438321854
1497.4963034622233
1139.4844454853051
1817.60332523135
2379.0966775756933
1533.7479203686992
2377.114705937805
1532.9149473982216
2376.9214811331449
1533.496245336315
3030.1965392951452
1497.2164576375399
485.30360327283682
2963.2369306481619
1239.161593096693
2396.5764449663329
1660.7085313620228
1508.0419153094763
1896.0191813079648
3153.7796491840281
730.87428877886327
1942.1368712267449
1008.437730083243
2963.446215401349
1240.038375975256
2397.1494366414922
1659.3181504339971
1507.376482833866
1894.6606209606766
3153.9309149673081
730.16050594589706
1942.6741487412278
1007.5010623141499
2963.5995044988094
1240.5890343827225
2397.4182379207482
1659.6068556891951
1507.0685735595321
1894.0214419438767
3153.877208544558
730.13306594795426
1943.4686143768183
1007.4178745724701"

And no B-value of "0", which should be resistration template.
I think when converting nifti to NRRD, unintented b-value change occurs.
I have tired with other datasets from other subjects and found this always occurs.

This maybe a problem with DWIConvert module, but I just want to ask have you ever faced and fixed this problem.
Or could I send you a sample dataset of mine for you to take a look?

Minchul
May 11, 2017  02:05 PM | Martin Styner
RE: Too many gradients excluded by "default"
This indeed looks like an issue for DWIConvert. Which version are you using, as earlier version had a known bug with multi-b-value images to go from NRRD to NIFTI (and back)? You may want to download it with the newest version of Slicer (maybe even a nightly build), or compile it yourself: https://github.com/BRAINSia/BRAINSTools/tree/master/DWIConvert

Btw, we do not maintain DWIConvert, it's a Slicer/BRAINS tool that we use (just like FSL's bet2)

Also, can you double-check the header of the NRRD file, whether the first DWI gradient is 0 0 0? This would narrow it down to see which of the 2 conversions messes things up.

Best
Martin
Originally posted by Minchul Kim:
Hello , Martin

I got into another problem of DTIPrep excluding the initial B0 gradient dataset. 
And I think it's due to "DWIConvert" module.
My DWI data is in nifti format and I tried to convert the data into NRRD format.
I used "DWIConvert" I could convert my data, but faced a problem of unwanted change of b-values.

I used "FSLtoNRRD" command and gave input b-value text file with one B0 and B1000, 1500 and 2000 values.
The given txt. file is " 0 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1000 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 1500 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 2000 "

After converting, I re-converted NRRD file into Nifti format to check everything is fine.(because DTIPrep kept on excluding initial B0 gradient).
But the B-value cameout with Nifti file is not same as I gave as input.
The numbers were ramdomly changed.
The b-value text file cameout is
"569.14677169328775
1210.4240156296401
2394.278657453659
2064.6115111473659
2436.8009231442984
1705.7749517509521
1516.4484019703827
1514.1840056875694
1719.9054732742741
2824.3546439594234
2039.8874449352959
1211.1570127685525
2394.0072468217677
2064.9339802438881
2437.7675224831487
1705.874096430357
1516.5357335934659
1514.0713333670778
1719.814571234273
2824.4174183001828
2040.1616777009938
1210.592447597332
2394.3981996158004
2064.7939236817956
2437.6351466971118
1706.1506126118095
1516.6813482354526
1514.5959658969289
1719.3471588181069
2824.4694227411933
2214.1862132865399
1818.5253904163474
2379.8818229158201
1534.0663695825635
2377.2198780491276
1531.6037871520705
2376.2974797197626
1533.6721350225935
3030.4495159553903
1497.819497263901
1139.6441598187059
1817.9113032772889
2379.4458786272944
1533.8076305638847
2377.341528007285
1532.092919034837
2376.8870089482484
1533.7763507339953
3030.39438321854
1497.4963034622233
1139.4844454853051
1817.60332523135
2379.0966775756933
1533.7479203686992
2377.114705937805
1532.9149473982216
2376.9214811331449
1533.496245336315
3030.1965392951452
1497.2164576375399
485.30360327283682
2963.2369306481619
1239.161593096693
2396.5764449663329
1660.7085313620228
1508.0419153094763
1896.0191813079648
3153.7796491840281
730.87428877886327
1942.1368712267449
1008.437730083243
2963.446215401349
1240.038375975256
2397.1494366414922
1659.3181504339971
1507.376482833866
1894.6606209606766
3153.9309149673081
730.16050594589706
1942.6741487412278
1007.5010623141499
2963.5995044988094
1240.5890343827225
2397.4182379207482
1659.6068556891951
1507.0685735595321
1894.0214419438767
3153.877208544558
730.13306594795426
1943.4686143768183
1007.4178745724701"

And no B-value of "0", which should be resistration template.
I think when converting nifti to NRRD, unintented b-value change occurs.
I have tired with other datasets from other subjects and found this always occurs.

This maybe a problem with DWIConvert module, but I just want to ask have you ever faced and fixed this problem.
Or could I send you a sample dataset of mine for you to take a look?

Minchul
May 18, 2017  08:05 AM | minchulusa
RE: Too many gradients excluded by "default"
I am replying to myself because I found the reason why B value change occurs.

I tried with transposed matrix of b value and b vectors and problem solved.

Also upgraded to newest version of DTIPrep.

Thanks for helping, Martin.

Minchul.
May 18, 2017  02:05 PM | Martin Styner
RE: Too many gradients excluded by "default"
Thanks Minchul
Yes, that's a common issue with DWIConvert, i.e. that the bvectors need to be in the transposed format.
Best regards
Martin