Hi:
I am trying to run a group registration of 6 multi-photon confocal stacks, and I am getting an error I have not received before. I am running on an Ubuntu 10.13 16-core processor sever.  Pairwise registration seems to run fin

After the error, I went back and made the command as basic as possible:
groupwise_init -O outputfile/initial -v --align-centers-of-mass image1.nrrd image 2.nrrd image3.nrrd image 4.nrrd image5.nrrd image6.nrrd


The error I get is as follows:

Template grid is 1817 x 2561 x 780 pixels of size 0.610000 x 0.610000 x 0.610000
Reformating output images
Caught signal 11, faulty address is (nil)
[stack] Execution path:
[stack] groupwise_init(cmtkStackBacktraceSignalHandler+0x55) [0x591008]
[stack] /lib/x86_64-linux-gnu/libpthread.so.0(+0xfbb0) [0x7f070fd16bb0]
[stack] groupwise_init(_ZN4cmtk13TemplateArrayIhE12SetPaddingAtEm+0x4a) [0x55bd02]
[stack] groupwise_init(_ZN4cmtk14ReformatVolume13PlainReformatEv+0x603) [0x4de8b9]
[stack] groupwise_init(_ZN4cmtk27GroupwiseRegistrationOutput17WriteAverageImageEPKcNS_13Interpolators17InterpolationEnumEb+0x677) [0x4dca23]
[stack] groupwise_init(_Z6doMainiPPKc+0x1181) [0x4c6cce]
[stack] groupwise_init(main+0x31) [0x4c7422]
[stack] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f070eb0ade5]
[stack] groupwise_init() [0x4c5a89]
ERROR: could not find file groupwise/initial/groupwise_init.xforms
ERROR: could not open file "groupwise/affine/groupwise_rmi.xforms" with mode "r"
Input archive groupwise/affine/groupwise_rmi.xforms could not be opened for reading.


Has anyone run into this error before?
Thanks!


NOTE: the commands are running in a bash script so hence the reason for the:
ERROR: could not find file groupwise/initial/groupwise_init.xforms
ERROR: could not open file "groupwise/affine/groupwise_rmi.xforms" with mode "r"
Input archive groupwise/affine/groupwise_rmi.xforms could not be opened for reading.
messages.

Hi Erik -

I have not seen this before, and I can't quite see from the stack trace alone what might be happening here.

Could you post the exact command line for the registration call here?

Torsten

Toresen:

The exact Comman line is:
groupwise_init -O groupwise/initial -v-align-centers-of-mass \
Image_Reg_Larval1_GFP.nrrd \
Image_Reg_Larval2_GFP.nrrd \
Image_Reg_Larval3_GFP.nrrd \
Image_Reg_Larval4_GFP.nrrd \
Image_Reg_Larval4_GFP.nrrd \
Image_Reg_Larval6_GFP.nrrd

obviously the '\'s are bit ub the command - they are just easier to see
Thanks!
-Erik

Erik,

Trying just the command line but with my own data, I cannot see a crash. So I suspect that there is something about your specific images that causes problems.

Would you mind making your files available to me for testing? Perhaps put them somewhere for download and send me the link by private email to rohlfing@ieee.org?

Torsten

Erik -

I tried to run the groupwise_init with your data. Of course I had forgotten that I no longer have access to a machine with really large memory... so I had to test with downsampled data.

Since that seems to work fine, let me ask you - how much memory does the machine have that you are trying to run this on? (While the error you are seeing doesn't look like you are running out of memory at first sight, I would still like to rule it out.)

That aside, since the script you sent me doesn't actually make any use of the initial group-averaged image, you could also add "--no-output-average" to the groupwise_init command line. That would prevent computation of the average image, thereby preventing the step that currently crashes.

Unfortunately, in trying this I found a bug that actually prevents you from doing this right now, since "--no-output-average" will crash for unrelated reasons. The fix for this problem is trivial but a little involved, so it will take me a few days to actually do it.

Best,
Torsten

Torsten:
The computing cluster I am running this on has a total of 48GB of memory.  I have also tried running this command with only three samples and got the same error, so my guess is that it is not a memory issue.
Any advice you may have is greatly appreciated!
- Erik

OK, so I would say, at this point the most promising option is for me to fix that crash I mentioned related to suppressing the generation of the initial group average image, which apparently you aren't using anyway. Even if we cannot sort out what causes your original problem, hopefully this will make it possible for you to just ignore that issue and at least get your computations done.

Give me a few days. I'll let you know when it's done.

TR

Hi Erik -

Probably about time for an update.

I looked into making groupwise_init work without creating an average image (the part that seems to be crashing for you), but it turns out that that isn't actually that simple to do. Long story short, the transformation files that are being created have to refer to some common space across all input images, and that common space is provided by the average. No average, no common space, no transformations.

That said, the latest code in the svn repository adds a bunch of command line options that allow you to pick a different pixel data type (previously, the type was float). So you could use, say, "byte", to greatly reduce memory consumption. In addition, I have changed an internal array from short to byte, which also reduces the amount of memory needed. Just in case this was indeed originally a memory issue, you may want to give the latest code a try (assuming you build your own binaries; otherwise let me know and I'll make some for you).

Sorry I don't have a complete solution, but maybe this is going to help you move forward.

Torsten

Erik,

There is a good chance that the root cause of your problem is a limitation of CMTK to 4GB maximum image size. See thread here: https://www.nitrc.org/forum/forum.php?thread_id=6300&forum_id=857

I am working on fixing that limitation, and while it's likely going to take a little while to make all the necessary changes, I'd say there is hope now.

Best,
  Torsten

I just released CMTK 3.3.0; 64bit binaries for Linux/Windows/Cygwin are in the Downloads area. See if these fix your problem.

Torsten