I'm trying to use the fit_spline_xform tool to produce a transformation that can be used as the inverse of another transformation, generated by a non-rigid registration. To do this, I have a forwards transformation, A -> B, saved as forwards.list. I'm then using
to produce an inverse transformation. Using this transformation with reformatx, as
should, I think, give me an A', very similar to A. However, this is very much not the case, with A' being horrendously distorted. Have I done something wrong, or has the numerical inversion just failed rather poorly?

Hi Ashley -

It is of course hard to tell what exactly is going wrong there, but a simple test would be to use the inverse transformation directly in reformatx, without fitting, like so:
If this gives you the same bad result, then the problem is the numerical inversion.

Personally, I'd also like to see whether
gives you a reformatted version of A that looks reasonably well-matched to B, i.e., it would be good to make sure the forward transformation is reasonable to begin with.

Now regarding the fitting - let me mention that, technically speaking, the algorithm in fit_spline_xform is not exactly what you want for your purpose. Although, practically speaking, it is the only one currently available in CMTK for this purpose.

To be more specific - the fitting algorithm takes uniformly-distributed points in transformation source space (here: image space of A) and maps each into target space (here: image space B). These pairs of source and target points then define landmarks, and to these landmarks, the B-spline transformation is fitted. As you know, the applied transformation can include an inverse transformation, in which cases the mapping from source to target is numerically approximated.

So this is what happens. In your case, however, what you would want to happen is for the source points to be in space B and target points in space A, thus applying the original, forward nonrigid registration, but you would then want to fit a B-spline to the landmark pairs with source and targets exchanged. So while you would apply the original transform from B to A, you would be fitting a spline to landmarks with sources in A and targets in B, which would invert the original transformation at the fitting stage, not by numerical approximation.

Unfortunately, this requires a very different fitting implementation, since the source landmarks are no longer uniformly distributed, and so far this algorithm simply hasn't been implemented (nor is it likely it will be in the foreseeable future).

All that said - I took another close look at your command lines above, and I do wonder - why is "--floating" in the "reformatx" command line giving A.nrrd as the fixed image, rather than B.nrrd, which would be more logical? Also, is "A_registered_onto_B.nrrd" the moving image you used when computing forwards.list? Just another couple of things you may want to look into...

Hope this helps a little.


Best,
  Torsten

Hi Torsten,

Thanks for your reply. Using the inverse transformation directly results in a reasonable image which, combined with the fact that a spline fitted to the forwards transformation produces poor results suggests that the issue lies with the spline fitting. I'll try and explain what I've done from the start, especially as I seem to be slightly confused about the terminology you're using. I've also uploaded a diagram of what I've done to as an attachment, in case that's easier to follow.

Initially, we have two images, A.nrrd and B.nrrd, along with a transformation AtoB.list. Running
produces an image AtoB.nrrd, which appears to be a successful mapping of A onto B. Using
gives an image A_prime.nrrd, which is very similar to A.nrrd, suggesting that the inversion of the transformation is correct.

This is where I am first confused by the terminology. You asked about my command for the inverse transformation. Is it not the case that for this, we want to keep A fixed as this is what we are trying to transform onto? The outputted image has the right dimensions when A is listed as the fixed image, but has the dimensions of B (as I'd expect) when B is the fixed image, which is not correct for transforming AtoB back into A.

Fitting a B-spline to the inverse transformation with
and reformatting with
gives a very distorted A_dagger.nrrd. Just as a check, I also ran
which produces another very distorted image, but with the dimensions of B and not A.

I also tried to fit a B-spline to the forwards transformation to see how well that fared and, while better, it is still not good. Increasing the number of control points, even to the point where there is one control point per voxel, does not seem to improve matters. Setting the number of levels > 1 causes the number of levels to be reset to 1 "to ensure sufficient number of control points" (which means that the --levels 4 in my original command was pointless). I also don't understand what the --inversion-tolerance command does, as I don't understand why any B-spline should be inverted. Indeed, I can set the value to be stupidly small (say 1e-300) or stupidly large (1) and the end result doesn't change. Does this mean that if I use the --inverse command before a transformation that it won't actually invert the transformation, but instead fit a B-spline to the forwards transformation and then invert the B-spline? If so, how does that fit in with your explanation of the source and target landmarks in the forwards and inverse transformations?

Sorry for asking so many questions,
Ashley



EDIT: I've just had a look at the transformations themselves and noticed that the original forwards transformation has a domain of ~dim(B), while the spline fit has a domain of ~dim(A). This probably explains why the spline fits have severe warping near the edges. Changing the command for the fit so that B is the input, not A, results in reformatted images that are less warped, but have another distortion that looks a bit like they are interlaced.

Hi Torsten,

Greg and I have looked some more at this and we reckon the problem lies with the fitting of the B-splines. The gist at https://gist.github.com/ajdm/8281813/ outlines our testing and, surprisingly, it seems like fitting a B-spline to a forwards transformation works pretty well, but fitting one to an inverse doesn't. Playing around with the inversion tolerance improves matters somewhat, but in the direction opposite to what I'd expect, i.e. lowering the tolerance produces better results.

Also, the help for fit_spline_xform states: However, this does not seem to work, as -- is used to denote the end of non-positional arguments. For example, the command necessary is
and not
Alternatively, the -- can just be omitted.

Thanks,
Ashley

Hi Ashley (and Greg).

Thanks for your testing... you gave me exactly the hint I needed, namely that inverse xform fitting fails.

Basically, I failed to check in the fitting routine whether for any pixels the inversion failed. That lead to very large values to screw up the fitting. It also explains why higher tolerance made things better, because then at least a somewhat reasonable inverse transformed vector was computed (and fitted), rather than complete garbage.

Coincidentally, this also affected transformation sequences without inversion, because concatenating an affine with a B-spline transformation typically puts some coordinates outside the B-spline's support region, which also leads to invalid transformed points.

I just checked a fix for all this into SVN (r5111). Give it a try!

Thanks again for tracking this down!

Torsten

Hi Torsten,

Thanks for the quick patch. Unfortunately, it still looks like it's failing to produce reasonable images, as the gist (with modifications to use the newly built tools) outputs the same images irrespective of whether the latest version from SVN, or version 2.2.6 is used.

Thanks,
Ashley

Huh... well that is curious. Especially that the result should be exactly the same before and after the fix...

I will need to take a closer look, but I can't promise any quick answers.

Torsten

Okey dokey - another stupid bug fixed.

Try the latest svn - r5123.

Frankly, it is a mystery to me how even the forward fitting ever worked. It should have all been broken entirely.

Ah well, maybe it is now ;) (That is to say - test carefully before trusting this now!)

Cheers!
  Torsten

Hi Torsten,

That works great, thanks!

Ashley