Hi Developers,

If I need to run cmtk on a multi-node cluster, should I rebuild the package from the source based on the cluster configurations, or can I just copy the pre-compiled binaries to my home directory under the cluster? If I need to rebuild the package, how can I configure it without CMake?

Best,
Dan

Hi Dan -

There are only three reasons to rebuild CMTK:

• The pre-built binaries are for a more recent CPU generation than what you have, resulting in crashes.
• The pre-built binaries are built for a CPU that doesn't support all the fancy features of your CPUs, and you expect a serious performance gain from optimizing for your specific machines.
• The pre-built binaries depend on a shared system library that is not available on your systems, or is available only as a different, non-compatible version.

I seriously doubt that you'd run into 1., since I build the releases for fairly vanilla CPU configs. Regarding 2., any performance gains would likely be very small, on the order of single-digit percentages, and you'd also need to have a compiler that actually can take advantage of whatever new CPU features there are.

As for 3. - you may be okay just installing a missing library dependency. If you do have incompatible shared libraries, or if the missing library just isn't available on your system, then indeed you'd have to rebuild the binaries.

But if you do rebuild, you will need CMake. It's the only config/build system that CMTK supports. Why would that be a problem though? It seems CMake is available on pretty much every platform you could wish for.

Best,
  Torsten

Thanks Torsten!  I extracted the pre-compiled binaries to my cluster account but it crashes at every attempt. Luckily the cluster has cmake and I have rebuilt the package using:

tar -xvf CMTK-3.3.1-Source.tar.gz
mkdir cmtk-build
cd cmtk-build
cmake ../cmtk-3.3.1
make

The building process seems successful except munger is missing in cmtk-build/bin. I remember manually enabling munger generation via the cmake GUI on my laptop,  but how can I do this via command line? I saw a couple of configuration files in the folder cmtk-build, but none of them contains the string "munger" or "MUNGER".

Sorry for so many silly questions.... :(

Best,
Dan

You need to build the optional tools. I forget the exact option but you can set it by running the text based gui from the command line. Just replace cmake with ccmake in your command sequence below.

OK that works (finally! ) Thanks a lot and have a nice weekend! :D

Cheers,
Dan

Hi Greg,

I have a few more questions regarding multi-threading in cmtk:

1. when building the package on a cluster, should I specify the number of threads to enable parallelization or I can just specify that number in the munger script (following the flag -T) ?

2. when submitting jobs using qsub, which parallel environment (for instance, SMP, openmpi or mpich2) should I specify?

3. To execute a parallel job, should I add "mpirun" ahead of the command line below?

"/netapp/home/danustc/cmtk-build/bin/munger" -b "/netapp/home/danustc/cmtk-build/bin" -a -w -r 010203 -X 26 -C 8 -G 80 -R 4 -A '--accuracy 0.4' -W '--accuracy 0.4' -v -T 2 -s "refbrain/Nov012016B3ZDref.nrrd" "images"

Thanks!

Dan

Hi Dan -

You do not need to specify a number of threads at build time. In theory, you need to enable support for multi-threading and OpenMP, but since these default to "on", there's really nothing to do.

At runtime, by default, a CMTK tool will use all available CPU cores. Obviously you'd want to reduce the number of threads if you're sharing a machine etc. For that, every CMTK tool accepts the "--threads N" command line argument, where N is the number of threads. Maybe Greg can comment on if/how that is exposed by munger.

Re qsub - I think that very much depends on how your cluster is configured, but I can definitely say, that CMTK does not use MPI. So the only option on your list that makes any sense is "SMP". But I don't think that's a predefined configuration of your cluster management software (what is it anyway, OpenPBS, SGE, ...?). More likely something your admins have set up to mean something specific. In that case, you'd want to consult with them.

And regarding mpirun - the answer is, no, absolutely not. No MPI support in CMTK, so no point in using mpirun.

Hope this helps.

Torsten