[Repronim-trd3] no meetup today - let's just work

Kyle Meyer kyle at kyleam.com
Thu Oct 8 12:22:17 PDT 2020


Haselgrove, Christian writes:

> And how are jobs split into multiple processes?  In other words, in
> what cases does the executing environment ever pass anything but 0 to
> runscript?

Ah, sorry for not addressing that with my previous reply.  The number of
subjobs is determined by run's --batch-parameter/--batch-spec.  These
are used in orchestrators.py to construct the _command_array item of the
job spec.  The length of this array is taken as the number of subjobs:

    def submit(self):
        """Submit the job with `submitter`.
        """
        njobs = len(self.job_spec["_command_array"])
        [...]

That value gets injected into the submission template to control the
number of subjobs.  So, to use condor as an example: with a command
array with 4 items, this bit of the condor.template

    arguments = "$(Process)"
    queue {{ _num_subjobs }}

gets expanded into

    arguments = "$(Process)"
    queue 4

Submitting that leads to condor executing

    [...]/runscript 1
    [...]/runscript 2
    [...]/runscript 3
    [...]/runscript 4


More information about the Repronim-trd3 mailing list