[Repronim-trd3] no meetup today - let's just work
Kyle Meyer
kyle at kyleam.com
Thu Oct 8 12:22:17 PDT 2020
Haselgrove, Christian writes:
> And how are jobs split into multiple processes? In other words, in
> what cases does the executing environment ever pass anything but 0 to
> runscript?
Ah, sorry for not addressing that with my previous reply. The number of
subjobs is determined by run's --batch-parameter/--batch-spec. These
are used in orchestrators.py to construct the _command_array item of the
job spec. The length of this array is taken as the number of subjobs:
def submit(self):
"""Submit the job with `submitter`.
"""
njobs = len(self.job_spec["_command_array"])
[...]
That value gets injected into the submission template to control the
number of subjobs. So, to use condor as an example: with a command
array with 4 items, this bit of the condor.template
arguments = "$(Process)"
queue {{ _num_subjobs }}
gets expanded into
arguments = "$(Process)"
queue 4
Submitting that leads to condor executing
[...]/runscript 1
[...]/runscript 2
[...]/runscript 3
[...]/runscript 4
More information about the Repronim-trd3
mailing list