Maybe I had the wrong expectation, but I assumed that if I created a job like this:


Then if 'myfile.txt' was missing, 'job' would be run. But what I actually see is that if I run psom_run_pipeline once, then delete 'myfile.txt', then run psom_run_pipeline again, 'job' is not rerun.

Was my expectation wrong, or is there some other way to trigger this behavior?

I somehow missed this post, sorry for the very late answer. PSOM does not automatically reprocess jobs with missing outputs. The main reason is that PSOM allows for "clean up" jobs, i.e. jobs that remove files. If you re-run a pipeline to change a job at the end, you will not expect to re-run all jobs that generate intermediate outputs, unless those are necessary to run the jobs that have changed. 

So, when re-running pipelines, any change in options or input/output file names will trigger a re-start of a job. If this job requires inputs that are missing, and that can be generated by other jobs, those will also be restarted (and that behavior is recursive).

You can force to restart the job called 'job' with the following option for `psom_run_pipeline`:

opt.restart = { 'job' };

There is no flag to restart jobs with missing outputs. Would not be hard to implement though, if you think this is useful. 

Best,

Pierre

Thanks for the reply!

Yes, I was thinking of a use case like make: a simple `make` or `make all` will regenerate everything necessary to make the `all` target, including to recreate any dependency files.

It sounds like a similar process for PSOM would be to create an 'all' job that depends on the others, or something similar. I'll play with it for my case.