local override
Created by: agraubert
As mentioned in the GDAN meeting, some people are interested in adding an option to save inputs to node-local storage instead of over the NFS. This is particularly useful for large input files which are only needed once and may clog NFS bandwidth and storage.
In terms of implementation, I think this makes sense as an override which follows the behavior of Delayed
, except that the file is downloaded to local storage (not over NFS). I'm leaning towards calling this override Local
, but it is very similar to Localize
, so it may not be the best choice.
I'm going to try to get to this today or tomorrow, and should have a PR open soon