Adding Call Cache
Created by: agraubert
Lapdog Call Caching System
Overview
Each workspace will contain an independent call cache, which can be used to skip previously succeeded jobs. As a cache-enabled submission starts, cromwell will download the current workspace call cache to it's local mysql database. After the submission ends, the updated state of the database is saved back to the workspace bucket.
Todo
-
100 Workflow scale test -
100 Workflow cache hit test -
Memory Benchmarking -
Deprecate Query Limit
and remove from UI -
Add Enable/Disable Call Cache
as option to UI and python -
100 Workflow scale test (cache disabled) -
100 Workflow scale test (cache enabled) -
1000 Workflow scale test -
Test concurrent call cache overwrites -
Add dynamic scaling boot disk size: 20 + max(0, cache - 10)