Skip to content

Adding Call Cache

Aaron Graubert requested to merge callcache into master

Created by: agraubert

Lapdog Call Caching System

Overview

Each workspace will contain an independent call cache, which can be used to skip previously succeeded jobs. As a cache-enabled submission starts, cromwell will download the current workspace call cache to it's local mysql database. After the submission ends, the updated state of the database is saved back to the workspace bucket.

Todo

  • 100 Workflow scale test
  • 100 Workflow cache hit test
  • Memory Benchmarking
  • Deprecate Query Limit and remove from UI
  • Add Enable/Disable Call Cache as option to UI and python
  • 100 Workflow scale test (cache disabled)
  • 100 Workflow scale test (cache enabled)
  • 1000 Workflow scale test
  • Test concurrent call cache overwrites
  • Add dynamic scaling boot disk size: 20 + max(0, cache - 10)

Merge request reports