History Rewrite: Switch from using Git-LFS to DVC for storing large, compressed dataset files.

[Geekdude]

Important Update: To better support storing larger dataset archives used by HeCBench, we have switched from using a mix of adding these files in Git directly and using a free tier of Git-LFS to using DVC to store these larger files. This migration will make it easier to have more control of where these large files are hosted and make it easier to store larger files in the future. By doing a larger history rewrite, the large files that were tracked in Git are now tracked via DVC with their history intact.

• Before
  • LFS: 1.27 GiB
  • Repository: 430.72 MiB
• After
  • Repository: 210.4 MiB

Unfortunately, this large history rewrite will require rebasing any changes that you have made off of the main HeCBench branch. The recommended rebase procedure will look something like this:

git fetch
git checkout <your branch>
git rebase --onto origin/master master

Where master is the older version of HeCBench that you built your changes on. See Git rebase --onto an overview and Pro Git: 3.6 Git Branching - Rebasing for more information on how to rebase.

Alternatively, you can reset your copy of HeCBench to discard all changes or do a fresh clone of HeCBench and reapply your changes manually.

The quickstart instructions for using DVC are included in the main README.md.

This discussion page can be used to provide feedback or ask for help regarding this migration.

[zjin-lcf]

Thank you very much for your technical support and writing the migration guide!