Large File Storage (LFS) in GIT

Updated 1,041 Days AgoPublic

Actions

Filesize on c4science is limited to 100MB and push size to 250MB. For special cases we support GIT LFS for large file storage outside of the git index. Only a reference to the file is stored in git tracking the changes, the file itself is stored in phabricator's storage.

Installation

• Download and install the package for your operating system: https://github.com/github/git-lfs/releases
• Profit!

Usage

• In an existing git repository, initialize lfs and start tracking files,

git lfs install
git lfs track bigfile.bin
git lfs track *.png
# etc

• Add the file to git, commit and push

git add bigfile.bin
git add myimage.png
git add .gitattributes
git commit -m "Added a bigfile using LFS"
git push origin master

• You can also track a pattern of files, so every new file matching this pattern will be added using lfs:

git lfs track *.png

• Voila!

Convert already commited files to Git LFS

• We will use BFG Repository Cleaner
• Download the JAR from https://rtyley.github.io/bfg-repo-cleaner/
• Find big files extensions

find -type f -exec ls -lh {} \; | awk '{print $5,$9}' | sort -h

• Go to your git repository directory and use bfg with the extensions you found

java -jar bfg.jar --convert-to-git-lfs '*.{wav,pdf,avi}'

• Inspect your history to see the changes and clean dirty data to complete the procedure

git reflog expire --expire=now --all && git gc --prune=now --aggressive

• Activate Dangereous Changes from the repository configuration: Actions → Manage Repository → Basics → Actions → Allow Dangerous Changes

• Push the repository back to c4science

git push --force

• Deactivate Dangerous Changes (same as before)