Git/Shallow: Difference between revisions
< Git
| (2 intermediate revisions by the same user not shown) | |||
| Line 95: | Line 95: | ||
rm -rf .git/objects/* | rm -rf .git/objects/* | ||
## optional cleanup: rm -f .git/info/refs | ## optional cleanup: rm -f .git/info/refs | ||
## optional cleanup: rm -f .git/{COMMIT_EDITMSG,} | ## optional cleanup: rm -f .git/{COMMIT_EDITMSG,FETCH_HEAD} | ||
# optional: rm .git/hooks/* | # optional: rm .git/hooks/* | ||
# optional: rm -rf .git/lfs | # optional: rm -rf .git/lfs | ||
| Line 152: | Line 152: | ||
# [origin] | # [origin] | ||
# replace: fetch = +refs/heads/*:refs/remotes/origin/* | # replace: fetch = +refs/heads/*:refs/remotes/origin/* | ||
# with: fetch = +refs/heads | # with: fetch = +refs/heads/MYBRANCH:refs/remotes/origin/MYBRANCH | ||
git config --get remote.origin.fetch | git config --get remote.origin.fetch | ||
| Line 158: | Line 158: | ||
# git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" | # git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" | ||
## Limit fetch to just the one branch: | ## Limit fetch to just the one branch: | ||
git config remote.origin.fetch "+refs/heads | git config remote.origin.fetch "+refs/heads/MYBRANCH:refs/remotes/origin/MYBRANCH" | ||
git config --get remote.origin.fetch | git config --get remote.origin.fetch | ||
+refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH | +refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH | ||
| Line 171: | Line 171: | ||
git fetch | git fetch | ||
</pre> | </pre> | ||
=== Unshallow Clone Example === | |||
Deep: | |||
<pre> | |||
$ git clone org-123@github.com:MYORG/MYLARGEREPO.git deep | |||
Cloning into 'deep'... | |||
remote: Enumerating objects: 2261419, done. | |||
remote: Counting objects: 100% (5695/5695), done. | |||
remote: Compressing objects: 100% (1506/1506), done. | |||
remote: Total 2261419 (delta 5130), reused 4298 (delta 4183), pack-reused 2255724 (from 2) | |||
Receiving objects: 100% (2261419/2261419), 2.07 GiB | 14.10 MiB/s, done. | |||
Resolving deltas: 100% (1694598/1694598), done. | |||
Updating files: 100% (26095/26095), done. | |||
$ du deep --si --max-depth=0 | |||
3.4G deep | |||
</pre> | |||
Shallow: | |||
<pre> | |||
$ git clone org-123@github.com:MYORG/MYLARGEREPO.git shallow --depth 1 | |||
Cloning into 'shallow'... | |||
remote: Enumerating objects: 25232, done. | |||
remote: Counting objects: 100% (25232/25232), done. | |||
remote: Compressing objects: 100% (16961/16961), done. | |||
remote: Total 25232 (delta 9701), reused 17625 (delta 7165), pack-reused 0 (from 0) | |||
Receiving objects: 100% (25232/25232), 266.90 MiB | 10.68 MiB/s, done. | |||
Resolving deltas: 100% (9701/9701), done. | |||
Updating files: 100% (26095/26095), done. | |||
$ du shallow --si --max-depth=0 | |||
1.4G shallow | |||
</pre> | |||
Deepend and Depth and Unshallow: | |||
<pre> | |||
$ rsync -avrt shallow/ shallow-test/ | |||
$ cd shallow-test/ | |||
$ git fetch --deepen 1 | |||
remote: Enumerating objects: 37, done. | |||
remote: Counting objects: 100% (37/37), done. | |||
remote: Compressing objects: 100% (18/18), done. | |||
remote: Total 19 (delta 17), reused 2 (delta 0), pack-reused 0 (from 0) | |||
Unpacking objects: 100% (19/19), 2.31 KiB | 473.00 KiB/s, done. | |||
$ git fetch --deepen 1 | |||
remote: Enumerating objects: 173, done. | |||
remote: Counting objects: 100% (120/120), done. | |||
remote: Compressing objects: 100% (57/57), done. | |||
remote: Total 59 (delta 51), reused 9 (delta 2), pack-reused 0 (from 0) | |||
Unpacking objects: 100% (59/59), 8.22 KiB | 290.00 KiB/s, done. | |||
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) | |||
From github.com:MYORG/MYREPO | |||
* [new tag] Releases/SOMETAG1 -> Releases/SOMETAG1 | |||
* [new tag] Releases/SOMETAG2 -> Releases/SOMETAG2 | |||
$ git fetch --depth 1 | |||
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) | |||
$ git fetch --depth 2 | |||
remote: Enumerating objects: 13, done. | |||
remote: Counting objects: 100% (13/13), done. | |||
remote: Compressing objects: 100% (5/5), done. | |||
remote: Total 5 (delta 4), reused 1 (delta 0), pack-reused 0 (from 0) | |||
Unpacking objects: 100% (5/5), 1.36 KiB | 1.36 MiB/s, done. | |||
$ git fetch --depth 2 | |||
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) | |||
$ git fetch --depth 2 | |||
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) | |||
$ git fetch --unshallow | |||
remote: Enumerating objects: 1737369, done. | |||
remote: Counting objects: 100% (1719287/1719287), done. | |||
remote: Compressing objects: 100% (455766/455766), done. | |||
remote: Total 1710144 (delta 1268632), reused 1647283 (delta 1208565), pack-reused 0 (from 0) | |||
Receiving objects: 100% (1710144/1710144), 1010.28 MiB | 11.93 MiB/s, done. | |||
Resolving deltas: 100% (1268632/1268632), completed with 5822 local objects. | |||
remote: Enumerating objects: 9, done. | |||
remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 9 (from 1) | |||
Unpacking objects: 100% (9/9), 2.69 KiB | 2.69 MiB/s, done. | |||
From github.com:MYREPO/MYORG | |||
* [new tag] SOMETAG -> SOMETAG | |||
$ du ../shallow-test --si --max-depth=0 | |||
2.6G ../shallow-test | |||
--- | |||
# Fix refspec to not target single branch | |||
$ git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" | |||
$ git fetch --unshallow | |||
fatal: --unshallow on a complete repository does not make sense | |||
# OOPS! Needed to fix the refspec before running unshallow | |||
</pre> | |||
Full Unshallow Test: | |||
<pre> | |||
$ rm -rf shallow-test/ | |||
$ rsync -avrt shallow/ shallow-test/ | |||
$ cd shallow-test/ | |||
$ git fetch --unshallow | |||
remote: Enumerating objects: 1719297, done. | |||
remote: Counting objects: 100% (1719291/1719291), done. | |||
remote: Compressing objects: 100% (455705/455705), done. | |||
remote: Total 1710222 (delta 1268719), reused 1647432 (delta 1208703), pack-reused 0 (from 0) | |||
Receiving objects: 100% (1710222/1710222), 1010.08 MiB | 11.94 MiB/s, done. | |||
Resolving deltas: 100% (1268719/1268719), completed with 5822 local objects. | |||
remote: Enumerating objects: 9, done. | |||
remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 9 (from 1) | |||
Unpacking objects: 100% (9/9), 2.69 KiB | 2.69 MiB/s, done. | |||
From github.com:MYREPO/MYORG | |||
* [new tag] Releases/SOMETAG1 -> Releases/SOMETAG1 | |||
* [new tag] Releases/SOMETAG2 -> Releases/SOMETAG2 | |||
* [new tag] SOMETAG -> SOMETAG | |||
$ du ../shallow-test --si --max-depth=0 | |||
2.6G ../shallow-test | |||
--- | |||
# Fix refspec to not target single branch | |||
$ git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" | |||
$ git fetch --unshallow | |||
fatal: --unshallow on a complete repository does not make sense | |||
# OOPS! Needed to fix the refspec before running unshallow | |||
</pre> | |||
Full No-Single-Branch Unshallow Test: | |||
<pre> | |||
$ rm -rf shallow-test/ | |||
$ rsync -avrt shallow/ shallow-test/ | |||
$ cd shallow-test/ | |||
# Fix refspec: | |||
$ git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" | |||
$ git fetch --unshallow | |||
??? | |||
??? SOOO MUCH is scrolled off the screen passed my buffer | |||
??? | |||
* [new tag] ... | |||
* [new tag] ... | |||
* [new tag] ... | |||
$ du ../shallow-test --si --max-depth=0 | |||
2.6G ../shallow-test | |||
</pre> | |||
== keywords == | |||
Latest revision as of 06:49, 24 March 2026
Shallow Clone
git clone --filter=blob:none <url> creates a blobless clone. ... git clone --filter=tree:0 <url> creates a treeless clone. ... git clone --depth=1 <url> creates a shallow clone.
git clone --depth=1 <url> -b <branch> # clone at branch
References:
- Get up to speed with partial clone and shallow clone - The GitHub Blog - https://github.blog/open-source/git/get-up-to-speed-with-partial-clone-and-shallow-clone/
Unshallow Clone
To Unshallow a Shallow Clone
git clone <url> --depth 1 git fetch --unshallow
vs
git clone <url>
Incrementally:
git clone <url> --depth 1
git fetch --deepen git fetch --deepen ... # or git fetch --depth 3 git fetch --depth 6 ..
git fetch --unshallow
Reference:
- git "shallow clone + unshallow" vs "normal clone" - Stack Overflow - https://stackoverflow.com/questions/67838180/git-shallow-clone-unshallow-vs-normal-clone
NOTE: Eventually, the final --unshallow gets you everything you would have gotten, had you been able to do a full clone all at once without error. Note that you may want to use --no-single-branch during the initial shallow clone, or fix up the fetch refspec after the initial shallow clone.
Convert Cloned Repo to Shallow
git clone --depth-1 <repo> -b <branch> git clone --depth 1 org-1234@github.com:MYORG/MYREPO.git -b MYREPO/MYBRANCH MYBRANCH git clean -x -f -d . git pull --depth 1 git fetch --depth 1 git reflog expire --expire=0 git reflog expire --expire=now --all git show-ref -s HEAD git show-ref -s HEAD > .git/shallow git prune git prune-packed git repack git pull --shallow-since=1999-01-01 git pull --shallow-since=2026-03-19 git gc git gc --prune=all git gc --prune=now git gc --aggressive git tag -d $(git tag -l) git fsck --full --no-reflogs # fix: # $ git gc # warning: reflog of 'HEAD' references pruned commits git reflog expire --all --stale-fix git gc du --si --max-depth=1
References:
- Converting git repository to shallow? - Stack Overflow - https://stackoverflow.com/questions/4698759/converting-git-repository-to-shallow
UPDATED BRUTE FORCE CONVERSION METHOD
git config remote.origin.fetch "+refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH"
rm -rf .git/objects/*
## optional cleanup: rm -f .git/info/refs
## optional cleanup: rm -f .git/{COMMIT_EDITMSG,FETCH_HEAD}
# optional: rm .git/hooks/*
# optional: rm -rf .git/lfs
# optional: rm -rf .git/refs/*
# optional: rm -rf .git/logs/*
rm .git/packed-refs
git reflog expire --all --stale-fix
git fetch
git checkout MYREPO/MYBRANCH
git pull
git gc
du --si --max-depth=1
BRUTE FORCE CONVERSION METHOD
cd .git # delete top level files --- # dir: branches/ hooks/ info/ lfs/ logs/ objects/ refs/ # files: COMMIT_EDITMSG config description FETCH_HEAD HEAD index ORIG_HEAD packed-refs shallow # removes extras: COMMIT_EDITMSG, FETCH_HEAD # ignore unused: description rm -f * # remove lfs/ rm -rf lfs/ # clear logs/ and sub-dirs rm -rf logs/* # clear hooks/ rm hooks/* # clear write-projected objects/pack #rm -f objects/pack/* #rm objects/info/packs rm -rf objects/* # remove refs rm -rf refs/* rm info/refs rm COMMIT_EDITMSG rm packed-refs # will be recreated: rm FETCH_HEAD # make origin head shallow cat ORIG_HEAD > shallow # fix config: # [origin] # replace: fetch = +refs/heads/*:refs/remotes/origin/* # with: fetch = +refs/heads/MYBRANCH:refs/remotes/origin/MYBRANCH git config --get remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" ## Limit fetch to just the one branch: git config remote.origin.fetch "+refs/heads/MYBRANCH:refs/remotes/origin/MYBRANCH" git config --get remote.origin.fetch +refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH # copy shallow copies .git/* -- config, description, FETCH_HEAD, HEAD, ORIG_HEAD, packed-refs, shallow # extras: cp ../../shallow/.git/* ./ cd .. # Fetch - should only fetch the current branch only git fetch
Unshallow Clone Example
Deep:
$ git clone org-123@github.com:MYORG/MYLARGEREPO.git deep Cloning into 'deep'... remote: Enumerating objects: 2261419, done. remote: Counting objects: 100% (5695/5695), done. remote: Compressing objects: 100% (1506/1506), done. remote: Total 2261419 (delta 5130), reused 4298 (delta 4183), pack-reused 2255724 (from 2) Receiving objects: 100% (2261419/2261419), 2.07 GiB | 14.10 MiB/s, done. Resolving deltas: 100% (1694598/1694598), done. Updating files: 100% (26095/26095), done. $ du deep --si --max-depth=0 3.4G deep
Shallow:
$ git clone org-123@github.com:MYORG/MYLARGEREPO.git shallow --depth 1 Cloning into 'shallow'... remote: Enumerating objects: 25232, done. remote: Counting objects: 100% (25232/25232), done. remote: Compressing objects: 100% (16961/16961), done. remote: Total 25232 (delta 9701), reused 17625 (delta 7165), pack-reused 0 (from 0) Receiving objects: 100% (25232/25232), 266.90 MiB | 10.68 MiB/s, done. Resolving deltas: 100% (9701/9701), done. Updating files: 100% (26095/26095), done. $ du shallow --si --max-depth=0 1.4G shallow
Deepend and Depth and Unshallow:
$ rsync -avrt shallow/ shallow-test/ $ cd shallow-test/ $ git fetch --deepen 1 remote: Enumerating objects: 37, done. remote: Counting objects: 100% (37/37), done. remote: Compressing objects: 100% (18/18), done. remote: Total 19 (delta 17), reused 2 (delta 0), pack-reused 0 (from 0) Unpacking objects: 100% (19/19), 2.31 KiB | 473.00 KiB/s, done. $ git fetch --deepen 1 remote: Enumerating objects: 173, done. remote: Counting objects: 100% (120/120), done. remote: Compressing objects: 100% (57/57), done. remote: Total 59 (delta 51), reused 9 (delta 2), pack-reused 0 (from 0) Unpacking objects: 100% (59/59), 8.22 KiB | 290.00 KiB/s, done. remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) From github.com:MYORG/MYREPO * [new tag] Releases/SOMETAG1 -> Releases/SOMETAG1 * [new tag] Releases/SOMETAG2 -> Releases/SOMETAG2 $ git fetch --depth 1 remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) $ git fetch --depth 2 remote: Enumerating objects: 13, done. remote: Counting objects: 100% (13/13), done. remote: Compressing objects: 100% (5/5), done. remote: Total 5 (delta 4), reused 1 (delta 0), pack-reused 0 (from 0) Unpacking objects: 100% (5/5), 1.36 KiB | 1.36 MiB/s, done. $ git fetch --depth 2 remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) $ git fetch --depth 2 remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) $ git fetch --unshallow remote: Enumerating objects: 1737369, done. remote: Counting objects: 100% (1719287/1719287), done. remote: Compressing objects: 100% (455766/455766), done. remote: Total 1710144 (delta 1268632), reused 1647283 (delta 1208565), pack-reused 0 (from 0) Receiving objects: 100% (1710144/1710144), 1010.28 MiB | 11.93 MiB/s, done. Resolving deltas: 100% (1268632/1268632), completed with 5822 local objects. remote: Enumerating objects: 9, done. remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 9 (from 1) Unpacking objects: 100% (9/9), 2.69 KiB | 2.69 MiB/s, done. From github.com:MYREPO/MYORG * [new tag] SOMETAG -> SOMETAG $ du ../shallow-test --si --max-depth=0 2.6G ../shallow-test --- # Fix refspec to not target single branch $ git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" $ git fetch --unshallow fatal: --unshallow on a complete repository does not make sense # OOPS! Needed to fix the refspec before running unshallow
Full Unshallow Test:
$ rm -rf shallow-test/ $ rsync -avrt shallow/ shallow-test/ $ cd shallow-test/ $ git fetch --unshallow remote: Enumerating objects: 1719297, done. remote: Counting objects: 100% (1719291/1719291), done. remote: Compressing objects: 100% (455705/455705), done. remote: Total 1710222 (delta 1268719), reused 1647432 (delta 1208703), pack-reused 0 (from 0) Receiving objects: 100% (1710222/1710222), 1010.08 MiB | 11.94 MiB/s, done. Resolving deltas: 100% (1268719/1268719), completed with 5822 local objects. remote: Enumerating objects: 9, done. remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 9 (from 1) Unpacking objects: 100% (9/9), 2.69 KiB | 2.69 MiB/s, done. From github.com:MYREPO/MYORG * [new tag] Releases/SOMETAG1 -> Releases/SOMETAG1 * [new tag] Releases/SOMETAG2 -> Releases/SOMETAG2 * [new tag] SOMETAG -> SOMETAG $ du ../shallow-test --si --max-depth=0 2.6G ../shallow-test --- # Fix refspec to not target single branch $ git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" $ git fetch --unshallow fatal: --unshallow on a complete repository does not make sense # OOPS! Needed to fix the refspec before running unshallow
Full No-Single-Branch Unshallow Test:
$ rm -rf shallow-test/ $ rsync -avrt shallow/ shallow-test/ $ cd shallow-test/ # Fix refspec: $ git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" $ git fetch --unshallow ??? ??? SOOO MUCH is scrolled off the screen passed my buffer ??? * [new tag] ... * [new tag] ... * [new tag] ... $ du ../shallow-test --si --max-depth=0 2.6G ../shallow-test