Git/Shallow: Difference between revisions
< Git
| Line 4: | Line 4: | ||
git clone --filter=tree:0 <url> creates a treeless clone. ... | git clone --filter=tree:0 <url> creates a treeless clone. ... | ||
git clone --depth=1 <url> creates a shallow clone. | git clone --depth=1 <url> creates a shallow clone. | ||
== Unshallow Clone == | |||
To Unshallow a Shallow Clone | |||
git clone <url> --depth 1 | |||
git fetch --unshallow | |||
vs | |||
git clone <url> | |||
Incrementally: | |||
git clone <url> --depth 1 | |||
git fetch --deepen | |||
git fetch --deepen | |||
... | |||
# or | |||
git fetch --depth 3 | |||
git fetch --depth 6 | |||
.. | |||
git fetch --unshallow | |||
Reference: | |||
* git "shallow clone + unshallow" vs "normal clone" - Stack Overflow - https://stackoverflow.com/questions/67838180/git-shallow-clone-unshallow-vs-normal-clone | |||
=== Example === | |||
Deep: | |||
<pre> | |||
$ git clone org-123@github.com:MYORG/MYLARGEREPO.git deep | |||
Cloning into 'deep'... | |||
remote: Enumerating objects: 2261419, done. | |||
remote: Counting objects: 100% (5695/5695), done. | |||
remote: Compressing objects: 100% (1506/1506), done. | |||
remote: Total 2261419 (delta 5130), reused 4298 (delta 4183), pack-reused 2255724 (from 2) | |||
Receiving objects: 100% (2261419/2261419), 2.07 GiB | 14.10 MiB/s, done. | |||
Resolving deltas: 100% (1694598/1694598), done. | |||
Updating files: 100% (26095/26095), done. | |||
$ du deep --si --max-depth=0 | |||
3.4G deep | |||
</pre> | |||
Shallow: | |||
<pre> | |||
$ git clone org-123@github.com:MYORG/MYLARGEREPO.git shallow --depth 1 | |||
Cloning into 'shallow'... | |||
remote: Enumerating objects: 25232, done. | |||
remote: Counting objects: 100% (25232/25232), done. | |||
remote: Compressing objects: 100% (16961/16961), done. | |||
remote: Total 25232 (delta 9701), reused 17625 (delta 7165), pack-reused 0 (from 0) | |||
Receiving objects: 100% (25232/25232), 266.90 MiB | 10.68 MiB/s, done. | |||
Resolving deltas: 100% (9701/9701), done. | |||
Updating files: 100% (26095/26095), done. | |||
$ du shallow --si --max-depth=0 | |||
1.4G shallow | |||
</pre> | |||
Deepend and Depth and Unshallow: | |||
<pre> | |||
$ rsync -avrt shallow/ shallow-test/ | |||
$ cd shallow-test/ | |||
$ git fetch --deepen 1 | |||
remote: Enumerating objects: 37, done. | |||
remote: Counting objects: 100% (37/37), done. | |||
remote: Compressing objects: 100% (18/18), done. | |||
remote: Total 19 (delta 17), reused 2 (delta 0), pack-reused 0 (from 0) | |||
Unpacking objects: 100% (19/19), 2.31 KiB | 473.00 KiB/s, done. | |||
$ git fetch --deepen 1 | |||
remote: Enumerating objects: 173, done. | |||
remote: Counting objects: 100% (120/120), done. | |||
remote: Compressing objects: 100% (57/57), done. | |||
remote: Total 59 (delta 51), reused 9 (delta 2), pack-reused 0 (from 0) | |||
Unpacking objects: 100% (59/59), 8.22 KiB | 290.00 KiB/s, done. | |||
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) | |||
From github.com:MYORG/MYREPO | |||
* [new tag] Releases/SOMETAG1 -> Releases/SOMETAG1 | |||
* [new tag] Releases/SOMETAG2 -> Releases/SOMETAG2 | |||
$ git fetch --depth 1 | |||
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) | |||
$ git fetch --depth 2 | |||
remote: Enumerating objects: 13, done. | |||
remote: Counting objects: 100% (13/13), done. | |||
remote: Compressing objects: 100% (5/5), done. | |||
remote: Total 5 (delta 4), reused 1 (delta 0), pack-reused 0 (from 0) | |||
Unpacking objects: 100% (5/5), 1.36 KiB | 1.36 MiB/s, done. | |||
$ git fetch --depth 2 | |||
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) | |||
$ git fetch --depth 2 | |||
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) | |||
$ git fetch --unshallow | |||
$ du ../shallow --si --max-depth=0 | |||
</pre> | |||
Full Unshallow Test: | |||
<pre> | |||
$ rm -rf shallow-test/ | |||
$ rsync -avrt shallow/ shallow-test/ | |||
$ cd shallow-test/ | |||
$ git fetch --unshallow | |||
$ du ../shallow --si --max-depth=0 | |||
</pre> | |||
== Convert Cloned Repo to Shallow == | == Convert Cloned Repo to Shallow == | ||
Revision as of 23:04, 22 March 2026
Shallow Clone
git clone --filter=blob:none <url> creates a blobless clone. ... git clone --filter=tree:0 <url> creates a treeless clone. ... git clone --depth=1 <url> creates a shallow clone.
Unshallow Clone
To Unshallow a Shallow Clone
git clone <url> --depth 1 git fetch --unshallow
vs
git clone <url>
Incrementally:
git clone <url> --depth 1
git fetch --deepen git fetch --deepen ... # or git fetch --depth 3 git fetch --depth 6 ..
git fetch --unshallow
Reference:
- git "shallow clone + unshallow" vs "normal clone" - Stack Overflow - https://stackoverflow.com/questions/67838180/git-shallow-clone-unshallow-vs-normal-clone
Example
Deep:
$ git clone org-123@github.com:MYORG/MYLARGEREPO.git deep Cloning into 'deep'... remote: Enumerating objects: 2261419, done. remote: Counting objects: 100% (5695/5695), done. remote: Compressing objects: 100% (1506/1506), done. remote: Total 2261419 (delta 5130), reused 4298 (delta 4183), pack-reused 2255724 (from 2) Receiving objects: 100% (2261419/2261419), 2.07 GiB | 14.10 MiB/s, done. Resolving deltas: 100% (1694598/1694598), done. Updating files: 100% (26095/26095), done. $ du deep --si --max-depth=0 3.4G deep
Shallow:
$ git clone org-123@github.com:MYORG/MYLARGEREPO.git shallow --depth 1 Cloning into 'shallow'... remote: Enumerating objects: 25232, done. remote: Counting objects: 100% (25232/25232), done. remote: Compressing objects: 100% (16961/16961), done. remote: Total 25232 (delta 9701), reused 17625 (delta 7165), pack-reused 0 (from 0) Receiving objects: 100% (25232/25232), 266.90 MiB | 10.68 MiB/s, done. Resolving deltas: 100% (9701/9701), done. Updating files: 100% (26095/26095), done. $ du shallow --si --max-depth=0 1.4G shallow
Deepend and Depth and Unshallow:
$ rsync -avrt shallow/ shallow-test/ $ cd shallow-test/ $ git fetch --deepen 1 remote: Enumerating objects: 37, done. remote: Counting objects: 100% (37/37), done. remote: Compressing objects: 100% (18/18), done. remote: Total 19 (delta 17), reused 2 (delta 0), pack-reused 0 (from 0) Unpacking objects: 100% (19/19), 2.31 KiB | 473.00 KiB/s, done. $ git fetch --deepen 1 remote: Enumerating objects: 173, done. remote: Counting objects: 100% (120/120), done. remote: Compressing objects: 100% (57/57), done. remote: Total 59 (delta 51), reused 9 (delta 2), pack-reused 0 (from 0) Unpacking objects: 100% (59/59), 8.22 KiB | 290.00 KiB/s, done. remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) From github.com:MYORG/MYREPO * [new tag] Releases/SOMETAG1 -> Releases/SOMETAG1 * [new tag] Releases/SOMETAG2 -> Releases/SOMETAG2 $ git fetch --depth 1 remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) $ git fetch --depth 2 remote: Enumerating objects: 13, done. remote: Counting objects: 100% (13/13), done. remote: Compressing objects: 100% (5/5), done. remote: Total 5 (delta 4), reused 1 (delta 0), pack-reused 0 (from 0) Unpacking objects: 100% (5/5), 1.36 KiB | 1.36 MiB/s, done. $ git fetch --depth 2 remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) $ git fetch --depth 2 remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0) $ git fetch --unshallow $ du ../shallow --si --max-depth=0
Full Unshallow Test:
$ rm -rf shallow-test/ $ rsync -avrt shallow/ shallow-test/ $ cd shallow-test/ $ git fetch --unshallow $ du ../shallow --si --max-depth=0
Convert Cloned Repo to Shallow
git clone --depth-1 <repo> -b <branch> git clone --depth 1 org-1234@github.com:MYORG/MYREPO.git -b MYREPO/MYBRANCH MYBRANCH git clean -x -f -d . git pull --depth 1 git fetch --depth 1 git reflog expire --expire=0 git reflog expire --expire=now --all git show-ref -s HEAD git show-ref -s HEAD > .git/shallow git prune git prune-packed git repack git pull --shallow-since=1999-01-01 git pull --shallow-since=2026-03-19 git gc git gc --prune=all git gc --prune=now git gc --aggressive git tag -d $(git tag -l) git fsck --full --no-reflogs # fix: # $ git gc # warning: reflog of 'HEAD' references pruned commits git reflog expire --all --stale-fix git gc du --si --max-depth=1
References:
- Converting git repository to shallow? - Stack Overflow - https://stackoverflow.com/questions/4698759/converting-git-repository-to-shallow
UPDATED BRUTE FORCE CONVERSION METHOD
git config remote.origin.fetch "+refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH"
rm -rf .git/objects/*
## optional cleanup: rm -f .git/info/refs
## optional cleanup: rm -f .git/{COMMIT_EDITMSG,}
# optional: rm .git/hooks/*
# optional: rm -rf .git/lfs
# optional: rm -rf .git/refs/*
# optional: rm -rf .git/logs/*
rm .git/packed-refs
git reflog expire --all --stale-fix
git fetch
git checkout MYREPO/MYBRANCH
git pull
git gc
du --si --max-depth=1
BRUTE FORCE CONVERSION METHOD
cd .git # delete top level files --- # dir: branches/ hooks/ info/ lfs/ logs/ objects/ refs/ # files: COMMIT_EDITMSG config description FETCH_HEAD HEAD index ORIG_HEAD packed-refs shallow # removes extras: COMMIT_EDITMSG, FETCH_HEAD # ignore unused: description rm -f * # remove lfs/ rm -rf lfs/ # clear logs/ and sub-dirs rm -rf logs/* # clear hooks/ rm hooks/* # clear write-projected objects/pack #rm -f objects/pack/* #rm objects/info/packs rm -rf objects/* # remove refs rm -rf refs/* rm info/refs rm COMMIT_EDITMSG rm packed-refs # will be recreated: rm FETCH_HEAD # make origin head shallow cat ORIG_HEAD > shallow # fix config: # [origin] # replace: fetch = +refs/heads/*:refs/remotes/origin/* # with: fetch = +refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH git config --get remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*" ## Limit fetch to just the one branch: git config remote.origin.fetch "+refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH" git config --get remote.origin.fetch +refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH # copy shallow copies .git/* -- config, description, FETCH_HEAD, HEAD, ORIG_HEAD, packed-refs, shallow # extras: cp ../../shallow/.git/* ./ cd .. # Fetch - should only fetch the current branch only git fetch