Git/Shallow: Difference between revisions

From Omnia
< Git
Jump to navigation Jump to search
 
(4 intermediate revisions by the same user not shown)
Line 5: Line 5:
  git clone --depth=1 <url> creates a shallow clone.
  git clone --depth=1 <url> creates a shallow clone.


git clone --depth=1 <url> -b <branch>  # clone at branch


References:
References:
Line 38: Line 40:
NOTE: Eventually, the final --unshallow gets you everything you would have gotten, had you been able to do a full clone all at once without error. Note that you may want to use --no-single-branch during the initial shallow clone, or fix up the fetch refspec after the initial shallow clone.
NOTE: Eventually, the final --unshallow gets you everything you would have gotten, had you been able to do a full clone all at once without error. Note that you may want to use --no-single-branch during the initial shallow clone, or fix up the fetch refspec after the initial shallow clone.


=== Example ===
== Convert Cloned Repo to Shallow ==
 
<pre>
git clone --depth-1 <repo> -b <branch>
git clone --depth 1  org-1234@github.com:MYORG/MYREPO.git -b MYREPO/MYBRANCH  MYBRANCH
 
 
git clean -x -f -d .
 
git pull --depth 1
 
git fetch --depth 1
 
git reflog expire --expire=0
git reflog expire --expire=now --all
 
git show-ref -s HEAD
git show-ref -s HEAD > .git/shallow
 
git prune
git prune-packed
git repack
 
git pull --shallow-since=1999-01-01
git pull --shallow-since=2026-03-19
 
git gc
git gc --prune=all
git gc --prune=now
git gc --aggressive
 
 
git tag -d $(git tag -l)
 
git fsck --full --no-reflogs
 
 
# fix:
#  $ git gc
#  warning: reflog of 'HEAD' references pruned commits
git reflog expire --all --stale-fix
git gc
 
 
du --si --max-depth=1
</pre>
 
References:
* Converting git repository to shallow? - Stack Overflow - https://stackoverflow.com/questions/4698759/converting-git-repository-to-shallow
 
== UPDATED BRUTE FORCE CONVERSION METHOD ==
<pre>
git config remote.origin.fetch "+refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH"
rm -rf .git/objects/*
## optional cleanup: rm -f .git/info/refs
## optional cleanup: rm -f .git/{COMMIT_EDITMSG,FETCH_HEAD}
# optional:  rm .git/hooks/*
# optional:  rm -rf .git/lfs
# optional:  rm -rf .git/refs/*
# optional:  rm -rf .git/logs/*
rm .git/packed-refs
git reflog expire --all --stale-fix
git fetch
git checkout MYREPO/MYBRANCH
git pull
git gc
du --si --max-depth=1
</pre>
 
== BRUTE FORCE CONVERSION METHOD ==
<pre>
cd .git
 
# delete top level files ---
#    dir:  branches/  hooks/  info/  lfs/  logs/  objects/  refs/
#    files:  COMMIT_EDITMSG  config  description  FETCH_HEAD  HEAD  index  ORIG_HEAD  packed-refs  shallow
#  removes extras: COMMIT_EDITMSG, FETCH_HEAD
#  ignore unused: description
rm -f *
 
# remove lfs/
rm -rf lfs/
 
# clear logs/ and sub-dirs
rm -rf logs/*
 
# clear hooks/
rm hooks/*
 
# clear write-projected objects/pack
#rm -f objects/pack/*
#rm objects/info/packs
rm -rf objects/*
 
# remove refs
rm -rf refs/*
 
rm info/refs
 
rm COMMIT_EDITMSG
rm packed-refs
 
# will be recreated:
rm FETCH_HEAD
 
# make origin head shallow
cat ORIG_HEAD > shallow
 
 
# fix config:
#  [origin]
# replace:  fetch = +refs/heads/*:refs/remotes/origin/*
# with: fetch = +refs/heads/MYBRANCH:refs/remotes/origin/MYBRANCH
 
git config --get remote.origin.fetch
  +refs/heads/*:refs/remotes/origin/*
# git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
## Limit fetch to just the one branch:
git config remote.origin.fetch "+refs/heads/MYBRANCH:refs/remotes/origin/MYBRANCH"
git config --get remote.origin.fetch
  +refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH
 
# copy shallow copies .git/* -- config, description, FETCH_HEAD, HEAD, ORIG_HEAD, packed-refs, shallow
#    extras:
cp ../../shallow/.git/* ./
 
cd ..
 
# Fetch - should only fetch the current branch only
git fetch
</pre>
 
=== Unshallow Clone Example ===


Deep:
Deep:
Line 127: Line 261:
  * [new tag]                SOMETAG  -> SOMETAG
  * [new tag]                SOMETAG  -> SOMETAG


$ du ../shallow --si --max-depth=0
$ du ../shallow-test --si --max-depth=0
2.6G    ../shallow
2.6G    ../shallow-test


---
---
Line 164: Line 298:
  * [new tag]                SOMETAG  -> SOMETAG
  * [new tag]                SOMETAG  -> SOMETAG


$ du ../shallow --si --max-depth=0
$ du ../shallow-test --si --max-depth=0
2.6G    ../shallow
2.6G    ../shallow-test


---
---
Line 192: Line 326:
$ git fetch --unshallow
$ git fetch --unshallow
???
???
From github.com:MYREPO/MYORG
???  SOOO MUCH is scrolled off the screen passed my buffer
  * [new tag]                 Releases/SOMETAG1 -> Releases/SOMETAG1
???
  * [new tag]                 Releases/SOMETAG2 -> Releases/SOMETAG2
  * [new tag] ...
  * [new tag]                 SOMETAG  -> SOMETAG
  * [new tag] ...
 
  * [new tag] ...
$ du ../shallow --si --max-depth=0
??
</pre>
 
== Convert Cloned Repo to Shallow ==
 
<pre>
git clone --depth-1 <repo> -b <branch>
git clone --depth 1  org-1234@github.com:MYORG/MYREPO.git -b MYREPO/MYBRANCH  MYBRANCH
 
 
git clean -x -f -d .
 
git pull --depth 1
 
git fetch --depth 1
 
git reflog expire --expire=0
git reflog expire --expire=now --all
 
git show-ref -s HEAD
git show-ref -s HEAD > .git/shallow
 
git prune
git prune-packed
git repack
 
git pull --shallow-since=1999-01-01
git pull --shallow-since=2026-03-19
 
git gc
git gc --prune=all
git gc --prune=now
git gc --aggressive


 
$ du ../shallow-test --si --max-depth=0
git tag -d $(git tag -l)
2.6G    ../shallow-test
 
git fsck --full --no-reflogs
 
 
# fix:
#  $ git gc
#  warning: reflog of 'HEAD' references pruned commits
git reflog expire --all --stale-fix
git gc
 
 
du --si --max-depth=1
</pre>
 
References:
* Converting git repository to shallow? - Stack Overflow - https://stackoverflow.com/questions/4698759/converting-git-repository-to-shallow
 
== UPDATED BRUTE FORCE CONVERSION METHOD ==
<pre>
git config remote.origin.fetch "+refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH"
rm -rf .git/objects/*
## optional cleanup: rm -f .git/info/refs
## optional cleanup: rm -f .git/{COMMIT_EDITMSG,}
# optional:  rm .git/hooks/*
# optional:  rm -rf .git/lfs
# optional:  rm -rf .git/refs/*
# optional:  rm -rf .git/logs/*
rm .git/packed-refs
git reflog expire --all --stale-fix
git fetch
git checkout MYREPO/MYBRANCH
git pull
git gc
du --si --max-depth=1
</pre>
</pre>


== BRUTE FORCE CONVERSION METHOD ==
== keywords ==
<pre>
cd .git
 
# delete top level files ---
#    dir:  branches/  hooks/  info/  lfs/  logs/  objects/  refs/
#    files:  COMMIT_EDITMSG  config  description  FETCH_HEAD  HEAD  index  ORIG_HEAD  packed-refs  shallow
#  removes extras: COMMIT_EDITMSG, FETCH_HEAD
#  ignore unused: description
rm -f *
 
# remove lfs/
rm -rf lfs/
 
# clear logs/ and sub-dirs
rm -rf logs/*
 
# clear hooks/
rm hooks/*
 
# clear write-projected objects/pack
#rm -f objects/pack/*
#rm objects/info/packs
rm -rf objects/*
 
# remove refs
rm -rf refs/*
 
rm info/refs
 
rm COMMIT_EDITMSG
rm packed-refs
 
# will be recreated:
rm FETCH_HEAD
 
# make origin head shallow
cat ORIG_HEAD > shallow
 
 
# fix config:
#  [origin]
# replace:  fetch = +refs/heads/*:refs/remotes/origin/*
# with: fetch = +refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH
 
git config --get remote.origin.fetch
  +refs/heads/*:refs/remotes/origin/*
# git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
## Limit fetch to just the one branch:
git config remote.origin.fetch "+refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH"
git config --get remote.origin.fetch
  +refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH
 
# copy shallow copies .git/* -- config, description, FETCH_HEAD, HEAD, ORIG_HEAD, packed-refs, shallow
#    extras:
cp ../../shallow/.git/* ./
 
cd ..
 
# Fetch - should only fetch the current branch only
git fetch
</pre>

Latest revision as of 06:49, 24 March 2026

Shallow Clone

git clone --filter=blob:none <url> creates a blobless clone. ...
git clone --filter=tree:0 <url> creates a treeless clone. ...
git clone --depth=1 <url> creates a shallow clone.



git clone --depth=1 <url> -b <branch>  # clone at branch

References:

Unshallow Clone

To Unshallow a Shallow Clone 

git clone <url> --depth 1
git fetch --unshallow

vs 

git clone <url>

Incrementally: 

git clone <url> --depth 1

git fetch --deepen
git fetch --deepen
...
# or
git fetch --depth 3
git fetch --depth 6
..

git fetch --unshallow

Reference:

NOTE: Eventually, the final --unshallow gets you everything you would have gotten, had you been able to do a full clone all at once without error. Note that you may want to use --no-single-branch during the initial shallow clone, or fix up the fetch refspec after the initial shallow clone.

Convert Cloned Repo to Shallow

git clone --depth-1 <repo> -b <branch>
git clone --depth 1  org-1234@github.com:MYORG/MYREPO.git -b MYREPO/MYBRANCH  MYBRANCH


git clean -x -f -d .

git pull --depth 1

git fetch --depth 1

git reflog expire --expire=0
git reflog expire --expire=now --all

git show-ref -s HEAD
git show-ref -s HEAD > .git/shallow

git prune
git prune-packed
git repack

git pull --shallow-since=1999-01-01
git pull --shallow-since=2026-03-19

git gc
git gc --prune=all
git gc --prune=now
git gc --aggressive


git tag -d $(git tag -l)

git fsck --full --no-reflogs


# fix:
#   $ git gc
#   warning: reflog of 'HEAD' references pruned commits
git reflog expire --all --stale-fix
git gc


du --si --max-depth=1

References:

UPDATED BRUTE FORCE CONVERSION METHOD

git config remote.origin.fetch "+refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH"
rm -rf .git/objects/*
## optional cleanup: rm -f .git/info/refs
## optional cleanup: rm -f .git/{COMMIT_EDITMSG,FETCH_HEAD}
# optional:  rm .git/hooks/*
# optional:  rm -rf .git/lfs
# optional:  rm -rf .git/refs/*
# optional:  rm -rf .git/logs/*
rm .git/packed-refs
git reflog expire --all --stale-fix
git fetch
git checkout MYREPO/MYBRANCH
git pull
git gc
du --si --max-depth=1

BRUTE FORCE CONVERSION METHOD

cd .git

# delete top level files ---
#    dir:  branches/  hooks/  info/  lfs/  logs/  objects/  refs/
#    files:  COMMIT_EDITMSG  config  description  FETCH_HEAD  HEAD  index  ORIG_HEAD  packed-refs  shallow
#  removes extras: COMMIT_EDITMSG, FETCH_HEAD
#  ignore unused: description
rm -f *

# remove lfs/
rm -rf lfs/

# clear logs/ and sub-dirs
rm -rf logs/*

# clear hooks/
rm hooks/*

# clear write-projected objects/pack
#rm -f objects/pack/*
#rm objects/info/packs
rm -rf objects/*

# remove refs
rm -rf refs/*

rm info/refs

rm COMMIT_EDITMSG
rm packed-refs

# will be recreated:
rm FETCH_HEAD

# make origin head shallow
cat ORIG_HEAD > shallow


# fix config:
#  [origin]
# replace:  fetch = +refs/heads/*:refs/remotes/origin/*
# with: fetch = +refs/heads/MYBRANCH:refs/remotes/origin/MYBRANCH

git config --get remote.origin.fetch
  +refs/heads/*:refs/remotes/origin/*
# git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
## Limit fetch to just the one branch:
git config remote.origin.fetch "+refs/heads/MYBRANCH:refs/remotes/origin/MYBRANCH"
git config --get remote.origin.fetch
  +refs/heads/MYREPO/MYBRANCH:refs/remotes/origin/MYREPO/MYBRANCH

# copy shallow copies .git/* -- config, description, FETCH_HEAD, HEAD, ORIG_HEAD, packed-refs, shallow
#    extras:
cp ../../shallow/.git/* ./

cd ..

# Fetch - should only fetch the current branch only
git fetch

Unshallow Clone Example

Deep: 

$ git clone org-123@github.com:MYORG/MYLARGEREPO.git deep
Cloning into 'deep'...
remote: Enumerating objects: 2261419, done.
remote: Counting objects: 100% (5695/5695), done.
remote: Compressing objects: 100% (1506/1506), done.
remote: Total 2261419 (delta 5130), reused 4298 (delta 4183), pack-reused 2255724 (from 2)
Receiving objects: 100% (2261419/2261419), 2.07 GiB | 14.10 MiB/s, done.
Resolving deltas: 100% (1694598/1694598), done.
Updating files: 100% (26095/26095), done.

$ du deep --si --max-depth=0
3.4G    deep

Shallow: 

$ git clone org-123@github.com:MYORG/MYLARGEREPO.git shallow --depth 1
Cloning into 'shallow'...
remote: Enumerating objects: 25232, done.
remote: Counting objects: 100% (25232/25232), done.
remote: Compressing objects: 100% (16961/16961), done.
remote: Total 25232 (delta 9701), reused 17625 (delta 7165), pack-reused 0 (from 0)
Receiving objects: 100% (25232/25232), 266.90 MiB | 10.68 MiB/s, done.
Resolving deltas: 100% (9701/9701), done.
Updating files: 100% (26095/26095), done.

$ du shallow --si --max-depth=0
1.4G    shallow

Deepend and Depth and Unshallow: 

$ rsync -avrt shallow/ shallow-test/

$ cd shallow-test/

$ git fetch --deepen 1
remote: Enumerating objects: 37, done.
remote: Counting objects: 100% (37/37), done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 19 (delta 17), reused 2 (delta 0), pack-reused 0 (from 0)
Unpacking objects: 100% (19/19), 2.31 KiB | 473.00 KiB/s, done.

$ git fetch --deepen 1
remote: Enumerating objects: 173, done.
remote: Counting objects: 100% (120/120), done.
remote: Compressing objects: 100% (57/57), done.
remote: Total 59 (delta 51), reused 9 (delta 2), pack-reused 0 (from 0)
Unpacking objects: 100% (59/59), 8.22 KiB | 290.00 KiB/s, done.
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)
From github.com:MYORG/MYREPO
 * [new tag]           Releases/SOMETAG1 -> Releases/SOMETAG1
 * [new tag]           Releases/SOMETAG2 -> Releases/SOMETAG2


$ git fetch --depth 1
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)

$ git fetch --depth 2
remote: Enumerating objects: 13, done.
remote: Counting objects: 100% (13/13), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 5 (delta 4), reused 1 (delta 0), pack-reused 0 (from 0)
Unpacking objects: 100% (5/5), 1.36 KiB | 1.36 MiB/s, done.

$ git fetch --depth 2
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)

$ git fetch --depth 2
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0 (from 0)


$ git fetch --unshallow
remote: Enumerating objects: 1737369, done.
remote: Counting objects: 100% (1719287/1719287), done.
remote: Compressing objects: 100% (455766/455766), done.
remote: Total 1710144 (delta 1268632), reused 1647283 (delta 1208565), pack-reused 0 (from 0)
Receiving objects: 100% (1710144/1710144), 1010.28 MiB | 11.93 MiB/s, done.
Resolving deltas: 100% (1268632/1268632), completed with 5822 local objects.
remote: Enumerating objects: 9, done.
remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 9 (from 1)
Unpacking objects: 100% (9/9), 2.69 KiB | 2.69 MiB/s, done.
From github.com:MYREPO/MYORG
 * [new tag]                 SOMETAG   -> SOMETAG

$ du ../shallow-test --si --max-depth=0
2.6G    ../shallow-test

---

# Fix refspec to not target single branch
$ git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"

$ git fetch --unshallow
fatal: --unshallow on a complete repository does not make sense

# OOPS!  Needed to fix the refspec before running unshallow

Full Unshallow Test: 

$ rm -rf shallow-test/

$ rsync -avrt shallow/ shallow-test/

$ cd shallow-test/

$ git fetch --unshallow
remote: Enumerating objects: 1719297, done.
remote: Counting objects: 100% (1719291/1719291), done.
remote: Compressing objects: 100% (455705/455705), done.
remote: Total 1710222 (delta 1268719), reused 1647432 (delta 1208703), pack-reused 0 (from 0)
Receiving objects: 100% (1710222/1710222), 1010.08 MiB | 11.94 MiB/s, done.
Resolving deltas: 100% (1268719/1268719), completed with 5822 local objects.
remote: Enumerating objects: 9, done.
remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 9 (from 1)
Unpacking objects: 100% (9/9), 2.69 KiB | 2.69 MiB/s, done.
From github.com:MYREPO/MYORG
 * [new tag]                 Releases/SOMETAG1 -> Releases/SOMETAG1
 * [new tag]                 Releases/SOMETAG2 -> Releases/SOMETAG2
 * [new tag]                 SOMETAG   -> SOMETAG

$ du ../shallow-test --si --max-depth=0
2.6G    ../shallow-test

---

# Fix refspec to not target single branch
$ git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"

$ git fetch --unshallow
fatal: --unshallow on a complete repository does not make sense

# OOPS!  Needed to fix the refspec before running unshallow


Full No-Single-Branch Unshallow Test: 

$ rm -rf shallow-test/

$ rsync -avrt shallow/ shallow-test/

$ cd shallow-test/

# Fix refspec:
$ git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"

$ git fetch --unshallow
???
???  SOOO MUCH is scrolled off the screen passed my buffer
???
 * [new tag] ...
 * [new tag] ...
 * [new tag] ...

$ du ../shallow-test --si --max-depth=0
2.6G    ../shallow-test

keywords

Retrieved from "https://aznot.com/index.php?title=Git/Shallow&oldid=9725"