Tools & IDEs Bug Report

Git commits disappeared after a detached HEAD checkout: recovering them with reflog

A hotfix made on a checked-out tag vanished when the developer switched back to main, and a reset --hard later took a day of work with it. Nothing was really gone: the reflog still pointed at every commit. Here is how we got them back and what gc changes.

Daniel Tran Daniel Tran @fun_dev · October 10, 2024 · 9 min read · 3 Views
Banner: a Git commit graph with a branch of orphaned commits being reconnected through the reflog

A developer on our payments team messaged the channel late on a Tuesday: "My hotfix commits are gone. Git log doesn't show them anywhere." Earlier that day he had checked out the v3.18.2 release tag to reproduce a production bug in currency rounding, fixed it with three commits, run the tests, and then switched back to main to open a pull request. The pull request had nothing to push. Later that afternoon, trying to "clean up" a separate feature branch, he had also run git reset --hard origin/feature/refund-export and lost a day of local commits there.

The message Git printed when he left the tag was still in his terminal scrollback:

Warning: you are leaving 3 commits behind, not connected to
any of your branches:

  9c1e4b2 Add regression test for JPY rounding
  4f07a1d Round minor units per currency exponent
  e83b5c9 Extract currency exponent lookup

If you want to keep them by creating a new branch, this may be a good time
to do so with:

 git branch <new-branch-name> 9c1e4b2

Switched to branch 'main'

Reproducing it

The sequence that loses commits is short and easy to do without noticing:

git switch --detach v3.18.2    # or: git checkout v3.18.2
# ... edit, then commit three times ...
git switch main
git log --all --oneline | grep -i rounding   # nothing

git log --all only walks commits reachable from branches, tags and other refs. Commits made on a detached HEAD belong to none of them once you leave.

What we ruled out

  • A failed or forgotten commit. His shell history showed three successful git commit calls, and the warning listed the hashes.
  • A different clone or worktree. git worktree list showed only one, and the hashes from the warning resolved in this repository with git cat-file -t 9c1e4b2, which printed commit.
  • Garbage collection. Objects are not pruned while the reflog still references them, and the default expiry for unreachable reflog entries is 30 days. These commits were hours old.

Root cause

Nothing was deleted. In detached HEAD state, HEAD points directly at a commit rather than at a branch. New commits move HEAD forward, but no branch name follows them. Switching to main moved HEAD away, leaving the three commits unreachable from any ref. The reset --hard case is the same mechanism: the branch pointer moved back to origin/feature/refund-export and the local commits were left without a name. In both cases the reflog, Git's local journal of where HEAD and each branch have pointed, still recorded the old positions.

What is happening under the hood

A branch is a file containing a commit hash. A commit stays visible in git log as long as some ref (branch, tag, remote-tracking branch, stash) can reach it. Separately, Git keeps a reflog for HEAD and for each branch, appending an entry every time they move:

git reflog -n 8
1d2e3f4 (HEAD -> main, origin/main) HEAD@{0}: checkout: moving from 9c1e4b2d6a0f8e7b5c3a1d9e8f7a6b5c4d3e2f10 to main
9c1e4b2 HEAD@{1}: commit: Add regression test for JPY rounding
4f07a1d HEAD@{2}: commit: Round minor units per currency exponent
e83b5c9 HEAD@{3}: commit: Extract currency exponent lookup
6a7b8c9 (tag: v3.18.2) HEAD@{4}: checkout: moving from main to v3.18.2
1d2e3f4 (HEAD -> main, origin/main) HEAD@{5}: pull: Fast-forward

Because reflog entries are references too, the objects they point at are protected from garbage collection until the entries expire. The defaults are:

Setting Default Meaning
gc.reflogExpire 90 days Reflog entries for commits still reachable from the ref
gc.reflogExpireUnreachable 30 days Entries for commits no longer reachable from the ref
gc.pruneExpire 2 weeks Grace period for loose unreachable objects once nothing references them

So you usually have at least a month to recover, unless someone runs an aggressive cleanup such as git reflog expire --expire=now --all followed by git gc --prune=now.

Environment

Component Version / value
Git 2.46.2 on macOS 14 (Homebrew)
Workflow Trunk-based, release tags vX.Y.Z, hotfix branches cut from tags
Repository Single clone, no worktrees, default gc settings
Commands involved git switch --detach, git switch, git reset --hard

Minimal reproduction

git init demo && cd demo
git commit --allow-empty -m "base"
git tag v1
git switch --detach v1
git commit --allow-empty -m "work on detached HEAD"
git switch -           # back to the default branch; Git prints the warning
git log --all --oneline   # only "base"
git reflog -n 3           # the detached commit is still listed

Who is affected

Anyone who checks out a tag, a specific commit, or a remote branch without creating a local branch, and then commits. IDEs make this easy because "checkout" on a tag or a commit in the history view often detaches silently. git reset --hard, an interrupted rebase, git commit --amend and git branch -D produce the same "lost" state for different reasons, and the reflog recovers all of them in the same way.

The fix

1. Find the commits. For the detached HEAD case, the HEAD reflog shows the last commit made before the checkout, here HEAD@{1}. For the reset case, the branch's own reflog is more precise:

git reflog show feature/refund-export -n 5
b44c0de (HEAD -> feature/refund-export, origin/feature/refund-export) feature/refund-export@{0}: reset: moving to origin/feature/refund-export
f19aa20 feature/refund-export@{1}: commit: Stream CSV rows instead of buffering
c2d7e31 feature/refund-export@{2}: commit: Add refund export endpoint
b44c0de (HEAD -> feature/refund-export, origin/feature/refund-export) feature/refund-export@{3}: branch: Created from origin/feature/refund-export

2. Inspect before you move anything. Confirm the commit is the one you want:

git log --oneline -5 9c1e4b2
git show --stat f19aa20

3. Give the commits a name. Creating a branch is non-destructive and is always the first move:

git branch hotfix/jpy-rounding 9c1e4b2
git branch rescue/refund-export f19aa20

4. Put the work where it belongs. For the hotfix we pushed the new branch and opened a pull request against the release line. For the feature branch, we moved the branch back to the rescued commit after checking that the remote had nothing new that we would discard:

git switch feature/refund-export
git log --oneline HEAD..rescue/refund-export     # the two lost commits
git log --oneline rescue/refund-export..HEAD     # empty: nothing would be lost
git reset --hard rescue/refund-export

If the remote had moved on, we would have rebased the rescued commits onto it instead of resetting.

Alternatives and trade-offs

  1. git cherry-pick the rescued commits onto an existing branch. Useful when the work should land on a different base, such as cherry-picking the hotfix onto main as well as the release branch.
  2. git fsck --lost-found when the reflog is gone (for example in a fresh clone or after the reflog expired but objects were not yet pruned). It lists dangling commits and writes them under .git/lost-found/. It is slower and returns more noise, so we use it only as a second resort.
  3. Restore from the server. Only works for commits that were pushed. Local-only commits exist nowhere else, which is the main argument for pushing work-in-progress branches early.

How to test the fix

We verified that the recovered branches contained exactly the expected commits and that the tests the developer had run earlier still passed:

git log --oneline v3.18.2..hotfix/jpy-rounding
# 9c1e4b2 Add regression test for JPY rounding
# 4f07a1d Round minor units per currency exponent
# e83b5c9 Extract currency exponent lookup

git switch hotfix/jpy-rounding
make test TEST=tests/currency

Confirming in production

The hotfix pull request was reviewed and merged the same evening and shipped as v3.18.3. The JPY rounding error that started this, refunds of 1 yen too much on about 0.4 percent of JPY transactions, stopped in the next day's reconciliation report. On the feature branch, git status reported "Your branch is ahead of 'origin/feature/refund-export' by 2 commits", which was the expected state before pushing.

Regression guard

  • We set git config --global advice.detachedHead true (the default) on shared developer images and asked the team not to disable it, because the message explains what to do.
  • Our hotfix runbook now starts with git switch -c hotfix/<name> v3.18.2, which creates a branch at the tag in one step and avoids detached HEAD entirely.
  • A shell prompt that shows the branch name, or HEAD detached at v3.18.2, makes the state visible at every command.

Security implications

The reflog is local and never pushed, so recovery does not expose anything to others. The reverse matters more: if a secret was committed and then "removed" with reset --hard, it still exists in the local object store and reflog until they expire and gc prunes it. Rotating the secret is the real fix; rewriting local history does not help if the commit was ever pushed.

Prevention checklist

  • Create a branch before committing on a tag or old commit: git switch -c name <tag>.
  • Read the "leaving N commits behind" warning; it contains the exact command to save the work.
  • Before git reset --hard, run git log --oneline origin/branch..HEAD to see what you would drop.
  • Push work-in-progress branches early so local accidents are not the only copy.
  • Reach for git reflog first whenever something "disappears".

References

Add a label

#git #reflog #detached-head #version-control
Written by
Daniel Tran @fun_dev

I have spent most of my career on the side of software that pages people at night: payment integrations, web platforms with real traffic, CI/CD pipelines, Kubernetes clusters and the databases underneath them. Somewhere along the way I started keeping a notebook of every bug that cost more than an hour — the exact error, what we ruled out, the real root cause and how we proved the fix. Hibug is where those notes live. Every post follows the same path: Investigate, Explain, Fix, Verify — with the environment spelled out, the fix tested, and the security angle checked.

Advertisement

Comments

0

No comments yet. Be the first to comment!

Sign In to post a comment.