How to Use Git Commit History for Accurate Sitemap Dates in Astro

Astro and other static site generators often fall short when it comes to providing reliable lastmod dates in their sitemaps. Many sites either skip the lastmod field or set it to the build time for every page, which doesn't show when content actually changed. This can erode search engine trust, since Google expects lastmod to be both consistent and verifiable. As detailed in Google Search Central’s sitemap documentation, the <lastmod> value should reflect the page’s last meaningful update, not just when the sitemap was built. Google only uses this field if it’s consistently accurate, and ignores the optional <priority> and <changefreq> fields.

Why Build Time Isn’t Enough

When every sitemap entry gets the build date, each deployment looks like a full-site update, even if only one file changed. This method doesn’t help search engines pinpoint which pages were actually updated. The right lastmod value is the date the page’s content last changed in a meaningful way, not the current date or the sitemap’s creation time. If unchanged pages keep getting new dates, crawlers may start ignoring the signal, which weakens your sitemap’s value as a change indicator.

Extracting lastmod from Git

Git tracks commits and the files affected by each one, so a build process can link a content file to the latest commit that touched it. Running a single git log command over your content directory pulls the most recent commit date for each file. This sidesteps unreliable file modification times, which don’t hold up in continuous integration setups where files are freshly checked out. The method tracks the history of the chosen file, but not every dependency that feeds into a rendered route. Changes in shared layouts or imported components might not show up unless you include them directly.

The JavaScript function below shows how to gather lastmod dates from Git for all content files in an Astro project:

import { execFileSync } from 'node:child_process';

export function gitDates(root = process.cwd(), dir = 'src') {
  const git = (...args) => execFileSync('git', args, { cwd: root, encoding: 'utf8', maxBuffer: 64 * 1024 * 1024 }).trim();

  if (git('rev-parse', '--is-shallow-repository') === 'true') {
    throw new Error('sitemap lastmod needs full git history (shallow clone detected)');
  }

  const log = git('log', '--format=%x00%cI%x00', '--name-only', '--', dir);
  const dates = new Map();
  let date;
  for (const line of log.split('\n')) {
    if (line.startsWith('\0')) date = new Date(line.slice(1, -1)).toISOString();
    else if (line && date && !dates.has(line)) dates.set(line, date);
  }
  return dates;
}

This function checks for a shallow clone and throws an error if the full history isn’t present. It parses the commit log to map each file to its latest commit date in ISO 8601 format. Shallow clones, which only have limited commit history, can block a build from finding the true latest-change date for older files. A sitemap process that relies on historical path traversal needs the right commit history in the build environment.

Integrating with Astro’s Sitemap

Astro’s @astrojs/sitemap integration lets you customize each sitemap entry with a serialize callback. By building the lastmod map once at the start, you can assign the right date to each route:

import { defineConfig } from 'astro/config';
import sitemap from '@astrojs/sitemap';
import { gitDates } from './scripts/git-lastmod.mjs';

let dates;
const sourceFor = (route) => route === '/' ? 'src/pages/index.md' : `src/pages${route.replace(/\/$/, '')}.md`;

export default defineConfig({
  site: 'https://example.com',
  trailingSlash: 'never',
  build: { format: 'file' },
  integrations: [
    {
      name: 'git-lastmod',
      hooks: {
        'astro:build:start': () => {
          dates = gitDates();
        },
      },
    },
    sitemap({
      serialize(item) {
        const route = new URL(item.url).pathname;
        const lastmod = dates.get(sourceFor(route));
        if (!lastmod) throw new Error(`No committed date for ${route}`);
        return {
          ...item,
          lastmod
        };
      },
    }),
  ],
});

This setup makes sure each sitemap entry shows the actual last commit date of its source file. Astro’s sitemap integration supports per-URL metadata, so you don’t have to use a single timestamp for the whole sitemap. You might need to adjust the sourceFor function if your routes are generated differently, such as from frontmatter fields in MDX files. Another option is to use explicit updatedDate frontmatter, which gives a clear editorial record and can better reflect updates involving more than one source file.

Understanding the Vercel Shallow Clone Trap

Vercel’s default build process clones repositories with --depth=10, so only the last ten commits are present. When Git hits the oldest commit in this shallow clone, it can’t compare it to a parent and lists every file as changed in that commit. This means most files in your content directory could get the same lastmod date, which misleads search engines. The gitDates() function checks for a shallow repository and fails the build if full history is missing, stopping the publication of a sitemap with wrong dates. This matches Google’s advice that it’s better to skip lastmod than to publish unverifiable dates.

How to Enable Full Git History on Vercel

Set the environment variable VERCEL_DEEP_CLONE=true in your Vercel project to make Vercel clone the full repository history. This variable isn’t in Vercel’s official documentation, but it comes up in community guides and discussions. Deep cloning can fail for very large repositories, so always check your build logs after turning it on. If your repository is private, trying to fetch the full history with git fetch --unshallow during the build will likely fail due to missing credentials. The environment variable is the preferred fix, but keep the shallow check in place to catch any future changes in Vercel’s process. For more on Vercel’s cloning, see the Vercel documentation on clone depth.

Choosing Which Files Define Page Content

For blog posts, the main MDX file usually sets the lastmod date. For index pages or pages that pull in data, you might need to include other source files, like JSON datasets. The production script can build a set of dependencies for each route and use the newest commit date among them. This way, when a post is deleted, the index page’s lastmod reflects the deletion commit, giving search engines an accurate signal. File-based Git dates work well for content files but can miss changes in shared layouts, components, data loaders, or other imported dependencies. A manually maintained update field is another way to set dates, making the editorial meaning clear and avoiding unrelated implementation changes being counted as content updates.

Key Technical Considerations

• Committer date vs. author date: The script uses the committer date (%cI), which updates on rebases and amends. If you want dates that survive rebases, use the author date (%aI).
• Merge commits: Standard git log doesn’t list files for merge commits. If you need the date a change went live, use squash merges or add --first-parent to your log command.
• Uncommitted changes: Only committed changes are included. Edits in your working directory don’t update lastmod until committed.
• No Git, no build: If the build runs without a .git directory, the script fails instead of falling back to unreliable file modification times.
• Performance: On a repository with over a thousand commits, the git log call finishes in under 0.2 seconds.

Testing and Validation

To check accuracy, you can write automated tests that create a fixture repository with known commit dates and verify that the sitemap lastmod values match what you expect. A post-build check can also parse the generated sitemap and compare each entry’s lastmod to the expected value, failing the build if there’s a mismatch.

Related Git Workflow Guidance

For more on managing commit history and how different Git strategies affect your repository, see this earlier breakdown.

Summary of Best Practices

• Use git log --name-only to get lastmod dates for sitemap entries, making sure each one reflects the actual last meaningful content change.
• Fail the build if a shallow clone is detected to avoid misleading dates, since shallow clones only have limited commit history and can cause inaccurate lastmod values.
• Set VERCEL_DEEP_CLONE=true on Vercel to make sure full history is available for accurate date extraction.
• Decide which files represent the main content for each route and skip layout-only changes in lastmod calculations, or use explicit editorial metadata when needed.

Following these steps and staying in line with Google’s official sitemap guidance lets you give search engines accurate update information for your Astro static site. This improves the reliability of your sitemap and helps with better indexing of your content.

For more technical details and implementation examples, see the original guide by Seppe Gadeyne: How to Use Git Commit History for Accurate Sitemap Dates in Astro.