# SEO Basics in TanStack Start: Meta Tags, robots.txt and a Dynamic Sitemap

%[https://www.youtube.com/watch?v=KoeZJ5NakEo&t=1s]

## What is SEO?

SEO stands for Search Engine Optimization. It is the practice of improving a website so it ranks higher in unpaid ("organic") search results on engines like Google.

When people search for a product, service or answer online, SEO helps search engines understand what your website is about and recommend it to the right visitors, without you paying for ads.

In this article we will add three basics to a TanStack Start demo store: page titles and descriptions, a `robots.txt` file and a sitemap built from the products in Directus.

## Add basic SEO to the About page

Open the About Us page and add a `head` section to the route:

```tsx
export const Route = createFileRoute('/_appLayout/about')({
  head: () => ({
    meta: [
      { title: 'Demo Store | About' },
      {
        name: 'description',
        content: 'This is the about us page',
      },
    ],
  }),
  component: RouteComponent,
})
```

Let's break it down:

- `head: () => ({ ... })` is a function that takes no arguments and returns an object. The extra parentheses around the braces tell JavaScript that `{ ... }` is an object, not the body of the function. TanStack Router runs this function when the route loads and uses the result to build the page's `<head>`.
- `meta: [ ... ]` is an array. Each item in it becomes one tag inside the `<head>`.
- `{ title: 'Demo Store | About' }` sets the `<title>` tag. The text appears in the browser tab, in bookmarks and as the clickable headline in Google search results.
- `{ name: 'description', content: '...' }` creates a `<meta name="description">` tag. Google often shows this text under the headline to tell people what the page is about, although it may write its own snippet instead.

The route above produces this HTML in the page's `<head>`:

```html
<head>
  <title>Demo Store | About</title>
  <meta name="description" content="This is the about us page" />
</head>
```

## Fix the tab label

Before you save the change, the browser tab says "TanStack Start Starter". In the starter template, that title comes from the root route. As soon as you save the About route, the tab changes to "Demo Store | About", because a child route's title replaces the root's.

It works, but typing the same pieces on every route gets repetitive and easy to get wrong. Can we improve on this? Yes.

## Build an SEO helper function

Create a file at `src/lib/seo.ts` and paste in this helper:

```ts
const SITE_NAME = 'Demo Store'
const SITE_URL = 'https://your-store-domain.com'

export function seo({
  title,
  description,
  image,
  path,
}: {
  title?: string
  description?: string
  image?: string | null
  path?: string
}) {
  const fullTitle = title ? `${title} | ${SITE_NAME}` : SITE_NAME

  return [
    { title: fullTitle },
    { name: 'description', content: description ?? '' },
    { property: 'og:title', content: fullTitle },
    { property: 'og:description', content: description ?? '' },
    ...(image ? [{ property: 'og:image', content: image }] : []),
    ...(path ? [{ property: 'og:url', content: `${SITE_URL}${path}` }] : []),
  ]
}
```

This helper is a reusable template. Instead of typing the title, description and Open Graph tags on every route, you pass in a few details and `seo()` builds the whole `meta` array for you.

What it does:

- **Formats titles consistently.** Pass `'About Us'` and you get `About Us | Demo Store`. Leave the title out and you get just `Demo Store`.
- **Handles the standard tags.** It sets the `<title>` and the description, so search engines know how to describe the page.
- **Prepares social previews.** The `og:` tags are Open Graph tags. When someone shares your link on Facebook, WhatsApp or LinkedIn, they produce a clean preview card.
- **Only adds what you give it.** The image and URL tags appear only when you pass an `image` or a `path`, so you never get a broken, empty tag.

Two details to notice. `path` should start with a slash, such as `/about`, and the helper adds `SITE_URL` in front, because Open Graph wants a full address. The `image` should also be a full address.

Now use it on the About page:

```tsx
import { seo } from '@/lib/seo'

export const Route = createFileRoute('/_appLayout/about')({
  head: () => ({
    meta: seo({
      title: 'About Us',
      description: 'This is the about us page',
      path: '/about',
    }),
  }),
  component: RouteComponent,
})
```

## Add a robots.txt file

A `robots.txt` file is a set of instructions for search engine bots. It tells them which pages or sections of your site they may crawl and which to skip. It lives at the root of your domain, for example `https://mystore.com/robots.txt`, and it is the first file crawlers look for before scanning your site.

Why you want one:

- **Keep bots out of private or irrelevant pages.** In a store you do not want crawlers wasting time on cart pages, checkout screens or account dashboards.
- **Save crawl budget.** Search engines spend a limited amount of time on your site. Skipping unimportant URLs, such as internal search results, keeps the focus on your products. (On a small site this matters little, but it is a good habit.)
- **Reduce duplicate URLs.** Stores often generate many addresses for one product, such as `?sort=price-asc`. Blocking those parameters helps, and a canonical tag on the page is the other tool for this job.
- **Point bots to your sitemap.** You can put the address of your `sitemap.xml` in `robots.txt` so crawlers find every page you want ranked.

One important limitation: `robots.txt` is not a security lock. Legitimate search engines follow it, but malicious bots can ignore it. If another site links to a blocked URL, Google can still list that URL without a description. Protect sensitive data with proper authentication.

Create `public/robots.txt` and paste in:

```txt
User-agent: *
Allow: /

Sitemap: https://your-store-domain.com/sitemap.xml
```

- `User-agent: *` says which crawler the rules apply to. Googlebot and Bingbot are examples of user agents. The asterisk is a wildcard that means all bots.
- `Allow: /` gives permission to crawl the whole site. The slash is the root of your site and everything beneath it. This is also the default, but writing it out makes the file easy to read.
- `Sitemap:` tells crawlers where your sitemap lives. It must be a full address. We build that sitemap next.

If your store has pages you want hidden from search, add a `Disallow` line for each one. Only do this for routes that exist in your app:

```txt
User-agent: *
Allow: /
Disallow: /cart
Disallow: /checkout

Sitemap: https://your-store-domain.com/sitemap.xml
```

To test, visit `http://localhost:3000/robots.txt`. You should see the file's text.

## Add a sitemap

A sitemap is a list of every page you want search engines to find. Ours will be built from the products in Directus, so new products appear automatically.

Create `src/routes/sitemap[.]xml.ts` and paste in this code:

```ts
// src/routes/sitemap[.]xml.ts
import { createFileRoute } from '@tanstack/react-router'
import { readItems } from '@directus/sdk'
import { directus } from '@/lib/directus'

const SITE_URL = 'https://your-store-domain.com'

export const Route = createFileRoute('/sitemap.xml')({
  server: {
    handlers: {
      GET: async () => {
        // 1. Get every product id from Directus
        const products = await directus.request(
          readItems('products', { fields: ['id'], limit: -1 }),
        )

        // 2. Build one <url> block per page
        const pages = ['/', '/products', ...products.map((p) => `/products/${p.id}`)]
        const urls = pages
          .map((path) => `<url><loc>${SITE_URL}${path}</loc></url>`)
          .join('')

        // 3. Wrap them in the sitemap XML
        const xml = `<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">${urls}</urlset>`

        // 4. Return it as XML, not as a web page
        return new Response(xml, {
          headers: { 'Content-Type': 'application/xml' },
        })
      },
    },
  },
})
```

Let's go through it piece by piece.

**The file name.** `sitemap[.]xml.ts` is a file-based route. TanStack Router turns file names into URLs. A plain dot would be read as a path separator, so the brackets tell it to keep the dot, and the URL becomes `/sitemap.xml`.

**The imports.** `createFileRoute` registers this file as a route. `readItems` is a Directus SDK helper that builds a "read these items" request. It doesn't send anything by itself. `directus` is the client you already set up, which knows your Directus address and token.

**SITE_URL.** Sitemaps need full addresses, such as `https://your-store-domain.com/products/1`, not just `/products/1`. This constant is the start of every address. Leave off the trailing slash, because the paths below begin with one.

**createFileRoute('/sitemap.xml').** This gives the route its URL, and the string must match the file name. TypeScript checks it against the generated route tree.

**server: { handlers: { GET } }.** Normal routes return a React page. A server route returns data, so there is no component. The `server` block runs only on the server, never in the browser. `GET` answers GET requests, which is what Google and your browser send. It is `async` because we wait for Directus.

**The Directus call.**

- `directus.request(...)` sends the request.
- `readItems('products', ...)` asks for items from the `products` collection. The name must match your schema type, including the lowercase.
- `fields: ['id']` asks for only the id, which keeps the response small.
- `limit: -1` means all items. Directus returns only 100 by default, so without it you would miss products.

**Building the page list.** `products.map(...)` turns each product into a path such as `/products/8ffebf9d-...`. The `...` spreads that list into the array after `'/'` and `'/products'`, so `pages` ends up as one flat list of every path.

**Building the URL blocks.** The second `map` wraps each path in `<url><loc>https://.../path</loc></url>`, which is the format Google reads. `.join('')` glues the blocks into one string.

**The XML wrapper.** The first line declares the file as XML. `<urlset xmlns="...">` is the required outer tag, and the address inside it is part of the sitemap standard, so don't change it. Our `urls` text goes inside it.

**The Response.** This is the standard web `Response`, the same type `fetch` returns. The `Content-Type: application/xml` header tells the browser and Google to read the text as XML. Without it, a string response is sent as plain text.

Test it in your browser at `http://localhost:3000/sitemap.xml`. Your TanStack Start app may run on a different port, so use your own.

Your sitemap should look something like this:

```xml
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://your-store-domain.com/</loc>
  </url>
  <url>
    <loc>https://your-store-domain.com/products</loc>
  </url>
  <url>
    <loc>https://your-store-domain.com/products/8ffebf9d-cef4-4685-89fa-3240ee899f1d</loc>
  </url>
  <!-- one <url> for each product -->
</urlset>
```

Because `SITE_URL` is a fixed value, the addresses start with your store's domain, not localhost. While testing, you can temporarily set `SITE_URL` to `http://localhost:3000` if you want to click through the links.

## Recap

- A route's `head` function sets the page title and description.
- A small `seo()` helper keeps titles consistent and adds Open Graph tags for social previews.
- `public/robots.txt` tells crawlers what they may visit and where your sitemap is.
- A server route at `/sitemap.xml` builds the sitemap from Directus, so it never goes out of date.

This is the minimum version. Later we can add `lastmod` dates, XML escaping and caching to the sitemap, and give each product page its own title and description. Once your store is live, submit the sitemap address in Google Search Console.
