GAP·MAP

SEO & metadata

[ junior depth ]

Search engines need a fetchable page

Discovery, crawling, and indexing are separate events. A search engine can discover a URL from a link or sitemap. Its crawler then requests the URL. Indexing is the later decision to process the page for search results.

Use real links for navigation that you want Google to discover:

<a href="/products/green-shoe">Green shoe</a>

Google documents <a> elements with href as the reliable crawlable form. A div with a click handler may navigate for a browser user, but Google does not reliably extract it as a link.

Wrong "If a browser can reach a route after a click, a crawler can discover it."

Right Important routes need URLs and crawlable anchors. A sitemap can supplement those links, but it does not repair navigation that hides the site graph.

Title and meta description

Every important page should have a concise, descriptive <title> and a page-specific description:

<head>
  <title>Green Trail Shoe | Northstar</title>
  <meta
    name="description"
    content="Water-resistant trail shoe with a grippy sole. Available in sizes 38 to 47."
  >
</head>

The title names the document and is one source Google can use for a search result's title link. Google may also use the visible page title, headings, prominent text, og:title, and link text. It can therefore show a different title link when those signals disagree.

Google primarily creates snippets from page content. It sometimes uses the meta description when that text describes the page better. There is no documented character limit for the element, and search results truncate snippets to fit the device.

Wrong "The meta description is the exact text Google must display."

Right It is a candidate summary. Keep it accurate and unique because the search engine can select another passage for a particular query.

Open Graph and Twitter cards

Search metadata and social-preview metadata have different consumers. The Open Graph protocol requires og:title, og:type, og:image, and og:url for a graph object. It recommends a description. Twitter card markup uses twitter:* names. Current Next.js documentation emits fields such as twitter:card, twitter:title, twitter:description, and twitter:image.

<meta property="og:title" content="Green Trail Shoe">
<meta property="og:type" content="website">
<meta property="og:url" content="https://shop.example/products/green-shoe">
<meta property="og:image" content="https://shop.example/images/green-shoe-card.jpg">
<meta property="og:description" content="A water-resistant trail shoe.">

<meta name="twitter:card" content="summary_large_image">
<meta name="twitter:title" content="Green Trail Shoe">
<meta name="twitter:description" content="A water-resistant trail shoe.">
<meta name="twitter:image" content="https://shop.example/images/green-shoe-card.jpg">

These tags describe a link preview. They do not make a page indexable and do not replace the HTML title or visible page content. Keep the URL, title, description, and image specific to the current route. A site-wide default that describes the home page gives product shares the wrong preview.

[ middle depth ]

Canonical URLs consolidate duplicates

Tracking parameters, filters, and campaign paths can expose the same content at several URLs. Use rel="canonical" to identify the representative HTML URL:

<link rel="canonical" href="https://shop.example/products/green-shoe">

Google recommends absolute URLs and a self-referential canonical on the preferred page. Point duplicate pages at the same URL, link internally to that URL, and list it in the sitemap. These consistent signals make the preference clear.

Wrong "A canonical tag redirects the crawler and guarantees which URL is indexed."

Right The browser stays on the requested page. Google treats the annotation as a canonicalization signal and ultimately chooses the canonical. Use a permanent redirect when a duplicate URL is being retired and users should move too.

Do not solve an ordinary duplicate by adding noindex to the preferred page. Google explicitly recommends canonical annotations instead of noindex for choosing among duplicate pages within a site.

Robots rules answer different questions

robots.txt controls which paths a cooperating crawler may request. It is not access control. Google may still index a disallowed URL when links reveal it, although it cannot crawl the blocked content.

User-agent: *
Disallow: /internal-search/

Sitemap: https://shop.example/sitemap.xml

A robots meta tag controls indexing and serving behavior for an HTML page the crawler can fetch:

<meta name="robots" content="noindex">

For a non-HTML resource such as a PDF, Google supports the equivalent response header:

X-Robots-Tag: noindex

Wrong "Disallow the page, then Google will crawl it and read its noindex tag."

Right Crawl blocking prevents that fetch. If a public page must be excluded from Google Search with noindex, Googlebot needs access to read the directive. Private data needs authentication because robots directives depend on crawler cooperation.

What a sitemap says

An XML sitemap tells Google about URLs you want in search results. Include canonical URLs rather than every parameter variant. Google treats sitemap entries as canonical suggestions and still determines which pages are duplicates.

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://shop.example/products/green-shoe</loc>
  </url>
</urlset>

A sitemap helps discovery, especially when pages are new or updated. It does not guarantee crawling, indexing, or ranking.

JSON-LD structured data

Structured data supplies explicit, machine-readable facts about the page. Google recommends JSON-LD when the site's setup supports it. Choose a structured-data feature that matches the page, include its required properties, and keep the values consistent with visible content.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "Product",
  "name": "Green Trail Shoe",
  "image": "https://shop.example/images/green-shoe.jpg",
  "description": "Water-resistant trail shoe with a grippy sole."
}
</script>

The exact required and recommended properties depend on Google's documentation for the selected search feature. Schema.org contains a broader vocabulary, while Google Search Central defines Google-specific eligibility.

Wrong "Valid JSON-LD guarantees a rich result and can describe details absent from the page."

Right Correct markup makes a page eligible. Google does not guarantee the enhanced display, and its guidelines reject structured data that is misleading or not represented in visible page content.

[ senior depth ]

Audit the response before the rendered DOM

A client-rendered route can look complete in a browser while its HTTP response contains only an app shell:

<div id="app"></div>
<script src="/assets/app.js"></script>

Google describes three phases for JavaScript applications: crawling, rendering, and indexing. A 200 response is queued for rendering unless a robots directive prevents indexing. When resources permit, Google runs the page in headless Chromium, parses rendered links, and uses the rendered HTML for indexing. The rendering queue can take longer than a few seconds, and Google notes that some bots cannot run JavaScript.

The safer delivery strategy is to put the route's meaningful content and metadata in the initial HTML through server rendering or prerendering. Client code can hydrate that document for interaction.

<head>
  <title>Green Trail Shoe | Northstar</title>
  <meta name="description" content="Water-resistant trail shoe with a grippy sole.">
  <link rel="canonical" href="https://shop.example/products/green-shoe">
  <meta property="og:title" content="Green Trail Shoe">
  <meta property="og:type" content="website">
  <meta property="og:url" content="https://shop.example/products/green-shoe">
  <meta property="og:image" content="https://shop.example/images/green-shoe-card.jpg">
</head>
<body>
  <main>
    <h1>Green Trail Shoe</h1>
    <p>Water-resistant trail shoe with a grippy sole.</p>
  </main>
  <script src="/assets/app.js"></script>
</body>

Wrong "Google runs JavaScript, so server rendering has no SEO value."

Right Google can index rendered JavaScript, but rendering is a separate phase and other bots may not execute it. Google says server-side rendering or prerendering remains useful for users and crawlers.

Metadata must agree across rendering phases

Google can observe a title, description, canonical, robots tag, and JSON-LD added through JavaScript. That capability does not make every transition safe. If the original HTML contains noindex, Google may skip rendering, so client code that removes noindex may never run from the indexer's point of view.

Do not ship an indexable route with a placeholder directive that JavaScript must repair:

<!-- Broken for a route intended to be indexed -->
<meta name="robots" content="noindex">
<script>
  loadProduct().then(() => {
    document.querySelector('meta[name="robots"]').remove();
  });
</script>

For canonicals, Google recommends putting the value in source HTML and preventing JavaScript from changing it. If source HTML cannot contain one, leave it out there and add one consistent canonical during rendering. Multiple or conflicting canonical elements can produce unexpected results.

Status codes are part of the document contract

An SPA fallback often returns 200 for a missing product and paints "Not found" later. Googlebot uses HTTP status codes to understand failures. Return 404 for a missing route, 401 for content behind login, and an appropriate redirect status when a resource has moved. Do not make crawlers infer the resource state from client-rendered text.

HTTP/1.1 404 Not Found
Content-Type: text/html; charset=utf-8

The same separation applies to discovery. Render product navigation as anchors with href, including when JavaScript creates them. Script-only click targets are not a reliable crawl graph.

Verify each layer after deployment

For representative pages and failure routes, inspect all of these outputs:

  • HTTP status and headers
  • raw response HTML
  • rendered DOM after JavaScript
  • canonical, robots, title, description, Open Graph, and Twitter card fields
  • JSON-LD against visible content and the selected Google feature

Use Google's Rich Results Test during development. After deployment, URL Inspection shows how Google sees a URL, and rich-result status reports can expose templating or serving regressions. Valid structured data establishes eligibility only. Search presentation still depends on Google's systems, and a social preview needs separate verification by the consuming platform.

dig deeper

Primary sources behind these notes - the specs and official docs worth reading in full.

beta

The interviewer part is in the works.

The diagnostic, personal maps, and AI mock interviews are being finished right now. The notes stay free either way. Leave an email and you'll get the first-cohort invite, plus a month of Pro when it opens.

builds on

more HTML & A11y

was this useful?