---
title: "PageFlash Docs - Route crawlers through PageFlash"
description: "Configure crawler middleware or the direct PageFlash API, then verify the exact crawler response before launch."
site_name: "PageFlash"
lang: "en"
url: "https://pageflash.io/docs/integrations"
og:
  image: "https://pageflash.io/og/pageflash-og.png"
  type: "website"
twitter:
  card: "summary_large_image"
robots: "index, follow, max-image-preview:large, max-snippet:-1, max-video-preview:-1"
---

**Browse docs 4 task-based guides**

Integrate

# Route crawlers through PageFlash

Choose crawler middleware or the direct API, then verify the exact bot request before launch.

Integration flow

## Choose the integration path that matches your job

Crawler middleware Send selected crawler requests to PageFlash. Normal visitors stay on your SPA. Direct API Request HTML, screenshots, PDFs, or Markdown from a backend job or internal tool.

Use crawler middleware when search, AI, or social bots need rendered HTML automatically.

1.  1\. Prepare
    Register the domain and create a scoped key.
2.  2\. Apply
    Choose your environment and copy one recipe.
3.  3\. Verify
    Check the crawler response before launch.

[Need a domain or key first?](https://pageflash.io/docs/getting-started)

Step 1

## Route crawler traffic through PageFlash

Choose the layer that receives your public traffic. The recipe keeps normal visitors on your SPA and sends selected crawler user-agents to PageFlash, plus AI agents that ask for Markdown with `Accept: text/markdown` or `?format=md`.

Deployment environment

Choose the edge, proxy, or framework where you can safely store `PAGEFLASH_KEY`.

src/worker.ts - Cloudflare Worker

```
// Bind PAGEFLASH_KEY as a Cloudflare secret, then deploy.
const BOT_REGEX = /(googlebot|google-inspectiontool|googleother|bingbot|bingpreview|slurp|yandex|baiduspider|sogou|applebot|duckduckbot|duckassistbot|twitterbot|facebookexternalhit|facebookbot|facebot|meta-externalagent|rogerbot|linkedinbot|embedly|quora|showyoubot|outbrain|pinterestbot|pinterest|slackbot|discordbot|vkShare|W3C_Validator|whatsapp|telegrambot|zalobot|zalo\/|zaloapp|zaloweb|linespider|Chrome-Lighthouse|gptbot|chatgpt-user|oai-searchbot|oai-adsbot|perplexitybot|perplexity-user|claudebot|claude-user|claude-searchbot|anthropic-ai|ccbot|bytespider|phindbot|redditbot|petalbot|semrushbot|ahrefsbot|mj12bot|dotbot)/i

export default {
  async fetch(request, env) {
    const ua = request.headers.get('user-agent') || ''
    const accept = request.headers.get('accept') || ''
    // AI agents ask for Markdown with Accept: text/markdown or ?format=md.
    const wantsMarkdown = accept.includes('text/markdown') || new URL(request.url).searchParams.get('format') === 'md'
    if (!BOT_REGEX.test(ua) && !wantsMarkdown) return fetch(request)

    const renderUrl = 'https://pageflash.io/render?url=' + encodeURIComponent(request.url)
    return fetch(renderUrl, {
      headers: { Authorization: `Bearer ${env.PAGEFLASH_KEY}`, Accept: accept },
    })
  },
}
```

Before deploying, add the registered hostname, store the key in the server-side environment, and never expose it in browser code.

Step 2

## Call the API from a backend job or tool

Use Direct API for on-demand HTML, screenshots, PDFs, or Markdown. Keep the key in your backend, job queue, or internal tool, never in client-side code.

API language

terminal - cURL

```
# HTML for SEO bots
curl -H "Authorization: Bearer YOUR_KEY" \
  "https://pageflash.io/render?url=https://your-app.com"

# Full-page PNG screenshot
curl -H "Authorization: Bearer YOUR_KEY" \
  "https://pageflash.io/render?url=https://your-app.com&renderType=png&fullpage=true" \
  --output snap.png

# PDF
curl -H "Authorization: Bearer YOUR_KEY" \
  "https://pageflash.io/render?url=https://your-app.com&renderType=pdf" \
  --output report.pdf

# Markdown for AI agents
curl -H "Authorization: Bearer YOUR_KEY" \
  -H "Accept: text/markdown" \
  "https://pageflash.io/render?url=https://your-app.com"
```

Set `renderType=jpeg`, `renderType=png`, or `renderType=pdf` for binary output. Add `Accept: text/markdown` or `format=md` for Markdown.

Step 3

## Verify before production bots

Run the same request path your middleware will use. Do this before moving live crawler traffic.

Googlebot check

```
curl -A "Googlebot/2.1 (+http://www.google.com/bot.html)" \
  -H "Authorization: Bearer $PAGEFLASH_KEY" \
  "https://pageflash.io/render?url=https://example.com/pricing" | head
```

Done when

-   Response status is successful and the rendered `<title>`, one `<h1>`, canonical URL, and social tags match the public route.
-   Dashboard Activity attributes the request to the expected registered domain.
-   For link previews, repeat the same check with the relevant social bot user-agent.

Stuck or have a question? Email [hello@pageflash.io](mailto:hello@pageflash.io).

## Structured data (JSON-LD)

```json
[{"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://pageflash.io/#organization","name":"PageFlash","url":"https://pageflash.io/","logo":{"@type":"ImageObject","url":"https://pageflash.io/android-chrome-512x512.png","width":512,"height":512}},{"@type":"WebSite","@id":"https://pageflash.io/#website","name":"PageFlash","url":"https://pageflash.io/","description":"Verify crawler HTML for registered public SPA routes before routing production crawler traffic.","publisher":{"@id":"https://pageflash.io/#organization"}},{"@type":"SoftwareApplication","@id":"https://pageflash.io/#software","name":"PageFlash","applicationCategory":"DeveloperApplication","operatingSystem":"Web","url":"https://pageflash.io/","description":"Render registered public SPA routes for the crawler user-agents your integration selects, then verify the HTML before production routing.","publisher":{"@id":"https://pageflash.io/#organization"},"offers":[{"@type":"Offer","name":"Free","price":"0","priceCurrency":"USD","url":"https://pageflash.io/pricing"},{"@type":"Offer","name":"Starter","price":"19","priceCurrency":"USD","url":"https://pageflash.io/pricing"},{"@type":"Offer","name":"Growth","price":"49","priceCurrency":"USD","url":"https://pageflash.io/pricing"},{"@type":"Offer","name":"Pro","price":"129","priceCurrency":"USD","url":"https://pageflash.io/pricing"},{"@type":"Offer","name":"Enterprise","price":"499","priceCurrency":"USD","url":"https://pageflash.io/pricing"}]}]},{"@context":"https://schema.org","@graph":[{"@type":"WebPage","@id":"https://pageflash.io/docs/integrations#webpage","url":"https://pageflash.io/docs/integrations","name":"PageFlash Docs - Route crawlers through PageFlash","isPartOf":{"@id":"https://pageflash.io/#website"},"about":{"@id":"https://pageflash.io/#software"}},{"@type":"BreadcrumbList","@id":"https://pageflash.io/docs/integrations#breadcrumbs","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://pageflash.io/"},{"@type":"ListItem","position":2,"name":"Docs","item":"https://pageflash.io/docs"},{"@type":"ListItem","position":3,"name":"Integrations","item":"https://pageflash.io/docs/integrations"}]}]}]
```