Integrate

Route crawlers through PageFlash

Choose crawler middleware or the direct API, then verify the exact bot request before launch.

Integration flow

Choose the integration path that matches your job

Use crawler middleware when search, AI, or social bots need rendered HTML automatically.

1. Prepare
Register the domain and create a scoped key.
2. Apply
Choose your environment and copy one recipe.
3. Verify
Check the crawler response before launch.

Need a domain or key first?

Step 1

Route crawler traffic through PageFlash

Choose the layer that receives your public traffic. The recipe keeps normal visitors on your SPA and only sends selected crawler user-agents to PageFlash.

Deployment environment

Choose the edge, proxy, or framework where you can safely store PAGEFLASH_KEY.

src/worker.ts - Cloudflare Worker

// Bind PAGEFLASH_KEY as a Cloudflare secret, then deploy.
const BOT_REGEX = /(googlebot|google-inspectiontool|googleother|bingbot|bingpreview|slurp|yandex|baiduspider|sogou|applebot|duckduckbot|duckassistbot|twitterbot|facebookexternalhit|facebookbot|facebot|meta-externalagent|rogerbot|linkedinbot|embedly|quora|showyoubot|outbrain|pinterestbot|pinterest|slackbot|discordbot|vkShare|W3C_Validator|whatsapp|telegrambot|zalobot|zalo\/|zaloapp|zaloweb|linespider|Chrome-Lighthouse|gptbot|chatgpt-user|oai-searchbot|perplexitybot|perplexity-user|claudebot|claude-user|anthropic-ai|ccbot|bytespider|phindbot|redditbot|petalbot|semrushbot|ahrefsbot|mj12bot|dotbot)/i

export default {
  async fetch(request, env) {
    const ua = request.headers.get('user-agent') || ''
    if (!BOT_REGEX.test(ua)) return fetch(request)

    const renderUrl = 'https://pageflash.io/render?url=' + encodeURIComponent(request.url)
    return fetch(renderUrl, {
      headers: { Authorization: `Bearer ${env.PAGEFLASH_KEY}` },
    })
  },
}

Before deploying, add the registered hostname, store the key in the server-side environment, and never expose it in browser code.

Step 2

Call the API from a backend job or tool

Use Direct API for on-demand HTML, screenshots, PDFs, or Markdown. Keep the key in your backend, job queue, or internal tool, never in client-side code.

API language

terminal - cURL

# HTML for SEO bots
curl -H "Authorization: Bearer YOUR_KEY" \
  "https://pageflash.io/render?url=https://your-app.com"

# Full-page PNG screenshot
curl -H "Authorization: Bearer YOUR_KEY" \
  "https://pageflash.io/render?url=https://your-app.com&renderType=png&fullpage=true" \
  --output snap.png

# PDF
curl -H "Authorization: Bearer YOUR_KEY" \
  "https://pageflash.io/render?url=https://your-app.com&renderType=pdf" \
  --output report.pdf

# Markdown for AI agents
curl -H "Authorization: Bearer YOUR_KEY" \
  -H "Accept: text/markdown" \
  "https://pageflash.io/render?url=https://your-app.com"

Set renderType=jpeg, renderType=png, or renderType=pdf for binary output. Add Accept: text/markdown for Markdown.

Step 3

Verify before production bots

Run the same request path your middleware will use. Do this before moving live crawler traffic.

Googlebot check

curl -A "Googlebot/2.1 (+http://www.google.com/bot.html)" \
  -H "Authorization: Bearer $PAGEFLASH_KEY" \
  "https://pageflash.io/render?url=https://example.com/pricing" | head

Done when

Response status is successful and the rendered <title>, one <h1>, canonical URL, and social tags match the public route.
Dashboard Activity attributes the request to the expected registered domain.
For link previews, repeat the same check with the relevant social bot user-agent.