User-agent: * Allow: /crawl-observation/public/ Disallow: /crawl-observation/restricted/ Sitemap: https://yamaarashi-crawl-observer.yamaarashix.workers.dev/sitemap.xml