# Set the delay for all crawlers to 10 seconds # User-agent: * Crawl-delay: 10 Disallow: /public/publicCourseDetailsPopup.do?method=popupDetails&courseId=* Disallow: /*.pdf # XML sitemap Sitemap: https://extension.berkeley.edu/sitemap_index.xml