# exclude help system from robots User-agent: * Disallow: /manual/ Disallow: /doc/ Disallow: /gif/ Disallow: /scripts/ Disallow: /Pics/ Disallow: /School/ Disallow: /resume.html Disallow: /resume-new.html Disallow: /resume-old.html Disallow: /~greg/ # but allow htdig to index our doc-tree User-agent: susedig Disallow: # disallow stress test user-agent: stress-agent Disallow: /