# # www.marshall.tstc.edu: robots.txt # This file is used to block or allow crawlers that index our site. # It is not used for any other purpose. # User-agent: * Disallow: /images/ Disallow: /Extras/ Disallow: /test/ Disallow: /doc/ Disallow: /pdf/ Disallow: /portfolio/ Disallow: /intranet/ Disallow: /SACS/ Disallow: /OLD_SACS Disallow: /VSACS/ Disallow: /SACS2/ Disallow: /SURVEY/ Disallow: /test/ Disallow: /title3/ Disallow: /video/ Disallow: /counter/ Disallow: tstc_asp.htm Disallow: gra_procedures.shtml Disallow: /intranet/ Sitemap: http://www.marshall.tstc.edu/sitemap.xml # User-agent: Slurp Disallow: / # # The next 2 lines were added to address any bandwidth issue that # can arise from the use of the Mozilla Firefox extension Fasterfox # which pre-fetches links on a web page. # User-agent: Fasterfox Disallow: /