####### Robots.txt 

####### Disallow directory 
User-agent: * 
Disallow: /images1/ 
User-agent: * 
Disallow: /cgi-bin/
User-agent: * 
Disallow: /ccp51/
User-agent: * 
Disallow: /AT/
User-agent: * 
Disallow: /AT2/
User-agent: * 
Disallow: /COMMConfig/

###### Disallow indexing of specific file extensions 


User-agent: * 
Disallow: /*.css 

##### Disallow Collectors and Spam 
User-agent: atSpider 
Disallow: / 

User-agent: cherrypicker 
Disallow: / 

User-agent: DSurf 
Disallow: / 

User-agent: EliteSys Entry 
Disallow: / 

User-agent: EmailCollector 
Disallow: / 

User-agent: EmailSiphon 
Disallow: / 

User-agent: EmailWolf 
Disallow: / 

User-agent: Mail Sweeper 
Disallow: / 

User-agent: munky 
Disallow: / 

User-agent: Roverbot 
Disallow: / 

User-agent: WebEmailExtrac 
Disallow: /