Возраст домена | 26 лет |
Дата окончания | Истек срок регистрации |
PR | 4 |
ИКС | |
Страниц в Google | 526 |
Страниц в Яндексе | 67 |
Dmoz | Нет |
Яндекс Каталог | Нет |
Alexa Traffic Rank | 4799644 |
Alexa Country | Нет данных |
История изменения показателей | Авторизация |
The Association of Delaware Valley Independent Schools - Pennsylvania, New Jersey, and Delaware
n/a
n/a
UTF-8
174.89 КБ
460
3 727 симв.
3 154 симв.
Данные предоставлены сервисом semrush
Сайт | Общие фразы | PR | тИЦ | Alexa Rank | Alexa Country | |
---|---|---|---|---|---|---|
![]() ![]() |
64 | 6 |
10 | 143596 | ![]() | |
![]() ![]() |
63 | 9 |
0 | 5 | ![]() | |
![]() ![]() |
59 | 6 |
0 | 11209 | ![]() | |
![]() |
53 | 4 |
0 | 3417272 | ![]() |
|
![]() ![]() |
48 | 6 |
0 | 122349 | ![]() | |
![]() ![]() |
45 | 7 |
0 | 7787 | ![]() | |
![]() |
44 | 7 |
0 | 11362 | ![]() | |
![]() ![]() |
34 | 8 |
0 | 138 | ![]() | |
![]() ![]() |
29 | 5 |
0 | 1511456 | ![]() | |
![]() ![]() |
28 | 5 |
0 | 1666507 | ![]() | |
Еще 40 сайтов после авторизации |
Данные предоставлены сервисом semrush
Счетчик | Посетители за 24 часа | Просмотры | Просмотров на посетителя |
---|---|---|---|
![]() | Нет доступа | Нет доступа | n/a |
Domain Name: ADVIS.ORG
Registry Domain ID: D785970-LROR
Registrar WHOIS Server: whois.networksolutions.com
Registrar URL: http://www.networksolutions.com
Updated Date: 2018-03-07T23:44:31Z
Creation Date: 1998-01-22T05:00:00Z
Registry Expiry Date: 2020-01-21T05:00:00Z
Registrar Registration Expiration Date:
Registrar: Network Solutions, LLC
Registrar IANA ID: 2
Registrar Abuse Contact Email: abuse@web.com
Registrar Abuse Contact Phone: +1.8003337680
Reseller:
Domain Status: clientTransferProhibited https://icann.org/epp#clientTransferProhibited
Registrant Organization:
Registrant State/Province: PA
Registrant Country: US
Name Server: NS-1381.AWSDNS-44.ORG
Name Server: NS-1642.AWSDNS-13.CO.UK
Name Server: NS-737.AWSDNS-28.NET
Name Server: NS-433.AWSDNS-54.COM
URL of the ICANN Whois Inaccuracy Complaint Form https://www.icann.org/wicf/)
>>> Last update of WHOIS database: 2018-10-04T19:03:15Z <<<
For more information on Whois status codes, please visit https://icann.org/epp
Access to Public Interest Registry WHOIS information is provided to assist persons in determining the contents of a domain name registration record in the Public Interest Registry registry database. The data in this record is provided by Public Interest Registry for informational purposes only, and Public Interest Registry does not guarantee its accuracy. This service is intended only for query-based access. You agree that you will use this data only for lawful purposes and that, under no circumstances will you use this data to (a) allow, enable, or otherwise support the transmission by e-mail, telephone, or facsimile of mass unsolicited, commercial advertising or solicitations to entities other than the data recipient's own existing customers; or (b) enable high volume, automated, electronic processes that send queries or data to the systems of Registry Operator, a Registrar, or Afilias except as reasonably necessary to register domain names or modify existing registrations. All rights reserved. Public Interest Registry reserves the right to modify these terms at any time. By submitting this query, you agree to abide by this policy.
The Registrar of Record identified in this output may have an RDDS service that can be queried for additional information on how to contact the Registrant, Admin, or Tech contact of the queried domain name.
User-agent: Googlebot
Allow: /
User-agent: Bingbot
Allow: /
User-agent: ia_archiver
Allow: /
User-agent: Slurp
Allow: /
User-agent: *
Disallow: /default.aspx?RelId=60586221
User-agent: 164.71.1.147,jp.co.fujitsu.t2 164.71.1.149,jp.co.fujitsu.t4 Robot
Disallow: /
User-agent: 194.20.32.131
Disallow: /
User-agent: 198.5.209.201
Disallow: /
User-agent: 204.95.207.62 user agent ?
Disallow: /
User-agent: 205.252.60.71
Disallow: /
User-agent: 4anything.com LinkChecker v2.0
Disallow: /
User-agent: 64.57.223.40 robot
Disallow: /
User-agent: AbachoBOT
Disallow: /
User-agent: About/0.1libwww-perl/5.47
Disallow: /
User-agent: acke.dc.luth.se
Disallow: /
User-agent: Acme.Spider
Disallow: /
User-agent: AcoiRobot
Disallow: /
User-agent: Acoon Robot v1.50.001
Disallow: /
User-agent: Ad Muncher v4.xx.x
Disallow: /
User-agent: AESOP_com_SpiderMan
Disallow: /
User-agent: Ahoy! The Homepage Finder
Disallow: /
User-agent: Aladin/3.324
Disallow: /
User-agent: Alkaline
Disallow: /
User-agent: AlkalineBOT/1.3
Disallow: /
User-agent: AlkalineBOT/1.4 (1.4.0326.0 RTM)
Disallow: /
User-agent: Allesklar/0.1 libwww-perl/5.46
Disallow: /
User-agent: Alligator 1.31 (www.nearsoftware.com)
Disallow: /
User-agent: AltaVista Intranet V2.0 AVS EVAL search@freeit.com
Allow: /
User-agent: AltaVista Intranet V2.0 Compaq Altavista Eval sveand@altavista.net
Allow: /
User-agent: AltaVista Intranet V2.0 evreka.com crawler@evreka.com
Allow: /
User-agent: Altavista robot
Allow: /
User-agent: Altavista robot ??
Allow: /
User-agent: AltaVista V2.0B crawler@evreka.com
Allow: /
User-agent: Amiga Voyager Browser, Amiga
Disallow: /
User-agent: Amiga-AWeb/3.4.167SE
Disallow: /
User-agent: ananzi
Disallow: /
User-agent: AnnoMille spider 0.1 alpha - http://www.annomille.it
Disallow: /
User-agent: Anonymized by ProxyOS: http://www.megaproxy.com
Disallow: /
User-agent: Anonymizer/1.1
Disallow: /
User-agent: Anthill
Disallow: /
User-agent: antibot-V1.1.5/i586-linux-2.2
Disallow: /
User-agent: AOL Web TV
Allow: /
User-agent: AOL, Mac
Allow: /
User-agent: AOL, Win 3.x
Allow: /
User-agent: A-Online Search
Disallow: /
User-agent: Aport
Disallow: /
User-agent: appie 1.1 (www.walhello.com)
Disallow: /
User-agent: Arachnoidea (arachnoidea@euroseek.com)
Disallow: /
User-agent: Arachnophilia
Disallow: /
User-agent: Arale
Disallow: /
User-agent: Araneo
Disallow: /
User-agent: ArchitextSpider
Disallow: /
User-agent: Aretha
Disallow: /
User-agent: ARIADNE
Disallow: /
User-agent: arks
Disallow: /
User-agent: ASpider (Associative Spider)
Disallow: /
User-agent: ASPSeek/1.2.4a
Disallow: /
User-agent: ASPSeek/1.2.5
Disallow: /
User-agent: ASPSeek/1.2.6
Disallow: /
User-agent: ASPSeek/1.2.8pre
Disallow: /
User-agent: ASPseek/1.2.9d
Disallow: /
User-agent: ASSORT/0.10
Disallow: /
User-agent: ATN Worldwide
Disallow: /
User-agent: Atomz.com Search Robot
Disallow: /
User-agent: Atomz/1.0
Disallow: /
User-agent: AURESYS
Disallow: /
User-agent: AV Fetch 1.0
Disallow: /
User-agent: AVSearch-3.0(AltaVista/AVC)
Allow: /
User-agent: BackRub
Disallow: /
User-agent: Bannerkiller
Disallow: /
User-agent: Big Brother
Disallow: /
User-agent: BigBrother/1.6e
Disallow: /
User-agent: Bizbot003
Disallow: /
User-agent: BizBot04 kirk.overleaf.com
Disallow: /
User-agent: Bjaaland
Disallow: /
User-agent: BlackWidow
Disallow: /
User-agent: Bloodhound
Disallow: /
User-agent: BMCLIENT
Disallow: /
User-agent: Borg-Bot
Disallow: /
User-agent: Bot mailto:craftbot@yahoo.com
Disallow: /
User-agent: bright.net caching robot
Disallow: /
User-agent: BSDSeek/1.0
Disallow: /
User-agent: BSpider
Disallow: /
User-agent: BullsEye
Disallow: /
User-agent: bumblebee@relevare.com
Disallow: /
User-agent: BunnySlippers
Disallow: /
User-agent: CACTVS Chemistry Spider
Disallow: /
User-agent: Calif
Disallow: /
User-agent: CaliforniaBrownSpider
Disallow: /
User-agent: Cancer Information and Support International;
Disallow: /
User-agent: Cassandra
Disallow: /
User-agent: Ceramic Tile Installation Guide (http://www.floorstransformed.com)
Disallow: /
User-agent: cg-eye interactive
Disallow: /
User-agent: Checkbot
Disallow: /
User-agent: Checkbot/1.51 LWP/5.18
Disallow: /
User-agent: Checkbot/1.6x LWP/5.46
Disallow: /
User-agent: CheckUrl
Disallow: /
User-agent: CheckWeb
Disallow: /
User-agent: ChristCrawler.com
Disallow: /
User-agent: churl
Disallow: /
User-agent: cIeNcIaFiCcIoN.nEt
Disallow: /
User-agent: CMC/0.01
Disallow: /
User-agent: ColdFusion
Disallow: /
User-agent: Collective
Disallow: /
User-agent: Combine System
Disallow: /
User-agent: combine/0.0
Disallow: /
User-agent: Commerce Browser Center
Disallow: /
User-agent: ComputingSite Robi/1.0
Disallow: /
User-agent: Conceptbot
Disallow: /
User-agent: Contact
Disallow: /
User-agent: contype
Disallow: /
User-agent: CoolBot
Disallow: /
User-agent: cosmos/0.8_(robot@xyleme.com)
Disallow: /
User-agent: cosmos/0.9_(robot@xyleme.com)
Disallow: /
User-agent: Crawler
Disallow: /
User-agent: Crawler admin@crawler.de
Disallow: /
User-agent: Crawler V 0.2.x admin@crawler.de
Disallow: /
User-agent: crawler@alexa.com
Disallow: /
User-agent: CrawlerBoy Pinpoint.com
Disallow: /
User-agent: Cuam Ver0.050bx
Disallow: /
User-agent: curl/7.8 (i686-pc-linux-gnu) libcurl 7.8 (OpenSSL 0.9.6)
Disallow: /
User-agent: curl/7.9.3 (win32) libcurl 7.9.3
Disallow: /
User-agent: CurryGuide SiteScan 1.1
Disallow: /
User-agent: Cusco
Disallow: /
User-agent: Custom Spider www.bisnisseek.com /1.0
Disallow: /
User-agent: cybercity.fr user robot / faked user agent ?
Disallow: /
User-agent: Cyberdog/2.0 (Macintosh; 68k)
Disallow: /
User-agent: CyberSpyder (amant@www.cyberspyder.com)
Disallow: /
User-agent: CyberSpyder Link Test
Disallow: /
User-agent: CyberSpyder Link Test/2.1.12 (admin@mspennyworth.com)
Disallow: /
User-agent: DA 3.5 (www.lidan.com)
Disallow: /
User-agent: DA 4.0
Disallow: /
User-agent: DA 4.0 (www.downloadaccelerator.com)
Disallow: /
User-agent: DA 5.0
Disallow: /
User-agent: dallas.mt.cs.cmu.edu
Disallow: /
User-agent: darkwing.cadvision.com
Disallow: /
User-agent: DaviesBot/1.7 (www.wholeweb.net)
Disallow: /
User-agent: dbDig(http://www.prairielandconsulting.com)
Disallow: /
User-agent: DBrowse 1.4d
Disallow: /
User-agent: dds explorer v1.0 beta
Disallow: /
User-agent: DeWeb(c) Katalog/Index
Disallow: /
User-agent: Die Blinde Kuh
Disallow: /
User-agent: DienstSpider
Disallow: /
User-agent: diff. IPs / services (for example Phenominet.com link checking)
Disallow: /
User-agent: diff. IPs / services (for example: in conjunction w. Weblink's Checker)
Disallow: /
User-agent: different IPs using the HTTPClient library (mostly link checking)
Disallow: /
User-agent: Digger
Disallow: /
User-agent: Digger/1.0 JDK/1.3.0rc3
Disallow: /
User-agent: Digimarc MarcSpider
Disallow: /
User-agent: Digimarc Marcspider/CGI
Disallow: /
User-agent: Digital Integrity Robot
Disallow: /
User-agent: DigOut4U
Disallow: /
User-agent: DIIbot/1.2
Disallow: /
User-agent: Direct Hit Grabber
Disallow: /
User-agent: DittoSpyder
Disallow: /
User-agent: dlman
Disallow: /
User-agent: DNAbot
Disallow: /
User-agent: DoCoMo/1.0/N210i/c10
Disallow: /
User-agent: Download Demon/3.x.x.x
Disallow: /
User-agent: DownLoad Express
Disallow: /
User-agent: Download Express 1.0
Disallow: /
User-agent: download manager
Disallow: /
User-agent: Download Wonder
Disallow: /
User-agent: Downloadmanager ?
Disallow: /
User-agent: DragonBot
Disallow: /
User-agent: dsl.net user robot
Disallow: /
User-agent: DSurf15a VA
Disallow: /
User-agent: DSurf15a xx
Disallow: /
User-agent: dtSearchSpider
Disallow: /
User-agent: Dual Proxy
Disallow: /
User-agent: DWCP (Dridus' Web Cataloging Project)
Disallow: /
User-agent: EbiNess
Disallow: /
User-agent: EBrowse 1.4b
Disallow: /
User-agent: eCatch/3.0
Disallow: /
User-agent: EchO!/2.0
Disallow: /
User-agent: e-collector
Disallow: /
User-agent: EI*Net/0.1 libwww/0.1
Disallow: /
User-agent: EIT Link Verifier Robot
Disallow: /
User-agent: ELFINBOT
Disallow: /
User-agent: Emacs-w3 Search Engine
Disallow: /
User-agent: EmailSiphon
Disallow: /
User-agent: EmailWolf 1.00
Disallow: /
User-agent: EnigmaBot
Disallow: /
User-agent: Enterprise_Search/1.0
Disallow: /
User-agent: EroCrawler
Disallow: /
User-agent: e-sense 1.0 ea(www.vigiltech.com/esensedisclaim.html)
Disallow: /
User-agent: ESISmartSpider
Disallow: /
User-agent: Esther
Disallow: /
User-agent: ESurf15a 15
Disallow: /
User-agent: Evliya Celebi
Disallow: /
User-agent: Excalibur Internet Spider V6.5.4
Disallow: /
User-agent: Express WebPictures (www.express-soft.com)
Disallow: /
User-agent: ExtractorPro
Disallow: /
User-agent: FairAd Client
Disallow: /
User-agent: fake ?
Disallow: /
User-agent: fake entry ??
Disallow: /
User-agent: faked user agent
Disallow: /
User-agent: FastCrawler
Disallow: /
User-agent: FastCrawler 3.0.1 (crawler@1klik.dk)
Disallow: /
User-agent: FAST-WebCrawler/2.2.10 (Multimedia Search) (crawler@fast.no; http://www.fast.
Disallow: /
User-agent: FAST-WebCrawler/2.2.6 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear
Disallow: /
User-agent: FAST-WebCrawler/2.2.7 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear
Disallow: /
User-agent: FAST-WebCrawler/2.2.8 (crawler@fast.no; http://www.fast.no/faq/faqfastwebsear
Disallow: /
User-agent: FAST-WebCrawler/3.2 test
Disallow: /
User-agent: FAST-WebCrawler/3.3 (crawler@fast.no; http://fast.no/support.php?c=faqs/crawl
Disallow: /
User-agent: FAST-WebCrawler/3.4/Nirvana (crawler@fast.no; http://fast.no/support.php?c=fa
Disallow: /
User-agent: FAST-WebCrawler/3.4/PartnerSite (crawler@fast.no; http://fast.no/support.php?
Disallow: /
User-agent: FAST-WebCrawler/3.5 (atw-crawler at fast dot no; http://fast.no/support.php?c
Disallow: /
User-agent: FavOrg
Disallow: /
User-agent: Favorites Sweeper v.2.03
Disallow: /
User-agent: Felix IDE
Disallow: /
User-agent: FetchRover
Disallow: /
User-agent: fido
Disallow: /
User-agent: FileHound 1.2
Disallow: /
User-agent: Firefly/1.0
Disallow: /
User-agent: Firefly/1.0 (compatible; Mozilla 4.0; MSIE 5.5)
Disallow: /
User-agent: Fish search
Disallow: /
User-agent: FlashGet
Disallow: /
User-agent: Floortransformed.com robot (link ckecking ??)
Disallow: /
User-agent: Fluid Dynamics Search Engine robot
Disallow: /
User-agent: flunky
Disallow: /
User-agent: fly/6.01 libwww/4.0D
Disallow: /
User-agent: Fouineur
Disallow: /
User-agent: Freecrawl
Disallow: /
User-agent: FSurf15a 01
Disallow: /
User-agent: FunnelWeb
Disallow: /
User-agent: GAIS Robot/1.0B2
Disallow: /
User-agent: GaisLab data gatherer, Gaisbot/3.0+(indexer@gais.cs.ccu.edu.tw;+http://gais.c
Disallow: /
User-agent: gammaSpider, FocusedCrawler
Disallow: /
User-agent: gazz
Disallow: /
User-agent: gazz/2.1 (gazz@nttrd.com)
Disallow: /
User-agent: GCreep
Disallow: /
User-agent: Gecko 0.7 Linux Pentium II
Disallow: /
User-agent: Gecko 0.9 Linux Pentium II
Disallow: /
User-agent: geckobot
Disallow: /
User-agent: GetBot
Disallow: /
User-agent: GetRight/3.3.3
Disallow: /
User-agent: GetRight/4.5xx
Disallow: /
User-agent: GetRight/4.x
Disallow: /
User-agent: GetterroboPlus Puu
Disallow: /
User-agent: GetURL
Disallow: /
User-agent: gigabaz/3.14 (baz@gigabaz.com; http://gigabaz.com/gigabaz/)
Disallow: /
User-agent: Gigabot/1.0
Disallow: /
User-agent: GNODSPIDER (www.gnod.net)
Disallow: /
User-agent: GNU file downloader
Disallow: /
User-agent: Go!Zilla 3.x (www.gozilla.com)
Disallow: /
User-agent: Go!Zilla/4.0.0.26
Disallow: /
User-agent: Go-Ahead-Got-It/1.1
Disallow: /
User-agent: Golem
Disallow: /
User-agent: Goofer/0.2
Disallow: /
User-agent: GotIt Web accelerator
Disallow: /
User-agent: Grapnel/0.01 Experiment
Disallow: /
User-agent: Griffon
Disallow: /
User-agent: Gromit
Disallow: /
User-agent: Gseek.com (site is offline) robot
Disallow: /
User-agent: Gulliver/1.3
Disallow: /
User-agent: Gulper Bot
Disallow: /
User-agent: Gulper Web Bot 0.2.4 (www.ecsl.cs.sunysb.edu/~maxim/cgi-bin/Link/GulperBot)
Disallow: /
User-agent: HamBot
Disallow: /
User-agent: HappyBot (gserver.kw.net)
Disallow: /
User-agent: Harvest
Disallow: /
User-agent: Harvest-NG/1.0.2
Disallow: /
User-agent: havIndex
Disallow: /
User-agent: heraSpider
Disallow: /
User-agent: HI (HTML Index) Search
Disallow: /
User-agent: Hippias/0.9 Beta
Disallow: /
User-agent: HKU WWW Octopus
Disallow: /
User-agent: HLoader
Disallow: /
User-agent: Home.com user agent
Disallow: /
User-agent: HomePageSearch(hpsearch.uni-trier.de)
Disallow: /
User-agent: Hometown Spider Pro
Disallow: /
User-agent: HotJava browser plus HTML Component 1.1.x
Disallow: /
User-agent: HotJava/1.0.1/JRE1.1.x
Disallow: /
User-agent: ht://Dig
Disallow: /
User-agent: Html Link Validator (www.lithopssoft.com)
Disallow: /
User-agent: html_analyzer
Disallow: /
User-agent: HTMLgobble
Disallow: /
User-agent: http://@nonymouse.com/ (Linux)
Disallow: /
User-agent: http://www.almaden.ibm.com/cs/crawler
Disallow: /
User-agent: Hyper-Decontextualizer
Disallow: /
User-agent: HSmShSkki
Disallow: /
User-agent: I, Robot
Disallow: /
User-agent: ia_archiver
Disallow: /
User-agent: ia_archiver/1.6
Disallow: /
User-agent: iajaBot
Disallow: /
User-agent: IBM_Planetwide
Disallow: /
User-agent: Ibot/1.0 libwww-perl/0.40
Disallow: /
User-agent: IBrowse/2.2 (AmigaOS 3.5)
Disallow: /
User-agent: IBrowse/2.2 (Windows 3.1)
Disallow: /
User-agent: iCab/2.5.2 (Macintosh; I; PPC)
Disallow: /
User-agent: ideare - SignSite/1.x
Disallow: /
User-agent: IE 2.x, Mac Power PC
Disallow: /
User-agent: IE 2.x, WinNT
Disallow: /
User-agent: IE 3.x, AOL, Win95
Disallow: /
User-agent: IE 3.x, Win 3.1
Disallow: /
User-agent: IE 3.x, Win95
Disallow: /
User-agent: IE 3.x, WinNT
Disallow: /
User-agent: IE 3.x, WinXP
Disallow: /
User-agent: IE 4.x, AOL, Win98
Disallow: /
User-agent: IE 4.x, Mac Power PC
Disallow: /
User-agent: IE 4.x, Win95
Disallow: /
User-agent: IE 4.x, WinNT
Disallow: /
User-agent: IE 5 fake user agent
Disallow: /
User-agent: IE 5.5 Compatible Browser
Disallow: /
User-agent: IE 5.5, Win2000 / Jaring.my user agent
Disallow: /
User-agent: IE 5.5, Win2000 / user agent
Disallow: /
User-agent: IE 5.5, Win2000 with MS .NET SDK
Disallow: /
User-agent: IE 5.x, Mac PowerPC, AtHome user
Disallow: /
User-agent: IE PDA Browser, Windows CE
Disallow: /
User-agent: IEFav172Free
Disallow: /
User-agent: image.kapsi.net
Disallow: /
User-agent: Imagelock
Disallow: /
User-agent: IncyWincy
Disallow: /
User-agent: IncyWincy data gatherer(webmaster@loopimprovements.com,http://www.loopimprove
Disallow: /
User-agent: IncyWincy page crawler(webmaster@loopimprovements.com,http://www.loopimprovem
Disallow: /
User-agent: IncyWincy(http://www.look.com)
Disallow: /
User-agent: IncyWincy(http://www.loopimprovements.com/robot.html)
Disallow: /
User-agent: IncyWincy/2.1(loopimprovements.com/robot.html)
Disallow: /
User-agent: Inet library
Disallow: /
User-agent: InfoNaviRobot(F107)
Disallow: /
User-agent: Informant
Disallow: /
User-agent: InfoSeek Robot 1.0
Disallow: /
User-agent: Infoseek Sidewinder
Disallow: /
User-agent: InfoSeek Sidewinder/0.9
Disallow: /
User-agent: InfoSpiders
Disallow: /
User-agent: Ingrid
Disallow: /
User-agent: Inktomi Search
Disallow: /
User-agent: Inktomi Slurp
Disallow: /
User-agent: Inspector Web
Disallow: /
User-agent: InstallShield DigitalWizard
Disallow: /
User-agent: IntelliAgent
Disallow: /
User-agent: Internet Cruiser Robot
Disallow: /
User-agent: Internet Direct Library for Borland (used as E-Mail collector)
Disallow: /
User-agent: Internet Explorer 4.0 URL check
Disallow: /
User-agent: Internet Ninja 6.0
Disallow: /
User-agent: Internet Shinchakubin
Disallow: /
User-agent: InternetLinkAgent/3.1
Disallow: /
User-agent: InternetSeer.com
Disallow: /
User-agent: Iplexx Austria spider ??
Disallow: /
User-agent: IRIX 6.5
Disallow: /
User-agent: Iron33
Disallow: /
User-agent: Israeli-search
Disallow: /
User-agent: ITI Spider
Disallow: /
User-agent: Jack
Disallow: /
User-agent: Janas (Ideare.com / Tiscali.it) robot
Disallow: /
User-agent: Java 1.1
Disallow: /
User-agent: Java Development Kit
Disallow: /
User-agent: Java1.0.21.0
Disallow: /
User-agent: Java1.1.xx.x
Disallow: /
User-agent: Java1.3.0rc1
Disallow: /
User-agent: Java1.3.x
Disallow: /
User-agent: Java1.4.0
Disallow: /
User-agent: JavaBee
Disallow: /
User-agent: JBot Java Web Robot
Disallow: /
User-agent: JCheckLinks Java hyperlink validator
Disallow: /
User-agent: JCheckLinks/0.1 RPT-HTTPClient/0.3-1
Disallow: /
User-agent: JCrawler
Disallow: /
User-agent: JDK/1.1
Disallow: /
User-agent: Jeeves
Allow: /
User-agent: Jeode/1.7.2
Disallow: /
User-agent: JetCar
Disallow: /
User-agent: Jigsaw/2.2.0 W3C_CSS_Validator_JFouffa/2.0
Disallow: /
User-agent: JoBo Java Web Robot
Disallow: /
User-agent: JoBo/@JOBO_VERSION@(http://www.matuschek.net/jobo.html)
Disallow: /
User-agent: Jobot
Disallow: /
User-agent: JoeBot
Disallow: /
User-agent: JumpStation
Disallow: /
User-agent: Katipo
Disallow: /
User-agent: KDD-Explorer
Disallow: /
User-agent: KE_1.0/2.0 libwww/5.2.8
Disallow: /
User-agent: Kenjin Spider
Disallow: /
User-agent: Kilroy
Disallow: /
User-agent: KIT-Fireball
Disallow: /
User-agent: KIT-Fireball/2.0
Disallow: /
User-agent: KIT-Fireball/2.0 (compatible; Mozilla 4.0; MSIE 5.5)
Disallow: /
User-agent: K-Meleon/0.6 (Windows; U; Windows NT 5.1; en-US; rv:0.9.5) Gecko/20011011
Disallow: /
User-agent: KO_Yappo_Robot
Disallow: /
User-agent: Konqueror 2.0.x, X11
Disallow: /
User-agent: Konqueror 2.1.x, X11
Disallow: /
User-agent: Konqueror 2.2.x
Disallow: /
User-agent: Konqueror 2.2.x, Linux
Disallow: /
User-agent: kulturarw3/0.1
Disallow: /
User-agent: LabelGrabber
Disallow: /
User-agent: larbin
Disallow: /
User-agent: larbin (samualt9@bigfoot.com)
Disallow: /
User-agent: larbin_2.1.1 larbin2.1.1@somewhere.com
Disallow: /
User-agent: larbin_2.2.0 (crawl@compete.com)
Disallow: /
User-agent: larbin_2.2.0 crawl@compete.com
Disallow: /
User-agent: larbin_2.2.1_de_Viennot (Laurent.Viennot@inria.fr)
Disallow: /
User-agent: larbin_2.2.2 (sugayama@lab7.kuis.kyoto-u.ac.jp)
Disallow: /
User-agent: larbin_2.2.2 sugayama@lab7.kuis.kyoto-u.ac.jp
Disallow: /
User-agent: larbin_2.2.2_guillaume (guillaume@liafa.jussieu.fr)
Disallow: /
User-agent: larbin_2.2.2_guillaume guillaume@liafa.jussieu.fr
Disallow: /
User-agent: larbin_2.6.0 (larbin2.6.0@unspecified.mail)
Disallow: /
User-agent: larbin_2.6.1 (larbin2.6.1@unspecified.mail)
Disallow: /
User-agent: larbin_2.6_basileocaml (basile.starynkevitch@cea.fr)
Disallow: /
User-agent: larbin_devel (http://pauillac.inria.fr/~ailleret/prog/larbin/)
Disallow: /
User-agent: LECodeChecker/3.0 libgetdoc/1.0
Disallow: /
User-agent: legs
Disallow: /
User-agent: LEIA/2.90
Disallow: /
User-agent: LEIA/3.01pr (LEIAcrawler; [SNIP])
Disallow: /
User-agent: LexiBot/1.00
Disallow: /
User-agent: Libertech-Rover www.libertech.com?
Disallow: /
User-agent: libWeb/clsHTTP -- hiongun@kt.co.kr
Disallow: /
User-agent: libwww-perl/5.41
Disallow: /
User-agent: libwww-perl/5.48
Disallow: /
User-agent: libwww-perl/5.50
Disallow: /
User-agent: libwww-perl/5.52 FP/2.1
Disallow: /
User-agent: libwww-perl/5.52 FP/4.0
Disallow: /
User-agent: libwww-perl/5.53
Disallow: /
User-agent: libwww-perl/5.63
Disallow: /
User-agent: Link checking software ?
Disallow: /
User-agent: Link Valet Online 1.1
Disallow: /
User-agent: Link Validator
Disallow: /
User-agent: Linkalarm link validation
Disallow: /
User-agent: LinkAlarm/2.x
Disallow: /
User-agent: Linkbot
Disallow: /
User-agent: Linkguard.com link validation (service is offline)
Disallow: /
User-agent: linklooker
Disallow: /
User-agent: LinkProver 2.1
Disallow: /
User-agent: Links - http://gossamer-threads.com/scripts/links/
Disallow: /
User-agent: Links (0.97pre12; Linux 2.2.14-5.0 i686; 80x24)
Disallow: /
User-agent: Links 2.0 (http://gossamer-threads.com/scripts/links/)
Disallow: /
User-agent: Links SQL (http://gossamer-threads.com/scripts/links-sql/)
Disallow: /
User-agent: LinkScan
Disallow: /
User-agent: LinkScan/8.x Unix
Disallow: /
User-agent: LinkScan/9.0 Unix
Disallow: /
User-agent: LinkScan/9.0g Unix
Disallow: /
User-agent: LinksManager.com (http://linksmanager.com/linkchecker.html)
Disallow: /
User-agent: LinkWalker
Disallow: /
User-agent: LNSpiderguy
Disallow: /
User-agent: Lockon
Disallow: /
User-agent: Lockstep Spider/1.0
Disallow: /
User-agent: logo.gif Crawler
Disallow: /
User-agent: Lotus Notes 5.0 browser
Disallow: /
User-agent: Lotus Notes browser
Disallow: /
User-agent: Lotus-Notes/4.5 ( Windows-NT )
Disallow: /
User-agent: LWP::Simple/5.22
Disallow: /
User-agent: LWP::Simple/5.36
Disallow: /
User-agent: LWP::Simple/5.50
Disallow: /
User-agent: LWP::Simple/5.53
Disallow: /
User-agent: LWP::Simple/5.63
Disallow: /
User-agent: lwp-trivial/1.32
Disallow: /
User-agent: lwp-trivial/1.34
Disallow: /
User-agent: lwp-trivial/1.35
Disallow: /
User-agent: Lycos
Disallow: /
User-agent: Lycos_Spider_(modspider)
Disallow: /
User-agent: Lycos_Spider_(T-Rex)
Disallow: /
User-agent: Lynx 2.6 Browser
Disallow: /
User-agent: Lynx 2.8 Browser
Disallow: /
User-agent: Lynx/2.6 libwww-FM/2.14
Disallow: /
User-agent: Lynx/2.8.3dev.9 libwww-FM/2.14 SSL-MM/1.4.1 OpenSSL/0.9.6
Disallow: /
User-agent: Mac WWWWorm
Disallow: /
User-agent: Mag-Net
Disallow: /
User-agent: Magpie
Disallow: /
User-agent: MantraAgent
Disallow: /
User-agent: Mariner/5.1b [de] (Win95; I ;Kolibri gncwebbot)
Disallow: /
User-agent: Marvin v0.3
Disallow: /
User-agent: marvin/infoseek
Disallow: /
User-agent: Mass Downloader 2.2
Disallow: /
User-agent: Mata Hari/2.00
Disallow: /
User-agent: Mattie
Disallow: /
User-agent: MediaCrawler-1.0 (Experimental)
Disallow: /
User-agent: MediaFox
Disallow: /
User-agent: MediaSearch/0.1
Disallow: /
User-agent: MegaSheep v1.0 (www.searchuk.com internet sheep)
Disallow: /
User-agent: Mercator-1.x
Disallow: /
User-agent: Mercator-2.0
Disallow: /
User-agent: Mercator-Scrub-1.1
Disallow: /
User-agent: Merritt/1.0
Disallow: /
User-agent: MerzScope
Disallow: /
User-agent: MetaGer-LinkChecker
Disallow: /
User-agent: MetaProducts Download Express/1.x
Disallow: /
User-agent: MFC_Tear_Sample
Disallow: /
User-agent: Microsoft URL Control - 6.00.8xxx
Disallow: /
User-agent: Microsoft.com user agent
Disallow: /
User-agent: MIIxpc/4.2
Disallow: /
User-agent: MindCrawler
Disallow: /
User-agent: Mindspring.com user robot
Disallow: /
User-agent: Mister PiX version.dll
Disallow: /
User-agent: mnoGoSearch search engine software
Disallow: /
User-agent: Mobile phone browser
Disallow: /
User-agent: moget
Disallow: /
User-agent: moget/2.1 (moget@goo.ne.jp)
Disallow: /
User-agent: MOMspider
Disallow: /
User-agent: Monster
Disallow: /
User-agent: Motor
Disallow: /
User-agent: Mozi!
Disallow: /
User-agent: Mozilla
Disallow: /
User-agent: Mozilla (Mozilla@somewhere.com)
Disallow: /
User-agent: Mozilla/1.1 (compatible; MSPIE 2.0; Windows CE)
Disallow: /
User-agent: Mozilla/1.22 (compatible; MSIE 2.0d; Windows NT)
Disallow: /
User-agent: Mozilla/2.0 (compatible; AOL 3.0; Mac_PowerPC)
Disallow: /
User-agent: Mozilla/2.0 (Compatible; AOL-IWENG 3.0; Win16)
Disallow: /
User-agent: Mozilla/2.0 (compatible; Ask Jeeves)
Allow: /
User-agent: Mozilla/2.0 (compatible; EZResult -- Internet Search Engine)
Disallow: /
User-agent: Mozilla/2.0 (compatible; MS FrontPage 3.0)
Disallow: /
User-agent: Mozilla/2.0 (compatible; MS FrontPage 4.0)
Disallow: /
User-agent: Mozilla/2.0 (compatible; MSIE 2.1; Mac_PowerPC)
Disallow: /
User-agent: Mozilla/2.0 (compatible; MSIE 3.0; AK; Windows 95)
Disallow: /
User-agent: Mozilla/2.0 (compatible; MSIE 3.0; Windows 3.1)
Disallow: /
User-agent: Mozilla/2.0 (compatible; MSIE 3.02; Update a; AK; Windows NT)
Disallow: /
User-agent: Mozilla/2.0 (compatible; MSIE 3.02; Update a; AOL 3.0; Windows 95)
Disallow: /
User-agent: Mozilla/2.0 (compatible; MSIE 3.0B; Win32)
Disallow: /
User-agent: Mozilla/2.0 (compatible; NEWT ActiveX; Win32)
Disallow: /
User-agent: Mozilla/2.0 (compatible; T-H-U-N-D-E-R-S-T-O-N-E)
Disallow: /
User-agent: Mozilla/2.0 compatible; Check&Get 1.1x (Windows 98)
Disallow: /
User-agent: Mozilla/2.01 (Win16; I)
Disallow: /
User-agent: Mozilla/2.02Gold (Win95; I)
Disallow: /
User-agent: Mozilla/3.0 (compatible)
Disallow: /
User-agent: Mozilla/3.0 (compatible; Fluffy the spider; http://www.searchhippo.com/; info
Disallow: /
User-agent: Mozilla/3.0 (compatible; Indy Library)
Disallow: /
User-agent: Mozilla/3.0 (compatible; MuscatFerret/1.5.4; claude@euroferret.com)
Disallow: /
User-agent: Mozilla/3.0 (compatible; MuscatFerret/1.5; olly@muscat.co.uk)
Disallow: /
User-agent: Mozilla/3.0 (compatible; MuscatFerret/1.6.x; claude@euroferret.com)
Disallow: /
User-agent: Mozilla/3.0 (compatible; NetPositive/2.2)
Disallow: /
User-agent: Mozilla/3.0 (compatible; Opera/3.0; Windows 3.1) v3.1
Disallow: /
User-agent: Mozilla/3.0 (compatible; Opera/3.0; Windows 95/NT4) 3.2
Disallow: /
User-agent: Mozilla/3.0 (compatible; REL Software, Web Link Validator 2.x)
Disallow: /
User-agent: Mozilla/3.0 (compatible; Web Link Validator 2.x)
Disallow: /
User-agent: Mozilla/3.0 (compatible; WebCapture 2.0; Auto; Windows)
Disallow: /
User-agent: Mozilla/3.0 (compatible; Webinator-DEV01.home.iprospect.com/2.56)
Disallow: /
User-agent: Mozilla/3.0 (Compatible;Viking/1.8)
Disallow: /
User-agent: Mozilla/3.0 (Liberate DTV 1.1)
Disallow: /
User-agent: Mozilla/3.0 (Planetweb/2.100 JS SSL US; Dreamcast US)
Disallow: /
User-agent: Mozilla/3.0 (Slurp.so/Goo; slurp@inktomi.com; http://www.inktomi.com/slurp.ht
Disallow: /
User-agent: Mozilla/3.0 (Slurp/cat; slurp@inktomi.com; http://www.inktomi.com/slurp.html)
Disallow: /
User-agent: Mozilla/3.0 (Slurp/si; slurp@inktomi.com; http://www.inktomi.com/slurp.html)
Disallow: /
User-agent: Mozilla/3.0 (Vagabondo/1.x MT; webagent@wise-guys.nl; http://webagent.wise-gu
Disallow: /
User-agent: Mozilla/3.0 (Win16; I)
Disallow: /
User-agent: Mozilla/3.0 (Win95; I)
Disallow: /
User-agent: Mozilla/3.0 (WinNT; I)
Disallow: /
User-agent: Mozilla/3.0 (WorldGate Gazelle 3.5.1 build 11; FreeBSD2.2.8-STABLE)
Disallow: /
User-agent: Mozilla/3.0 (X11; I; OSF1 V4.0 alpha)
Disallow: /
User-agent: Mozilla/3.0 NAVIO_AOLTV (11; 13; Philips; PH200; 1; R2.0C36_AOL.0110OPTIK; R2
Disallow: /
User-agent: Mozilla/3.0 WebTV/1.2 (compatible; MSIE 2.0)
Disallow: /
User-agent: Mozilla/3.01 (compatible; AmigaVoyager/2.95; AmigaOS/MC680x0)
Disallow: /
User-agent: Mozilla/3.01 (Compatible; Links2Go Similarity Engine)
Disallow: /
User-agent: Mozilla/3.01 (compatible; Netbox/3.5 R92; Linux 2.2)
Disallow: /
User-agent: Mozilla/3.01-C-MACOS8 (Macintosh; I; PPC)
Disallow: /
User-agent: Mozilla/3.01Gold (X11; I; Linux 2.0.32 i486)
Disallow: /
User-agent: Mozilla/3.01Gold (X11; I; SunOS 5.5.1 sun4m)
Disallow: /
User-agent: Mozilla/3.01SGoldC-SGI (X11; I; IRIX 6.3 IP32)
Disallow: /
User-agent: Mozilla/3.04 (compatible; ANTFresco/2.13; RISC OS 4.02)
Disallow: /
User-agent: Mozilla/3.04 (compatible; NCBrowser/2.35; ANTFresco/2.17; RISC OS-NC 5.13 Laz
Disallow: /
User-agent: Mozilla/3.x (I-Opener 1.1; Netpliance)
Disallow: /
User-agent: Mozilla/4.0 (compatible; Advanced Email Extractor v2.xx)
Disallow: /
User-agent: Mozilla/4.0 (compatible; BorderManager 3.0)
Disallow: /
User-agent: Mozilla/4.0 (compatible; DnloadMage 1.0)
Disallow: /
User-agent: Mozilla/4.0 (compatible; FastCrawler3, support-fastcrawler3@fast.no)
Disallow: /
User-agent: Mozilla/4.0 (compatible; FDSE robot)
Disallow: /
User-agent: Mozilla/4.0 (compatible; grub-client-0.2.x; Crawl your stuff with http://grub
Disallow: /
User-agent: Mozilla/4.0 (compatible; grub-client-0.3.x; Crawl your own stuff with http://
Disallow: /
User-agent: Mozilla/4.0 (compatible; ICS 1.2.xxx)
Disallow: /
User-agent: Mozilla/4.0 (compatible; Iplexx Spider/1.0, http://www.iplexx.at)
Disallow: /
User-agent: Mozilla/4.0 (compatible; Lotus-Notes/5.0; Windows-NT)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; Site Server 3.0 Robot) Indones
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; Site Server 3.0 Robot) WebQues
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.0; Windows NT; Snap.home.1)
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.01; AOL 4.0; Windows 98)
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.01; Mac_PowerPC)
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.01; MSIECrawler; Windows 95)
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.01; Windows 95; Snap.home.5)
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.01; Windows CE; MSN Companion 2.0; 800x600; C
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.01; Windows NT Windows CE)
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.01; Windows NT)
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 4.01; Windows NT; MS Search 4.0 Robot) Microsof
Allow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.0; AOL 5.0; Windows 95; DigExt; Gateway2000;
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.0; Mac_PowerPC; AtHome021)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.0; Win32) via proxy gateway CERN-HTTPD/3.0 li
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) TrueRobot; 1.5
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows 95) VoilaBot; 1.6
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows 98; DigExt; KITV4.7 Wanadoo)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.0; Windows ME) Opera 5.11 [en]
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.0; www.galaxy.com; www.psychedelix.com)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows 95) via Avirt Gateway Server
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows 98; Madasafish)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0) (samualt9@bigfoot.com)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0; NetCaptor 6.5.0RC1)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; AOL 4.0; Windows 98; GoBeez (www.gobeez.co
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 95) via proxy gateway CERN-HTTPD/3
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 95; Transmission Segment; Hotbar 2
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; KITV4.7 Wanadoo)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; SAFEXPLORER TL)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; SYMPA; Katiesoft 7; SimulBrows
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90; BTinternet V8.1)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90; MSIECrawler)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; obot)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; QXW03018)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) Active Cache Request
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0) Fetch API Request
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; .NET CLR 1.0.3705)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; AspTear 1.5)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0; N_o_k_i_a)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Win32) WebWasher 3.0
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98) REL Software, Web Link Validat
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98) Web Link Validator 2.x)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98; Net M@nager V3.02 - www.vinn.c
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Hotbar 3.0)
Disallow: /
User-agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; Q312461; IOpener Release 1
Disallow: /
User-agent: Mozilla/4.0 (compatible; Netcraft Web Server Survey)
Disallow: /
User-agent: Mozilla/4.0 (compatible; Opera/3.0; Windows 4.10) 3.51 [en]
Disallow: /
User-agent: Mozilla/4.0 (compatible; Powermarks/3.5; Windows 95/98/2000/NT)
Disallow: /
User-agent: Mozilla/4.0 (compatible; SiteKiosk 4.0; MSIE 5.0; Windows 98; SiteCoach 1.0)
Disallow: /
User-agent: Mozilla/4.0 (compatible; Win32; WinHttp.WinHttpRequest.5)
Disallow: /
User-agent: Mozilla/4.0 (compatible; www.galaxy.com)
Disallow: /
User-agent: Mozilla/4.0 (compatible; www.linkguard.com Linkguard Online 1.0; Windows NT)
Disallow: /
User-agent: Mozilla/4.0 (fantomBrowser)
Disallow: /
User-agent: Mozilla/4.0 (fantomCrew Browser)
Disallow: /
User-agent: Mozilla/4.0 (hhjhj@yahoo.com)
Disallow: /
User-agent: Mozilla/4.0 (Sleek Spider/1.2)
Disallow: /
User-agent: Mozilla/4.0 compatible ZyBorg/1.0 (ZyBorg@WISEnutbot.com; http://www.WISEnutb
Disallow: /
User-agent: Mozilla/4.0 compatible ZyBorg/1.0 for Homepage (ZyBorg@WISEnutbot.com; http:/
Disallow: /
User-agent: Mozilla/4.0 WebTV/2.6 (compatible; MSIE 4.0)
Disallow: /
User-agent: Mozilla/4.0_(compatible;_MSIE_5.0;_Windows_95)_TrueRobot/1.4 libwww/5.2.8
Disallow: /
User-agent: Mozilla/4.0_(compatible;_MSIE_5.0;_Windows_95)_VoilaBot/1.6 libwww/5.3.2
Disallow: /
User-agent: Mozilla/4.01 [en](Win95;I)
Disallow: /
User-agent: Mozilla/4.02 [en] (X11; I; SunOS 5.6 sun4u)
Disallow: /
User-agent: Mozilla/4.04 [en] (X11; I; HP-UX B.10.20 9000/712)
Disallow: /
User-agent: Mozilla/4.04 [en] (X11; I; IRIX 5.3 IP22)
Disallow: /
User-agent: Mozilla/4.05 (Macintosh; I; 68K, Nav)
Disallow: /
User-agent: Mozilla/4.05 (Macintosh; I; PPC, Nav)
Disallow: /
User-agent: Mozilla/4.05 [en] (X11; I; SunOS 4.1.4 sun4m)
Disallow: /
User-agent: Mozilla/4.5 (compatible; HTTrack 3.0x; Windows 98)
Disallow: /
User-agent: Mozilla/4.5 (compatible; iCab 2.5.3; Macintosh; I; PPC)
Disallow: /
User-agent: Mozilla/4.5 (compatible; OmniWeb/4.0.5; Mac_PowerPC)
Disallow: /
User-agent: Mozilla/4.5 (compatible; OmniWeb/4.1-beta-1; Mac_PowerPC)
Disallow: /
User-agent: Mozilla/4.5 [en]C-CCK-MCD {RuralNet} (Win98; I)
Disallow: /
User-agent: Mozilla/4.5 RPT-HTTPClient/0.3-2
Disallow: /
User-agent: Mozilla/4.5b1 [en] (X11; I; Linux 2.0.35 i586)
Disallow: /
User-agent: Mozilla/4.61 [de] (OS/2; I)
Disallow: /
User-agent: Mozilla/4.7 (compatible; http://eidetica.com/spider)
Disallow: /
User-agent: Mozilla/4.7 (compatible; OffByOne; Windows 98) Webster Pro V3.2
Disallow: /
User-agent: Mozilla/4.7 (compatible; Whizbang)
Disallow: /
User-agent: Mozilla/4.7 (compatible; WhizBang; http://www.whizbang.com/crawler)
Disallow: /
User-agent: Mozilla/4.7 [en]C-CCK-MCD {BresnanLink} (Win95; U)
Disallow: /
User-agent: Mozilla/4.7 [en]C-CCK-MCD {Yahoo;YIP052400} (Win95; I)
Allow: /
User-agent: Mozilla/4.72 [en] (BACS http://www.ba.be)
Disallow: /
User-agent: Mozilla/4.72C-CCK-MCD Caldera Systems OpenLinux [en] (X11; U; Linux 2.2.14 i6
Disallow: /
User-agent: Mozilla/4.75C-ja [ja] (X11; U; OSF1 V5.1 alpha)
Disallow: /
User-agent: Mozilla/4.76 (Windows 98; U) Opera 5.12 [en]
Disallow: /
User-agent: Mozilla/4.76 [en] (X11; U; FreeBSD 4.4-STABLE i386)
Disallow: /
User-agent: Mozilla/4.76 [en] (X11; U; SunOS 5.7 sun4u)
Disallow: /
User-agent: Mozilla/4.77C-SGI [en] (X11; U; IRIX 6.5 IP32)
Disallow: /
User-agent: Mozilla/5.0
Disallow: /
User-agent: Mozilla/5.0 (compatible; Konqueror/2.0.1; X11); Supports MD5-Digest; Supports
Disallow: /
User-agent: Mozilla/5.0 (compatible; Konqueror/2.1.1; X11)
Disallow: /
User-agent: Mozilla/5.0 (compatible; Konqueror/2.2.2)
Disallow: /
User-agent: Mozilla/5.0 (compatible; Konqueror/2.2.2; Linux 2.4.14-xfs; X11; i686)
Disallow: /
User-agent: Mozilla/5.0 (Macintosh; U; PPC; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1
Disallow: /
User-agent: Mozilla/5.0 (Slurp/cat; slurp@inktomi.com; http://www.inktomi.com/slurp.html)
Disallow: /
User-agent: Mozilla/5.0 (SunOS 5.8 sun4u; U) Opera 5.0 [en]
Disallow: /
User-agent: Mozilla/5.0 (Windows; U; Win98; en-US; rv:0.9.2) Gecko/20010726 Netscape6/6.1
Disallow: /
User-agent: Mozilla/5.0 (X11; U; Linux 2.4.2-2 i586; en-US; m18) Gecko/20010131 Netscape6
Disallow: /
User-agent: Mozilla/5.0 (X11; U; Linux 2.4.2-2 i686; en-US; 0.7) Gecko/20010316
Disallow: /
User-agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:0.9.3) Gecko/20010801
Disallow: /
User-agent: Mozilla/5.0 Galeon/1.0.2 (X11; Linux i686; U;) Gecko/20011224
Disallow: /
User-agent: Mozilla/5.0 URL-Spider
Disallow: /
User-agent: Mozilla/5.0 usww.com-Spider-for-w8.net
Disallow: /
User-agent: Mozilla/6.01 (compatible; noydb v0.42)
Disallow: /
User-agent: MS Frontpage 4.x
Disallow: /
User-agent: MSFrontPage/4.0
Disallow: /
User-agent: MSIE 4.0 (Win95)
Disallow: /
User-agent: MSProxy/2.0
Disallow: /
User-agent: multiBlocker browser
Disallow: /
User-agent: MultiText/0.1
Disallow: /
User-agent: Muncher
Disallow: /
User-agent: murph.cais.net (rapid fire... sigh)
Disallow: /
User-agent: Muscat Ferret
Disallow: /
User-agent: Mwd.Search
Disallow: /
User-agent: MyCNNSpider
Disallow: /
User-agent: myDaemon
Disallow: /
User-agent: MyGetRight/1.0.0
Disallow: /
User-agent: MyGetRight/1.0b
Disallow: /
User-agent: NABOT/5.0
Disallow: /
User-agent: nabot_1.0
Disallow: /
User-agent: NationalDirectoryAddURL/1.0
Disallow: /
User-agent: NationalDirectory-WebSpider/1.3
Disallow: /
User-agent: NCBrowser, Risc OS
Disallow: /
User-agent: NCSA Beta 1 (http://vias.ncsa.uiuc.edu/viasarchivinginformation.html)
Disallow: /
User-agent: NEC Research Agent -- compuman at research.nj.nec.com
Disallow: /
User-agent: NEC-Hayek/1.0
Disallow: /
User-agent: NEC-MeshExplorer
Disallow: /
User-agent: Nederland.zoek
Disallow: /
User-agent: NetAnts/1.2x
Disallow: /
User-agent: NetCarta WebMap Engine
Disallow: /
User-agent: NETCOMplete/x.xx
Disallow: /
User-agent: NetLookout/2.24
Disallow: /
User-agent: NetMechanic
Disallow: /
User-agent: NetMechanic Vx.0
Disallow: /
User-agent: NetPositive BEOS browser
Disallow: /
User-agent: Netprospector JavaCrawler
Disallow: /
User-agent: NetResearchServer/2.2(loopimprovements.com/robot.html)
Disallow: /
User-agent: Netscape 2.x Gold, Win95
Disallow: /
User-agent: Netscape 2.x, Win3.x, International
Disallow: /
User-agent: Netscape 3.x, FreeBSD
Disallow: /
User-agent: Netscape 3.x, Irix
Disallow: /
User-agent: Netscape 3.x, Linux
Disallow: /
User-agent: Netscape 3.x, Mac
Disallow: /
User-agent: Netscape 3.x, OSF1 V4.0 alpha
Disallow: /
User-agent: Netscape 3.x, Risc OS
Disallow: /
User-agent: Netscape 3.x, SunOS
Disallow: /
User-agent: Netscape 3.x, Win3.x
Disallow: /
User-agent: Netscape 3.x, Win95
Disallow: /
User-agent: Netscape 3.x, WinNT
Disallow: /
User-agent: Netscape 4.7x Japan OSF1 alpha
Disallow: /
User-agent: Netscape 4.7x, FreeBSD
Disallow: /
User-agent: Netscape 4.7x, SunOS
Disallow: /
User-agent: Netscape 4.x HP-Unix
Disallow: /
User-agent: Netscape 4.x IRIX
Disallow: /
User-agent: Netscape 4.x Macintosh 68k
Disallow: /
User-agent: Netscape 4.x Macintosh PowerPC
Disallow: /
User-agent: Netscape 4.x SunOS 4.1.4
Disallow: /
User-agent: Netscape 4.x SunOS 5.6
Disallow: /
User-agent: Netscape 4.x, Linux
Disallow: /
User-agent: Netscape 4.x, OS/2
Disallow: /
User-agent: Netscape 6.x, Linux, Pentium I
Disallow: /
User-agent: Netscape 6.x, Mac PowerPC
Disallow: /
User-agent: Netscape 6.x, Win98
Disallow: /
User-agent: NetScoop
Disallow: /
User-agent: Netscpape 4.7x, Caldera Open Linux, Pentium III
Disallow: /
User-agent: NetZippy
Disallow: /
User-agent: NeuralBot/0.2
Disallow: /
User-agent: newscan-online
Disallow: /
User-agent: NHSE Web Forager
Disallow: /
User-agent: NICO/1.0
Disallow: /
User-agent: Noago Spider
Disallow: /
User-agent: Nocilla/1.0
Disallow: /
User-agent: Nokia.com network
Disallow: /
User-agent: Nokia7110/1.0 (05.01) (Google WAP Proxy/1.0)
Allow: /
User-agent: Nokia-WAPToolkit/1.2 googlebot(at)googlebot.com
Allow: /
User-agent: Nomad
Disallow: /
User-agent: Northern Light Gulliver
Disallow: /
User-agent: Novell Border Manager security suite
Disallow: /
User-agent: nttdirectory_robot/0.9 (super-robot@super.navi.ocn.ne.jp)
Disallow: /
User-agent: nttrd.com / Infobee.ne.jp robot
Disallow: /
User-agent: nzexplorer
Disallow: /
User-agent: oBot ((compatible;Win32))
Disallow: /
User-agent: Occam
Disallow: /
User-agent: Octopus
Disallow: /
User-agent: Offline Explorer 1.*
Disallow: /
User-agent: OGspider
Disallow: /
User-agent: OliverPerry
Disallow: /
User-agent: OmniWeb 4.0.x Mac browser
Disallow: /
User-agent: OmniWeb 4.1.x Mac browser
Disallow: /
User-agent: Open Text Index Robot
Disallow: /
User-agent: Openfind data gatherer
Disallow: /
User-agent: Openfind data gatherer, Openbot/3.0+(robot-response@openfind.com.tw;+http://w
Disallow: /
User-agent: Openfind Robot/1.1A2
Disallow: /
User-agent: OpenTextSiteCrawler/2.9.2
Disallow: /
User-agent: Opera 3.x WinNT
Disallow: /
User-agent: Opera 3.x, Win3.x
Disallow: /
User-agent: Opera 3.x, Win95/NT
Disallow: /
User-agent: Opera 5.0 Linux
Disallow: /
User-agent: Opera 5.11 faked WinME referer
Disallow: /
User-agent: Opera 5.12 Win98
Disallow: /
User-agent: Opera 5.x, SunOS
Disallow: /
User-agent: Opera 5.x, Win 98
Disallow: /
User-agent: Opera 6.0 WinNT
Disallow: /
User-agent: Opera 6.x Linux
Disallow: /
User-agent: Opera/5.0 (Linux 2.0.38 i386; U) [en]
Disallow: /
User-agent: Opera/5.11 (Windows ME; U) [ru]
Disallow: /
User-agent: Opera/5.12 (Windows 98; U) [en]
Disallow: /
User-agent: Opera/6.0 (Linux 2.4.8-26mdk i686; U) [en]
Disallow: /
User-agent: Opera/6.0 (Windows NT 4.0; U) [de]
Disallow: /
User-agent: Oracle iMTCrawler
Disallow: /
User-agent: Oracle Ultra Search
Disallow: /
User-agent: Orb Search
Disallow: /
User-agent: Pack Rat
Disallow: /
User-agent: Packard Bell Net user robot (*not* Google)
Disallow: /
User-agent: PageBoy
Disallow: /
User-agent: PagmIEDownload
Disallow: /
User-agent: parallelContextFocusCrawler1.1parallelContextFocusCrawler1.1
Disallow: /
User-agent: ParaSite
Disallow: /
User-agent: ParaSite/1.0b (http://www.ianett.com/parasite/)
Disallow: /
User-agent: Patric
Disallow: /
User-agent: pavuk/0.9pl29b i686-pc-linux-gnu
Disallow: /
User-agent: PBrowse 1.4b
Disallow: /
User-agent: PDA Pocket IE 2.x, Windows CE
Disallow: /
User-agent: pegasus
Disallow: /
User-agent: PerlCrawler 1.0
Disallow: /
User-agent: PEval 1.4b
Disallow: /
User-agent: PGP Key Agent
Disallow: /
User-agent: Phantom
Disallow: /
User-agent: PHP/3.0.16
Disallow: /
User-agent: PHP/4.0.4pl1
Disallow: /
User-agent: PHP/4.0.6
Disallow: /
User-agent: PHP/4.1.1
Disallow: /
User-agent: PHP/4.1.2
Disallow: /
User-agent: PhpDig
Disallow: /
User-agent: PicoSearch/1.0
Disallow: /
User-agent: PiltdownMan
Disallow: /
User-agent: Pimptrain.com's robot
Disallow: /
User-agent: PingALink Monitoring Services 1.0
Disallow: /
User-agent: PingALink Monitoring Services 1.0 (http://www.pingalink.com)
Disallow: /
User-agent: Pioneer
Disallow: /
User-agent: Pizilla++ ver 2.45
Disallow: /
User-agent: PJspider/3.0 (pjspider@portaljuice.com; http://www.portaljuice.com)
Disallow: /
User-agent: PlantyNet_WebRobot_V1.9 dhkang@plantynet.com
Disallow: /
User-agent: PlumtreeWebAccessor
Disallow: /
User-agent: Pompos/1.0 pompos@iliad.fr
Disallow: /
User-agent: Poppi
Disallow: /
User-agent: Popular Iconoclast
Disallow: /
User-agent: Portal Juice Spider
Disallow: /
User-agent: PortalB Spider
Disallow: /
User-agent: processor/0.0ALPHA libwww-perl/0.20
Disallow: /
User-agent: Progressive Download
Disallow: /
User-agent: Progressive Download HTTP check
Disallow: /
User-agent: Proxy server
Disallow: /
User-agent: psbot
Disallow: /
User-agent: psbot/0.1 (+http://www.picsearch.com/bot.html)
Disallow: /
User-agent: PSurf15a 11
Disallow: /
User-agent: PSurf15a 51
Disallow: /
User-agent: PSurf15a VA
Disallow: /
User-agent: PureSight
Disallow: /
User-agent: Python-urllib/2.0a1
Disallow: /
User-agent: Quarterdecks WebCompass search tool
Disallow: /
User-agent: QueryN Metasearch
Disallow: /
User-agent: Rainbot1.1
Disallow: /
User-agent: ramBot xtreme x.x
Disallow: /
User-agent: Raven Search
Disallow: /
User-agent: RBSE Spider
Disallow: /
User-agent: rcn.com user agent ?
Disallow: /
User-agent: RealDownload/4.0.0.4x
Disallow: /
User-agent: RepoMonkey Bait & Tackle/v1.01
Disallow: /
User-agent: Resume Robot
Disallow: /
User-agent: RMA/1.0 (compatible; RealMedia)
Disallow: /
User-agent: Road Runner: The ImageScape Robot
Disallow: /
User-agent: RoadHouse Crawling System
Disallow: /
User-agent: Robbie the Robot
Disallow: /
User-agent: RoboCrawl Spider
Disallow: /
User-agent: RoboFox
Disallow: /
User-agent: Robot Francoroute
Disallow: /
User-agent: Robot@SuperSnooper.Com
Disallow: /
User-agent: Robozilla
Disallow: /
User-agent: Robozilla/1.0
Disallow: /
User-agent: Rotondo/3.1 libwww/5.3.1
Disallow: /
User-agent: Roverbot
Disallow: /
User-agent: RPT-HTTPClient/0.3-x
Disallow: /
User-agent: RRC (crawler_admin@bigfoot.com)
Disallow: /
User-agent: RRC crawler_admin@bigfoot.com
Disallow: /
User-agent: RSurf15a 41
Disallow: /
User-agent: RSurf15a 51
Disallow: /
User-agent: RSurf15a 81
Disallow: /
User-agent: RuLeS
Disallow: /
User-agent: SafetyNet Robot
Disallow: /
User-agent: SandBot
Disallow: /
User-agent: Science Traveller International 1X/1.0
Disallow: /
User-agent: Scooter
Disallow: /
User-agent: Scooter/1.0 scooter@pa.dec.com
Disallow: /
User-agent: Scooter/1.1 (custom)
Disallow: /
User-agent: Scooter/2.0 G.R.A.B. V1.1.0
Disallow: /
User-agent: Scooter/2.0 G.R.A.B. X2.0
Disallow: /
User-agent: Scooter_bh0-3.0.3
Disallow: /
User-agent: Scooter_trk3-3.0.3
Disallow: /
User-agent: Scooter2_Mercator_x-x.0
Disallow: /
User-agent: Scooter-3.0.EU
Disallow: /
User-agent: Scooter-3.0.FS
Disallow: /
User-agent: Scooter-3.0.HD
Disallow: /
User-agent: Scooter-3.0.VNS
Disallow: /
User-agent: Scooter-3.0QI
Disallow: /
User-agent: Scooter-3.2
Disallow: /
User-agent: Scooter-3.2.DIL
Disallow: /
User-agent: Scooter-3.2.EX
Disallow: /
User-agent: Scooter-3.2.JT
Disallow: /
User-agent: Scooter-3.2.NIV
Disallow: /
User-agent: Scooter-3.2.snippet
Disallow: /
User-agent: Scooter-3.3dev
Disallow: /
User-agent: Scooter-ARS-1.1-ih
Disallow: /
User-agent: scooter-venus-3.0.vns
Disallow: /
User-agent: Scooter-W3.1.2
Disallow: /
User-agent: Scooter-W3-1.0
Disallow: /
User-agent: ScoutAbout
Disallow: /
User-agent: Search.Aus-AU.COM
Disallow: /
User-agent: search.ch V1.4
Disallow: /
User-agent: search.ch V1.4.2 (spiderman@search.ch; http://www.search.ch)
Disallow: /
User-agent: Search+
Disallow: /
User-agent: SearchExpress Spider0.99
Disallow: /
User-agent: SearchProcess
Disallow: /
User-agent: SearchTone2.0 - IDEARE
Disallow: /
User-agent: secure-netz.de link checking (in conjunction w. LWP::Simple/5.50)
Disallow: /
User-agent: Senrigan
Disallow: /
User-agent: SG-Scout
Disallow: /
User-agent: ShagSeeker
Disallow: /
User-agent: Shai'Hulud
Disallow: /
User-agent: Sift
Disallow: /
User-agent: Simmany Robot Ver1.0
Disallow: /
User-agent: Site Searcher
Disallow: /
User-agent: Site Valet
Disallow: /
User-agent: sitecheck.internetseer.com
Disallow: /
User-agent: sitecheck.internetseer.com (For more info see: http://sitecheck.internetseer.
Disallow: /
User-agent: SiteSnagger
Disallow: /
User-agent: SiteTech-Rover
Disallow: /
User-agent: Skymob.com
Disallow: /
User-agent: Slarp/0.1
Disallow: /
User-agent: SLCrawler
Disallow: /
User-agent: Sleek
Disallow: /
User-agent: Slider_Search_v1-de
Disallow: /
User-agent: SlowBot
Disallow: /
User-agent: Slurp/2.0 (slurp@inktomi.com; http://www.inktomi.com/slurp.html)
Disallow: /
User-agent: Slurp/2.0-KiteWeekly (slurp@inktomi.com; http://www.inktomi.com/slurp.html)
Disallow: /
User-agent: Slurp/si (slurp@inktomi.com; http://www.inktomi.com/slurp.html)
Disallow: /
User-agent: SlySearch (slysearch@slysearch.com)
Disallow: /
User-agent: Smart Spider
Disallow: /
User-agent: SmartDownload/1.2.67 (Win32; Jan 12 1999)
Disallow: /
User-agent: SmartDownload/1.2.77 (Win32; Feb 1 2000)
Disallow: /
User-agent: SmartDownload/1.2.77 (Win32; Jun 19 2001)
Disallow: /
User-agent: Snooper
Disallow: /
User-agent: Solbot
Disallow: /
User-agent: some download agent, in conjunction w. "dlman"
Disallow: /
User-agent: some download agent, in conjunction w. "Wildsoft Surfer"
Disallow: /
User-agent: Some download manager
Disallow: /
User-agent: Some download manager (spoofed user-agent)
Disallow: /
User-agent: Some download manager spoofing as MS Frontpage 3.x
Disallow: /
User-agent: Some download manager spoofing Netscape 4.01
Disallow: /
User-agent: Some private robot (Wanadoo.fr client)
Disallow: /
User-agent: Spanner
Disallow: /
User-agent: speedfind ramBot xtreme 8.1
Disallow: /
User-agent: Speedy Spider
Disallow: /
User-agent: Spida/0.1
Disallow: /
User-agent: spider.batsch.com
Disallow: /
User-agent: spider.yellopet.com - www.yellopet.com
Disallow: /
User-agent: Spider/maxbot.com admin@maxbot.com
Disallow: /
User-agent: spider_monkey
Disallow: /
User-agent: SpiderBot
Disallow: /
User-agent: Spiderline Crawler
Disallow: /
User-agent: SpiderMan
Disallow: /
User-agent: SpiderView(tm)
Disallow: /
User-agent: Spinne/2.0 med
Disallow: /
User-agent: Spry Wizard Robot
Disallow: /
User-agent: spyder3.microsys.com
Disallow: /
User-agent: SQ Webscanner
Disallow: /
User-agent: Sqworm/2.9.81-BETA (beta_release; 20011102-760; i686-pc-linux-gnu)
Disallow: /
User-agent: Sqworm/2.9.85-BETA (beta_release; 20011115-775; i686-pc-linux-gnu)
Disallow: /
User-agent: Sqworm/2.9.89-BETA (beta_release; 20020130-839; i686-pc-linux-gnu)
Disallow: /
User-agent: SSurf15a 11
Disallow: /
User-agent: Stamina/1.4
Disallow: /
User-agent: Star Downloader
Disallow: /
User-agent: StatFetcher/1.0
Disallow: /
User-agent: Suke
Disallow: /
User-agent: SummyCrawler
Disallow: /
User-agent: suntek search engine
Disallow: /
User-agent: SuperBot/2.6 (Win32)
Disallow: /
User-agent: Surfbot
Disallow: /
User-agent: SurferF3 1/0
Disallow: /
User-agent: suzuran
Disallow: /
User-agent: Sven
Disallow: /
User-agent: swbot/0.9c libwww/5.3.1
Disallow: /
User-agent: SyncBot
Disallow: /
User-agent: SyncIT link validation
Disallow: /
User-agent: SyncIT/1.x
Disallow: /
User-agent: SyncIT/2.x
Disallow: /
User-agent: Szukacz/1.x (robot; www.szukacz.pl/jakdzialarobot.html; szukacz@proszynski.pl
Disallow: /
User-agent: TACH Black Widow
Disallow: /
User-agent: Tarantula
Disallow: /
User-agent: targetblaster.com/0.9k
Disallow: /
User-agent: tarspider
Disallow: /
User-agent: Tcl W3 Robot
Disallow: /
User-agent: TE
Disallow: /
User-agent: TeacherSoft/1.0 libwww/2.17
Disallow: /
User-agent: TeamSoft WinInet Component
Disallow: /
User-agent: TechBOT
Disallow: /
User-agent: telefonica.es user robot
Disallow: /
User-agent: Teleport Pro/1.29.1xxx
Disallow: /
User-agent: Templeton
Disallow: /
User-agent: teomaagent crawler-admin@teoma.com
Disallow: /
User-agent: teomaagent1 [crawler-admin@teoma.com]
Disallow: /
User-agent: TeomaTechnologies
Disallow: /
User-agent: The Expert HTML Source Viewer (http://www.expert-html.com)
Disallow: /
User-agent: The Jubii Indexing Robot
Disallow: /
User-agent: The NorthStar Robot
Disallow: /
User-agent: The NWI Robot
Disallow: /
User-agent: The Peregrinator
Disallow: /
User-agent: The Python Robot
Disallow: /
User-agent: The TkWWW Robot
Disallow: /
User-agent: The Web Moose
Disallow: /
User-agent: The Web Wombat
Disallow: /
User-agent: The Webfoot Robot
Disallow: /
User-agent: the World Wide Web Wanderer
Disallow: /
User-agent: The World Wide Web Worm
Disallow: /
User-agent: Time Warner Telecom user robot ?
Disallow: /
User-agent: TITAN
Disallow: /
User-agent: TitIn
Disallow: /
User-agent: tivraSpider/1.0 (crawler@tivra.com)
Disallow: /
User-agent: TJG/Spider
Disallow: /
User-agent: TLSpider
Disallow: /
User-agent: Toutatis 2.5-2
Disallow: /
User-agent: Trampelpfad-Spider
Disallow: /
User-agent: Trampelpfad-Spider-v0.1
Disallow: /
User-agent: tricosMetaCheck 1.2216-08-1999 (http://www.tricos.com/metacheck)
Disallow: /
User-agent: TSurf15a 11
Disallow: /
User-agent: Twotrees Reactive Filter V2.0
Disallow: /
User-agent: UCmore
Disallow: /
User-agent: UCMore Crawler App
Disallow: /
User-agent: UCSD Crawl
Disallow: /
User-agent: UDM
Disallow: /
User-agent: UdmSearch
Disallow: /
User-agent: UdmSearch/3.1.x
Disallow: /
User-agent: UIowaCrawler/1.0
Disallow: /
User-agent: Ultimate Search / Smartdesk (no website) robot
Disallow: /
User-agent: Ultraseek
Disallow: /
User-agent: unet.ca
Disallow: /
User-agent: unknown link-checking
Disallow: /
User-agent: Unknown robots from 64.5.216.xx, 66.98.36.xx, 148.74.255.xx
Disallow: /
User-agent: unknown user robot (24.124.34.42)
Disallow: /
User-agent: unknown.Level3.net robot
Disallow: /
User-agent: unnamed
Disallow: /
User-agent: UP.Browser/3.01-IG01 UP.Link/3.2.3.4
Disallow: /
User-agent: URL Check
Disallow: /
User-agent: URL Spider Pro
Disallow: /
User-agent: URLGetFile
Disallow: /
User-agent: user agent
Disallow: /
User-agent: user agent (HTTP header f. transferencoding)
Disallow: /
User-agent: user agent, looks for form-mail components (spam-bot)
Disallow: /
User-agent: user faked referer
Disallow: /
User-agent: UtilMind HTTPGet
Disallow: /
User-agent: Utopia WebWasher 3.0
Disallow: /
User-agent: uu.net user robot/agent
Disallow: /
User-agent: Vagabondo/1.x MT (webagent@wise-guys.nl)
Disallow: /
User-agent: Valkyrie
Disallow: /
User-agent: Verticrawl
Disallow: /
User-agent: Victoria
Disallow: /
User-agent: vision-search
Disallow: /
User-agent: Voyager
Disallow: /
User-agent: vspider/3.x
Disallow: /
User-agent: VWbot
Disallow: /
User-agent: w@pSpider by wap4.com
Disallow: /
User-agent: W3C_Validator/1.122 libwww-perl/5.50
Disallow: /
User-agent: w3index
Disallow: /
User-agent: w3m/0.1.xx
Disallow: /
User-agent: W3M2
Disallow: /
User-agent: w3mir
Disallow: /
User-agent: waldec.com
Disallow: /
User-agent: Walhello appie
Disallow: /
User-agent: WallPaper
Disallow: /
User-agent: Wanadoo Internet provider
Disallow: /
User-agent: Wanadoo Internet services
Disallow: /
User-agent: WDG_Validator/1.1
Disallow: /
User-agent: Web Core / Roots
Disallow: /
User-agent: Web Image Collector
Disallow: /
User-agent: Web Link Validator 1.5
Disallow: /
User-agent: WebBandit Web Spider
Disallow: /
User-agent: webbandit/4.45.0
Disallow: /
User-agent: WebCatcher
Disallow: /
User-agent: webcollage/1.7x
Disallow: /
User-agent: webcollage/1.8x
Disallow: /
User-agent: WebCollector
Disallow: /
User-agent: WebCompass 2.0
Disallow: /
User-agent: WebCopier v2.5
Disallow: /
User-agent: WebCopier v2.7a
Disallow: /
User-agent: WebCopy
Disallow: /
User-agent: webfetcher
Disallow: /
User-agent: WebFindBot(http://www.web-find.com)
Disallow: /
User-agent: webhack
Disallow: /
User-agent: weblayers
Disallow: /
User-agent: WebLinker
Disallow: /
User-agent: Weblink's checker/
Disallow: /
User-agent: Weblog Monitor
Disallow: /
User-agent: WebMirror
Disallow: /
User-agent: WebQuest
Disallow: /
User-agent: WebReaper
Disallow: /
User-agent: WebReaper vx.x - www.webreaper.net
Disallow: /
User-agent: webs
Disallow: /
User-agent: Websnarf
Disallow: /
User-agent: WebSpider
Disallow: /
User-agent: Webspinne/1.0 webmaster@webspinne.de
Disallow: /
User-agent: WebStolperer
Disallow: /
User-agent: WebStripper/2.0x
Disallow: /
User-agent: WebTrends/3.0 (WinNT)
Disallow: /
User-agent: WebTV
Disallow: /
User-agent: WebVac
Disallow: /
User-agent: webwalk
Disallow: /
User-agent: WebWalker
Disallow: /
User-agent: WebWatch
Disallow: /
User-agent: WebZinger
Disallow: /
User-agent: WebZIP/4.1 (http://www.spidersoft.com)
Disallow: /
User-agent: WFARC
Disallow: /
User-agent: Wget
Disallow: /
User-agent: Wget/1.x
Disallow: /
User-agent: whatUseek Winona
Disallow: /
User-agent: whatUseek_winona/3.0
Disallow: /
User-agent: WhizBang! Lab
Disallow: /
User-agent: WhoWhere Robot
Disallow: /
User-agent: Wild Ferret Web Hopper #1, #2, #3
Disallow: /
User-agent: Wildsoft Surfer
Disallow: /
User-agent: Willow Internet Crawler by Twotrees V2.1
Disallow: /
User-agent: Wind.it client user-agent ?
Disallow: /
User-agent: Windows ME, BTOpenworld Internet services
Disallow: /
User-agent: Windows ME, Internet Explorer URL check
Disallow: /
User-agent: WinME, Opera 5.x
Disallow: /
User-agent: WinXP
Disallow: /
User-agent: Wired Digital
Disallow: /
User-agent: WISEbot/1.0 (WISEbot@koreawisenut.com; http://wisebot.koreawisenut.com)
Disallow: /
User-agent: WiseWire-Spider2
Disallow: /
User-agent: wobot/1.0 from 206.214.202.45
Disallow: /
User-agent: WWW Collector
Disallow: /
User-agent: www.arianna.it
Disallow: /
User-agent: www.freeloader.com.
Disallow: /
User-agent: www2000.ogsm.vanderbilt.edu
Disallow: /
User-agent: WWWC Ver 0.2.5
Disallow: /
User-agent: WWWOFFLE/2.x
Disallow: /
User-agent: Xenu Link Sleuth 1.xx
Disallow: /
User-agent: Xenu's Link Sleuth 1.1a
Disallow: /
User-agent: XGET
Disallow: /
User-agent: xpc-mii.net HTTP server message
Disallow: /
User-agent: XYLEME Robot
Disallow: /
User-agent: xyro_(xcrawler@cosmos.inria.fr)
Disallow: /
User-agent: ZBot/1.00 (icaulfield@zeus.com)
Disallow: /
User-agent: Zeus 1680 Webster Pro V2.9 Win32
Disallow: /
User-agent: Zeus 37389 Webster Pro V2.9 Win32
Disallow: /
User-agent: Zeus 86122 Webster Pro V2.9 Win32
Disallow: /
User-agent: Zeus ThemeSite Viewer Webster Pro V2.9 Win32
Disallow: /
User-agent: Zippy v2.0 - Zippyfinder.com
Disallow: /
User-agent: ZyBorg/1.0 (ZyBorg@WISEnut.com; http://www.WISEnut.com)
Disallow: /
США - 64.39.13.168
Rackspace Hosting
Rackspace Hosting
HTTP/1.1 200 OK
Cache-Control: no-cache, must-revalidate
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Expires: -1
Server: Microsoft-IIS/8.5
Set-Cookie: ASP.NET_SessionId=xt1llwjxd3ocu13kmdv13rad; path=/; HttpOnly
X-Frame-Options: SAMEORIGIN
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Mon, 06 Jan 2020 02:18:23 GMT
Content-Length: 179091
Кнопка для анализа сайта в один клик, для установки перетащите ссылку на "Панель закладок"