Возраст домена | 17 лет |
Дата окончания | Истек срок регистрации |
ИКС | 30 |
Страниц в Google | 8690 |
Страниц в Яндексе | 24 |
Dmoz | Нет |
Яндекс Каталог | Нет |
Alexa Traffic Rank | 10964291 |
Alexa Country | Нет данных |
История изменения показателей | Авторизация |
Идет сбор информации... Обновить
Snowhawk Clan
n/a
We are the hard people of the unforgiving north.
UTF-8
39.33 КБ
841
6 045 симв.
4 961 симв.
Данные предоставлены сервисом semrush
Сайт | Общие фразы | PR | тИЦ | Alexa Rank | Alexa Country | |
---|---|---|---|---|---|---|
snowhawk.com | 5 | 3 |
0 | 4469177 | Нет данных | |
snowest.com | 4 | 4 |
0 | 290588 | 74648 | |
motosportsthibeault.com | 4 | 2 |
0 | 22316104 | Нет данных | |
youtube.com | 3 | 9 |
0 | 2 | 2 | |
wikia.com | 3 | 6 |
0 | 7125 | 5510 | |
imdb.com | 2 | 8 |
0 | 48 | 26 | |
ebay.com | 2 | 8 |
0 | 42 | 11 | |
motormaxintl.com | 2 | n/a | 0 | Нет данных | Нет данных | |
adboivin.com | 2 | 4 |
10 | 13119812 | Нет данных | |
facebook.com | 2 | 9 |
0 | 3 | 3 | |
brpclub.ru | 1 | 1 |
40 | 1164327 | 79874 | |
avito.ru | 1 | 5 |
0 | 123 | 6 | |
auto.ru | 1 | 5 |
0 | 1012 | 43 | |
adboivin.ru | 1 | 2 |
20 | 26269203 | Нет данных | |
Еще 23 сайта после авторизации |
Данные предоставлены сервисом semrush
Domain Name:SNOWHAWKCLAN.ORG
Domain ID: D136581003-LROR
Creation Date: 2007-01-05T11:51:48Z
Updated Date: 2014-01-06T01:22:11Z
Registry Expiry Date: 2015-01-05T11:51:48Z
Sponsoring Registrar:New Dream Network, LLC dba DreamHost Web Hosting (R173-LROR)
Sponsoring Registrar IANA ID: 431
WHOIS Server:
Referral URL:
Domain Status: clientTransferProhibited
Registrant ID:7eafc4b6c0d931f2
Registrant Name:Private Registrant
Registrant Organization:A Happy DreamHost Customer
Registrant Street: 417 Associated Rd #324
Registrant City:Brea
Registrant State/Province:CA
Registrant Postal Code:92821
Registrant Country:US
Registrant Phone:+1.7147064182
Registrant Phone Ext:
Registrant Fax:
Registrant Fax Ext:
Registrant Email:snowhawkclan.org@proxy.dreamhost.com
Admin ID:7eafc4b6c0d931f2
Admin Name:Private Registrant
Admin Organization:A Happy DreamHost Customer
Admin Street: 417 Associated Rd #324
Admin City:Brea
Admin State/Province:CA
Admin Postal Code:92821
Admin Country:US
Admin Phone:+1.7147064182
Admin Phone Ext:
Admin Fax:
Admin Fax Ext:
Admin Email:snowhawkclan.org@proxy.dreamhost.com
Tech ID:7eafc4b6c0d931f2
Tech Name:Private Registrant
Tech Organization:A Happy DreamHost Customer
Tech Street: 417 Associated Rd #324
Tech City:Brea
Tech State/Province:CA
Tech Postal Code:92821
Tech Country:US
Tech Phone:+1.7147064182
Tech Phone Ext:
Tech Fax:
Tech Fax Ext:
Tech Email:snowhawkclan.org@proxy.dreamhost.com
Name Server:NS1.DREAMHOST.COM
Name Server:NS2.DREAMHOST.COM
Name Server:NS3.DREAMHOST.COM
Name Server:
Name Server:
Name Server:
Name Server:
Name Server:
Name Server:
Name Server:
Name Server:
Name Server:
Name Server:
User-agent:
Disallow: /
User-agent: [NC,OR]
Disallow: /
User-agent: [NC]
Disallow: /
User-agent: #1000 new rule
Disallow: /
User-agent: #500 new rule
Disallow: /
User-agent: #bad bots end
Disallow: /
User-agent: 12soso|\
Disallow: /
User-agent: 192\.comagent|\
Disallow: /
User-agent: 1noonbot|\
Disallow: /
User-agent: 1on1searchbot|\
Disallow: /
User-agent: 3d\_search|\
Disallow: /
User-agent: 3de\_search2|\
Disallow: /
User-agent: 3g\ bot|\
Disallow: /
User-agent: 3gse|\
Disallow: /
User-agent: 50\.nu|\
Disallow: /
User-agent: RewriteCond %{HTTP_USER_AGENT} \
Disallow: /
User-agent: RewriteRule .* - [F]
Disallow: /
User-agent: a1\ sitemap\ generator|\
Disallow: /
User-agent: a1\ website\ download|\
Disallow: /
User-agent: a6\-indexer|\
Disallow: /
User-agent: aasp|\
Disallow: /
User-agent: abachobot|\
Disallow: /
User-agent: abonti|\
Disallow: /
User-agent: abotemailsearch|\
Disallow: /
User-agent: aboundex|\
Disallow: /
User-agent: aboutusbot|\
Disallow: /
User-agent: accmonitor\ compliance\ server|\
Disallow: /
User-agent: accoon|\
Disallow: /
User-agent: achulkov\.net\ page\ walker|\
Disallow: /
User-agent: acme\.spider|\
Disallow: /
User-agent: acoonbot|\
Disallow: /
User-agent: acquia\-crawler|\
Disallow: /
User-agent: activetouristbot|\
Disallow: /
User-agent: ad\ muncher|\
Disallow: /
User-agent: adamm\ bot|\
Disallow: /
User-agent: adbeat\_bot|\
Disallow: /
User-agent: adminshop\.com|\
Disallow: /
User-agent: advanced\ email\ extractor|\
Disallow: /
User-agent: aesop\_com\_spiderman|\
Disallow: /
User-agent: aespider|\
Disallow: /
User-agent: af\ knowledge\ now\ verity\ spider|\
Disallow: /
User-agent: aggregator:vocus|\
Disallow: /
User-agent: ah\-ha\.com\ crawler|\
Disallow: /
User-agent: ahrefs|\
Disallow: /
User-agent: aibot|\
Disallow: /
User-agent: aidu|\
Disallow: /
User-agent: aihitbot|\
Disallow: /
User-agent: aipbot|\
Disallow: /
User-agent: aisiid|\
Disallow: /
User-agent: aitcsrobot/1\.1|\
Disallow: /
User-agent: ajsitemap|\
Disallow: /
User-agent: akamai\-sitesnapshot|\
Disallow: /
User-agent: alexawebsearchplatform|\
Disallow: /
User-agent: alexfdownload|\
Disallow: /
User-agent: alexibot|\
Disallow: /
User-agent: alkalinebot|\
Disallow: /
User-agent: all\ acronyms\ bot|\
Disallow: /
User-agent: alpha\ search\ agent|\
Disallow: /
User-agent: amerla\ search\ bot|\
Disallow: /
User-agent: amfibibot|\
Disallow: /
User-agent: ampmppc\.com|\
Disallow: /
User-agent: amznkassocbot|\
Disallow: /
User-agent: anemone|\
Disallow: /
User-agent: anonymous|\
Disallow: /
User-agent: anotherbot|\
Disallow: /
User-agent: answerbot|\
Disallow: /
User-agent: answerbus|\
Disallow: /
User-agent: answerchase\ prove|\
Disallow: /
User-agent: antbot|\
Disallow: /
User-agent: antibot|\
Disallow: /
User-agent: antisantyworm|\
Disallow: /
User-agent: antro\.net|\
Disallow: /
User-agent: aonde\-spider|\
Disallow: /
User-agent: aport|\
Disallow: /
User-agent: appengine\-google|\
Disallow: /
User-agent: appid\:\ s\~stremor\-crawler\-|\
Disallow: /
User-agent: aqua\_products|\
Disallow: /
User-agent: arabot|\
Disallow: /
User-agent: arachmo|\
Disallow: /
User-agent: arachnophilia|\
Disallow: /
User-agent: archive\.org\_bot|\
Disallow: /
User-agent: aria\ equalizer|\
Disallow: /
User-agent: arianna\.libero\.it|\
Disallow: /
User-agent: arikus\_spider|\
Disallow: /
User-agent: art\-online\.com|\
Disallow: /
User-agent: artavisbot|\
Disallow: /
User-agent: artera|\
Disallow: /
User-agent: asaha\ search\ engine\ turkey|\
Disallow: /
User-agent: ask|\
Disallow: /
User-agent: aspider|\
Disallow: /
User-agent: aspseek|\
Disallow: /
User-agent: asterias|\
Disallow: /
User-agent: astrofind|\
Disallow: /
User-agent: athenusbot|\
Disallow: /
User-agent: atlocalbot|\
Disallow: /
User-agent: atomic\_email\_hunter|\
Disallow: /
User-agent: attach|\
Disallow: /
User-agent: attrakt|\
Disallow: /
User-agent: attributor|\
Disallow: /
User-agent: augurfind|\
Disallow: /
User-agent: auresys|\
Disallow: /
User-agent: autobaron\ crawler|\
Disallow: /
User-agent: autoemailspider|\
Disallow: /
User-agent: autowebdir|\
Disallow: /
User-agent: avsearch\-|\
Disallow: /
User-agent: axfeedsbot|\
Disallow: /
User-agent: axonize\-bot|\
Disallow: /
User-agent: ayna|\
Disallow: /
User-agent: b2w|\
Disallow: /
User-agent: backdoorbot|\
Disallow: /
User-agent: backrub|\
Disallow: /
User-agent: backstreet\ browser|\
Disallow: /
User-agent: backweb|\
Disallow: /
User-agent: baidu|\
Disallow: /
User-agent: bandit|\
Disallow: /
User-agent: batchftp|\
Disallow: /
User-agent: baypup|\
Disallow: /
User-agent: bdfetch|\
Disallow: /
User-agent: becomebot|\
Disallow: /
User-agent: becomejpbot|\
Disallow: /
User-agent: beetlebot|\
Disallow: /
User-agent: bender|\
Disallow: /
User-agent: besserscheitern\-crawl|\
Disallow: /
User-agent: betabot|\
Disallow: /
User-agent: big\ brother|\
Disallow: /
User-agent: big\ data|\
Disallow: /
User-agent: bigado\.com|\
Disallow: /
User-agent: bigcliquebot|\
Disallow: /
User-agent: bigfoot|\
Disallow: /
User-agent: biglotron|\
Disallow: /
User-agent: bilbo|\
Disallow: /
User-agent: bilgibetabot|\
Disallow: /
User-agent: bilgibot|\
Disallow: /
User-agent: bintellibot|\
Disallow: /
User-agent: bitlybot|\
Disallow: /
User-agent: bitvouseragent|\
Disallow: /
User-agent: bizbot003|\
Disallow: /
User-agent: bizbot04|\
Disallow: /
User-agent: bizworks\ retriever|\
Disallow: /
User-agent: black\ hole|\
Disallow: /
User-agent: black\.hole|\
Disallow: /
User-agent: blackbird|\
Disallow: /
User-agent: blackmask\.net\ search\ engine|\
Disallow: /
User-agent: blackwidow|\
Disallow: /
User-agent: bladder\ fusion|\
Disallow: /
User-agent: blaiz\-bee|\
Disallow: /
User-agent: blexbot|\
Disallow: /
User-agent: blinkx|\
Disallow: /
User-agent: blitzbot|\
Disallow: /
User-agent: blog\ conversation\ project|\
Disallow: /
User-agent: blogmyway|\
Disallow: /
User-agent: blogpulselive|\
Disallow: /
User-agent: blogrefsbot|\
Disallow: /
User-agent: blogscope|\
Disallow: /
User-agent: blogslive|\
Disallow: /
User-agent: bloobybot|\
Disallow: /
User-agent: blowfish|\
Disallow: /
User-agent: blt|\
Disallow: /
User-agent: bnf\.fr\_bot|\
Disallow: /
User-agent: boaconstrictor|\
Disallow: /
User-agent: boardreader|\
Disallow: /
User-agent: boi\_crawl\_00|\
Disallow: /
User-agent: boia\-scan\-agent|\
Disallow: /
User-agent: boia\.org|\
Disallow: /
User-agent: boitho|\
Disallow: /
User-agent: bookmark\ buddy\ bookmark\ checker|\
Disallow: /
User-agent: bookmark\ search\ tool|\
Disallow: /
User-agent: bosug|\
Disallow: /
User-agent: bot\ apoena|\
Disallow: /
User-agent: botalot|\
Disallow: /
User-agent: botrighthere|\
Disallow: /
User-agent: botswana|\
Disallow: /
User-agent: bottybot|\
Disallow: /
User-agent: bpbot|\
Disallow: /
User-agent: braintime\_search|\
Disallow: /
User-agent: brokenlinkcheck\.com|\
Disallow: /
User-agent: browseremulator|\
Disallow: /
User-agent: browsermob|\
Disallow: /
User-agent: bruinbot|\
Disallow: /
User-agent: bsearchr&d|\
Disallow: /
User-agent: bspider|\
Disallow: /
User-agent: btbot|\
Disallow: /
User-agent: btsearch|\
Disallow: /
User-agent: bubing|\
Disallow: /
User-agent: buddy|\
Disallow: /
User-agent: buibui|\
Disallow: /
User-agent: buildcms\ crawler|\
Disallow: /
User-agent: builtbottough|\
Disallow: /
User-agent: bullseye|\
Disallow: /
User-agent: bumblebee|\
Disallow: /
User-agent: bunnyslippers|\
Disallow: /
User-agent: buscadorclarin|\
Disallow: /
User-agent: buscaplus\ robi|\
Disallow: /
User-agent: butterfly|\
Disallow: /
User-agent: buyhawaiibot|\
Disallow: /
User-agent: buzzbot|\
Disallow: /
User-agent: byindia|\
Disallow: /
User-agent: byspider|\
Disallow: /
User-agent: byteserver|\
Disallow: /
User-agent: bzbot|\
Disallow: /
User-agent: c\ r\ a\ w\ l\ 3\ r|\
Disallow: /
User-agent: cacheblaster|\
Disallow: /
User-agent: caddbot|\
Disallow: /
User-agent: cafi|\
Disallow: /
User-agent: camcrawler|\
Disallow: /
User-agent: camelstampede|\
Disallow: /
User-agent: canon\-webrecord|\
Disallow: /
User-agent: careerbot|\
Disallow: /
User-agent: cataguru|\
Disallow: /
User-agent: catchbot|\
Disallow: /
User-agent: cazoodle|\
Disallow: /
User-agent: ccbot|\
Disallow: /
User-agent: ccgcrawl|\
Disallow: /
User-agent: ccubee|\
Disallow: /
User-agent: cd\-preload|\
Disallow: /
User-agent: ce\-preload|\
Disallow: /
User-agent: cegbfeieh|\
Disallow: /
User-agent: cerberian\ drtrs|\
Disallow: /
User-agent: cert\ figleafbot|\
Disallow: /
User-agent: cfetch|\
Disallow: /
User-agent: cfnetwork|\
Disallow: /
User-agent: chameleon|\
Disallow: /
User-agent: charlotte|\
Disallow: /
User-agent: check&get|\
Disallow: /
User-agent: checkbot|\
Disallow: /
User-agent: checklinks|\
Disallow: /
User-agent: cheesebot|\
Disallow: /
User-agent: chemiede\-nodebot|\
Disallow: /
User-agent: cherrypicker|\
Disallow: /
User-agent: chilkat|\
Disallow: /
User-agent: chinaclaw|\
Disallow: /
User-agent: cipinetbot|\
Disallow: /
User-agent: cis455crawler|\
Disallow: /
User-agent: citeseerxbot|\
Disallow: /
User-agent: cizilla|\
Disallow: /
User-agent: clariabot|\
Disallow: /
User-agent: climate\ ark|\
Disallow: /
User-agent: climateark\ spider|\
Disallow: /
User-agent: clshttp|\
Disallow: /
User-agent: clushbot|\
Disallow: /
User-agent: coast\ scan\ engine|\
Disallow: /
User-agent: coast\ webmaster\ pro|\
Disallow: /
User-agent: coccoc|\
Disallow: /
User-agent: collapsarweb|\
Disallow: /
User-agent: collector|\
Disallow: /
User-agent: colocrossing|\
Disallow: /
User-agent: combine|\
Disallow: /
User-agent: connectsearch|\
Disallow: /
User-agent: conpilot|\
Disallow: /
User-agent: contentsmartz|\
Disallow: /
User-agent: contextad\ bot|\
Disallow: /
User-agent: contype|\
Disallow: /
User-agent: cookienet|\
Disallow: /
User-agent: coolbot|\
Disallow: /
User-agent: coolcheck|\
Disallow: /
User-agent: copernic|\
Disallow: /
User-agent: copier|\
Disallow: /
User-agent: copyrightcheck|\
Disallow: /
User-agent: core\-project|\
Disallow: /
User-agent: cosmos|\
Disallow: /
User-agent: covario\-ids|\
Disallow: /
User-agent: cowbot\-|\
Disallow: /
User-agent: cowdog\ bot|\
Disallow: /
User-agent: crabbybot|\
Disallow: /
User-agent: craftbot\@yahoo\.com|\
Disallow: /
User-agent: crawl\_application|\
Disallow: /
User-agent: crawler43\.ejupiter\.com|\
Disallow: /
User-agent: crawler4j|\
Disallow: /
User-agent: crawler@|\
Disallow: /
User-agent: crawler\.kpricorn\.org|\
Disallow: /
User-agent: crawler\_for\_infomine|\
Disallow: /
User-agent: crawly|\
Disallow: /
User-agent: creativecommons|\
Disallow: /
User-agent: crescent|\
Disallow: /
User-agent: cs\-crawler|\
Disallow: /
User-agent: cse\ html\ validator|\
Disallow: /
User-agent: cshttpclient|\
Disallow: /
User-agent: cuasarbot|\
Disallow: /
User-agent: culsearch|\
Disallow: /
User-agent: curl|\
Disallow: /
User-agent: custo|\
Disallow: /
User-agent: cvaulev|\
Disallow: /
User-agent: cyberdog|\
Disallow: /
User-agent: cybernavi\_webget|\
Disallow: /
User-agent: cyberpatrol\ sitecat\ webbot|\
Disallow: /
User-agent: cyberspyder|\
Disallow: /
User-agent: cydralspider|\
Disallow: /
User-agent: d1garabicengine|\
Disallow: /
User-agent: datacha0s|\
Disallow: /
User-agent: datafountains|\
Disallow: /
User-agent: dataparksearch|\
Disallow: /
User-agent: dataprovider\.com|\
Disallow: /
User-agent: datascape\ robot|\
Disallow: /
User-agent: dataspearspiderbot|\
Disallow: /
User-agent: dataspider|\
Disallow: /
User-agent: dattatec\.com|\
Disallow: /
User-agent: daumoa|\
Disallow: /
User-agent: dblbot|\
Disallow: /
User-agent: dcpbot|\
Disallow: /
User-agent: declumbot|\
Disallow: /
User-agent: deepindex|\
Disallow: /
User-agent: deepnet\ crawler|\
Disallow: /
User-agent: deeptrawl|\
Disallow: /
User-agent: dejan|\
Disallow: /
User-agent: del\.icio\.us\-thumbnails|\
Disallow: /
User-agent: deltascan|\
Disallow: /
User-agent: delvubot|\
Disallow: /
User-agent: der\ gro§e\ bildersauger|\
Disallow: /
User-agent: der\ große\ bildersauger|\
Disallow: /
User-agent: deusu|\
Disallow: /
User-agent: dfs\-fetch|\
Disallow: /
User-agent: diagem|\
Disallow: /
User-agent: diamond|\
Disallow: /
User-agent: dibot|\
Disallow: /
User-agent: didaxusbot|\
Disallow: /
User-agent: digext|\
Disallow: /
User-agent: digger|\
Disallow: /
User-agent: digi\-rssbot|\
Disallow: /
User-agent: digitalarchivesbot|\
Disallow: /
User-agent: digout4u|\
Disallow: /
User-agent: diibot|\
Disallow: /
User-agent: dillo|\
Disallow: /
User-agent: dir\_snatch\.exe|\
Disallow: /
User-agent: disco|\
Disallow: /
User-agent: distilled\-reputation\-monitor|\
Disallow: /
User-agent: djangotraineebot|\
Disallow: /
User-agent: dkimrepbot|\
Disallow: /
User-agent: dmoz\ downloader|\
Disallow: /
User-agent: docomo|\
Disallow: /
User-agent: dof\-verify|\
Disallow: /
User-agent: domaincrawler|\
Disallow: /
User-agent: domainscan|\
Disallow: /
User-agent: domainwatcher\ bot|\
Disallow: /
User-agent: dotbot|\
Disallow: /
User-agent: dotspotsbot|\
Disallow: /
User-agent: dow\ jones\ searchbot|\
Disallow: /
User-agent: download|\
Disallow: /
User-agent: doy|\
Disallow: /
User-agent: dragonfly|\
Disallow: /
User-agent: drip|\
Disallow: /
User-agent: drone|\
Disallow: /
User-agent: dtaagent|\
Disallow: /
User-agent: dtsearchspider|\
Disallow: /
User-agent: dumbot|\
Disallow: /
User-agent: dwaar|\
Disallow: /
User-agent: dxseeker|\
Disallow: /
User-agent: e\-societyrobot|\
Disallow: /
User-agent: eah|\
Disallow: /
User-agent: earth\ platform\ indexer|\
Disallow: /
User-agent: earth\ science\ educator\ \ robot|\
Disallow: /
User-agent: easydl|\
Disallow: /
User-agent: ebingbong|\
Disallow: /
User-agent: ec2linkfinder|\
Disallow: /
User-agent: ecairn\-grabber|\
Disallow: /
User-agent: ecatch|\
Disallow: /
User-agent: echoosebot|\
Disallow: /
User-agent: edisterbot|\
Disallow: /
User-agent: edugovsearch|\
Disallow: /
User-agent: egothor|\
Disallow: /
User-agent: eidetica\.com|\
Disallow: /
User-agent: eirgrabber|\
Disallow: /
User-agent: elblindo\ the\ blind\ bot|\
Disallow: /
User-agent: elisabot|\
Disallow: /
User-agent: ellerdalebot|\
Disallow: /
User-agent: email\ exractor|\
Disallow: /
User-agent: emailcollector|\
Disallow: /
User-agent: emailleach|\
Disallow: /
User-agent: emailsiphon|\
Disallow: /
User-agent: emailwolf|\
Disallow: /
User-agent: emeraldshield|\
Disallow: /
User-agent: empas\_robot|\
Disallow: /
User-agent: enabot|\
Disallow: /
User-agent: endeca|\
Disallow: /
User-agent: enigmabot|\
Disallow: /
User-agent: enswer\ neuro\ bot|\
Disallow: /
User-agent: enter\ user\-agent|\
Disallow: /
User-agent: entitycubebot|\
Disallow: /
User-agent: erocrawler|\
Disallow: /
User-agent: estylesearch|\
Disallow: /
User-agent: esyndicat\ bot|\
Disallow: /
User-agent: eurosoft\-bot|\
Disallow: /
User-agent: evaal|\
Disallow: /
User-agent: eventware|\
Disallow: /
User-agent: everest\-vulcan\ inc\.|\
Disallow: /
User-agent: exabot|\
Disallow: /
User-agent: exactsearch|\
Disallow: /
User-agent: exactseek|\
Disallow: /
User-agent: exooba|\
Disallow: /
User-agent: exploder|\
Disallow: /
User-agent: express\ webpictures|\
Disallow: /
User-agent: extractor|\
Disallow: /
User-agent: eyenetie|\
Disallow: /
User-agent: ez\-robot|\
Disallow: /
User-agent: ezooms|\
Disallow: /
User-agent: f\-bot\ test\ pilot|\
Disallow: /
User-agent: factbot|\
Disallow: /
User-agent: fairad\ client|\
Disallow: /
User-agent: falcon|\
Disallow: /
User-agent: fast\ data\ search\ document\ retriever|\
Disallow: /
User-agent: fast\ esp|\
Disallow: /
User-agent: fast\-search\-engine|\
Disallow: /
User-agent: fastbot\ crawler|\
Disallow: /
User-agent: fastbot\.de\ crawler|\
Disallow: /
User-agent: fatbot|\
Disallow: /
User-agent: favcollector|\
Disallow: /
User-agent: faviconizer|\
Disallow: /
User-agent: favorites\ sweeper|\
Disallow: /
User-agent: fdm|\
Disallow: /
User-agent: fdse\ robot|\
Disallow: /
User-agent: fedcontractorbot|\
Disallow: /
User-agent: fembot|\
Disallow: /
User-agent: fetch\ api\ request|\
Disallow: /
User-agent: fetch\_ici|\
Disallow: /
User-agent: fgcrawler|\
Disallow: /
User-agent: filangy|\
Disallow: /
User-agent: filehound|\
Disallow: /
User-agent: findanisp\.com\_isp\_finder|\
Disallow: /
User-agent: findlinks|\
Disallow: /
User-agent: findweb|\
Disallow: /
User-agent: firebat|\
Disallow: /
User-agent: firstgov\.gov\ search|\
Disallow: /
User-agent: flaming\ attackbot|\
Disallow: /
User-agent: flamingo\_searchengine|\
Disallow: /
User-agent: flashcapture|\
Disallow: /
User-agent: flashget|\
Disallow: /
User-agent: flickysearchbot|\
Disallow: /
User-agent: fluffy\ the\ spider|\
Disallow: /
User-agent: flunky|\
Disallow: /
User-agent: focused\_crawler|\
Disallow: /
User-agent: followsite|\
Disallow: /
User-agent: foobot|\
Disallow: /
User-agent: fooooo\_web\_video\_crawl|\
Disallow: /
User-agent: fopper|\
Disallow: /
User-agent: formulafinderbot|\
Disallow: /
User-agent: forschungsportal|\
Disallow: /
User-agent: fr\_crawler|\
Disallow: /
User-agent: francis|\
Disallow: /
User-agent: freewebmonitoring\ sitechecker|\
Disallow: /
User-agent: freshcrawler|\
Disallow: /
User-agent: freshdownload|\
Disallow: /
User-agent: freshlinks\.exe|\
Disallow: /
User-agent: friendfeedbot|\
Disallow: /
User-agent: frodo\.at|\
Disallow: /
User-agent: froggle|\
Disallow: /
User-agent: frontpage|\
Disallow: /
User-agent: froola\ bot|\
Disallow: /
User-agent: fu\-nbi|\
Disallow: /
User-agent: full\_breadth\_crawler|\
Disallow: /
User-agent: funnelback|\
Disallow: /
User-agent: furlbot|\
Disallow: /
User-agent: g10\-bot|\
Disallow: /
User-agent: gaisbot|\
Disallow: /
User-agent: galaxybot|\
Disallow: /
User-agent: gazz|\
Disallow: /
User-agent: gbplugin|\
Disallow: /
User-agent: generate\_infomine\_category\_classifiers|\
Disallow: /
User-agent: genevabot|\
Disallow: /
User-agent: geniebot|\
Disallow: /
User-agent: genieo|\
Disallow: /
User-agent: geomaxenginebot|\
Disallow: /
User-agent: geometabot|\
Disallow: /
User-agent: geonabot|\
Disallow: /
User-agent: geovisu|\
Disallow: /
User-agent: germcrawler\ |\
Disallow: /
User-agent: gethtmlcontents|\
Disallow: /
User-agent: getleft|\
Disallow: /
User-agent: getright|\
Disallow: /
User-agent: getsmart|\
Disallow: /
User-agent: geturl\.rexx|\
Disallow: /
User-agent: getweb!|\
Disallow: /
User-agent: giant|\
Disallow: /
User-agent: gigablastopensource|\
Disallow: /
User-agent: gigabot|\
Disallow: /
User-agent: girafabot|\
Disallow: /
User-agent: gleamebot|\
Disallow: /
User-agent: gnome\-vfs|\
Disallow: /
User-agent: go!zilla|\
Disallow: /
User-agent: go\-ahead\-got\-it|\
Disallow: /
User-agent: go\-http\-client|\
Disallow: /
User-agent: goforit\.com|\
Disallow: /
User-agent: goforitbot|\
Disallow: /
User-agent: gold\ crawler|\
Disallow: /
User-agent: goldfire\ server|\
Disallow: /
User-agent: golem|\
Disallow: /
User-agent: goodjelly|\
Disallow: /
User-agent: gordon\-college\-google\-mini|\
Disallow: /
User-agent: goroam|\
Disallow: /
User-agent: goseebot|\
Disallow: /
User-agent: gotit|\
Disallow: /
User-agent: govbot|\
Disallow: /
User-agent: gpu\ p2p\ crawler|\
Disallow: /
User-agent: grabber|\
Disallow: /
User-agent: grabnet|\
Disallow: /
User-agent: grafula|\
Disallow: /
User-agent: grapefx|\
Disallow: /
User-agent: grapeshot|\
Disallow: /
User-agent: grbot|\
Disallow: /
User-agent: greenyogi|\
Disallow: /
User-agent: gromit|\
Disallow: /
User-agent: grub|\
Disallow: /
User-agent: gsa|\
Disallow: /
User-agent: gslfbot|\
Disallow: /
User-agent: gulliver|\
Disallow: /
User-agent: gulperbot|\
Disallow: /
User-agent: gurujibot|\
Disallow: /
User-agent: gvc\ business\ crawler|\
Disallow: /
User-agent: gvc\ crawler|\
Disallow: /
User-agent: gvc\ search\ bot|\
Disallow: /
User-agent: gvc\ web\ crawler|\
Disallow: /
User-agent: gvc\ weblink\ crawler|\
Disallow: /
User-agent: gvc\ world\ links|\
Disallow: /
User-agent: gvcbot\.com|\
Disallow: /
User-agent: happyfunbot|\
Disallow: /
User-agent: harvest|\
Disallow: /
User-agent: hatena\ antenna|\
Disallow: /
User-agent: hawler|\
Disallow: /
User-agent: hcat|\
Disallow: /
User-agent: hclsreport\-crawler|\
Disallow: /
User-agent: hd\ nutch\ agent|\
Disallow: /
User-agent: header\_test\_client|\
Disallow: /
User-agent: healia\
Disallow: /
User-agent: helix|\
Disallow: /
User-agent: here\ will\ be\ link\ to\ crawler\ site|\
Disallow: /
User-agent: heritrix|\
Disallow: /
User-agent: hiscan|\
Disallow: /
User-agent: hisoftware\ accmonitor\ server|\
Disallow: /
User-agent: hisoftware\ accverify|\
Disallow: /
User-agent: hitcrawler|\
Disallow: /
User-agent: hivabot|\
Disallow: /
User-agent: hloader|\
Disallow: /
User-agent: hmsebot|\
Disallow: /
User-agent: hmview|\
Disallow: /
User-agent: hoge|\
Disallow: /
User-agent: holmes|\
Disallow: /
User-agent: homepagesearch|\
Disallow: /
User-agent: hooblybot\-image|\
Disallow: /
User-agent: hoowwwer|\
Disallow: /
User-agent: hostcrawler|\
Disallow: /
User-agent: hsft\ \\-\ link\ scanner|\
Disallow: /
User-agent: hsft\ \\-\ lvu\ scanner|\
Disallow: /
User-agent: hslide|\
Disallow: /
User-agent: ht://check|\
Disallow: /
User-agent: htdig|\
Disallow: /
User-agent: html\ link\ validator|\
Disallow: /
User-agent: htmlparser|\
Disallow: /
User-agent: httplib|\
Disallow: /
User-agent: httrack|\
Disallow: /
User-agent: huaweisymantecspider|\
Disallow: /
User-agent: hul\-wax|\
Disallow: /
User-agent: humanlinks|\
Disallow: /
User-agent: hyperestraier|\
Disallow: /
User-agent: hyperix|\
Disallow: /
User-agent: ia\_archiver|\
Disallow: /
User-agent: iaarchiver\-|\
Disallow: /
User-agent: ibuena|\
Disallow: /
User-agent: icab|\
Disallow: /
User-agent: icds\-ingestion|\
Disallow: /
User-agent: ichiro|\
Disallow: /
User-agent: icopyright\ conductor|\
Disallow: /
User-agent: ieautodiscovery|\
Disallow: /
User-agent: iecheck|\
Disallow: /
User-agent: ihwebchecker|\
Disallow: /
User-agent: iiitbot|\
Disallow: /
User-agent: iim\_405|\
Disallow: /
User-agent: ilsebot|\
Disallow: /
User-agent: iltrovatore|\
Disallow: /
User-agent: image\ stripper|\
Disallow: /
User-agent: image\ sucker|\
Disallow: /
User-agent: image\-fetcher|\
Disallow: /
User-agent: imagebot|\
Disallow: /
User-agent: imagefortress|\
Disallow: /
User-agent: imageshereimagesthereimageseverywhere|\
Disallow: /
User-agent: imagevisu|\
Disallow: /
User-agent: imds\_monitor|\
Disallow: /
User-agent: imo\-google\-robot\-intelink|\
Disallow: /
User-agent: inagist\.com\ url\ crawler|\
Disallow: /
User-agent: indexer|\
Disallow: /
User-agent: industry\ cortex\ webcrawler|\
Disallow: /
User-agent: indy\ library|\
Disallow: /
User-agent: indylabs\_marius|\
Disallow: /
User-agent: inelabot|\
Disallow: /
User-agent: inet32\ ctrl|\
Disallow: /
User-agent: inetbot|\
Disallow: /
User-agent: info\ seeker|\
Disallow: /
User-agent: infolink|\
Disallow: /
User-agent: infomine|\
Disallow: /
User-agent: infonavirobot|\
Disallow: /
User-agent: informant|\
Disallow: /
User-agent: infoseek\ sidewinder|\
Disallow: /
User-agent: infotekies|\
Disallow: /
User-agent: infousabot|\
Disallow: /
User-agent: ingrid|\
Disallow: /
User-agent: inktomi|\
Disallow: /
User-agent: insightscollector|\
Disallow: /
User-agent: insightsworksbot|\
Disallow: /
User-agent: inspirebot|\
Disallow: /
User-agent: insumascout|\
Disallow: /
User-agent: intelix|\
Disallow: /
User-agent: intelliseek|\
Disallow: /
User-agent: interget|\
Disallow: /
User-agent: internet\ ninja|\
Disallow: /
User-agent: internet\ radio\ crawler|\
Disallow: /
User-agent: internetlinkagent|\
Disallow: /
User-agent: interseek|\
Disallow: /
User-agent: ioi|\
Disallow: /
User-agent: ip\-web\-crawler\.com|\
Disallow: /
User-agent: ipadd\ bot|\
Disallow: /
User-agent: ips\-agent|\
Disallow: /
User-agent: ipselonbot|\
Disallow: /
User-agent: iria|\
Disallow: /
User-agent: irlbot|\
Disallow: /
User-agent: iron33|\
Disallow: /
User-agent: isara|\
Disallow: /
User-agent: isearch|\
Disallow: /
User-agent: isilox|\
Disallow: /
User-agent: istellabot|\
Disallow: /
User-agent: its\-learning\ crawler|\
Disallow: /
User-agent: iu\_csci\_b659\_class\_crawler|\
Disallow: /
User-agent: ivia|\
Disallow: /
User-agent: jadynave|\
Disallow: /
User-agent: java|\
Disallow: /
User-agent: jbot|\
Disallow: /
User-agent: jemmathetourist|\
Disallow: /
User-agent: jennybot|\
Disallow: /
User-agent: jetbot|\
Disallow: /
User-agent: jetbrains\ omea\ pro|\
Disallow: /
User-agent: jetcar|\
Disallow: /
User-agent: jim|\
Disallow: /
User-agent: jobo|\
Disallow: /
User-agent: jobspider\_ba|\
Disallow: /
User-agent: joc|\
Disallow: /
User-agent: joedog|\
Disallow: /
User-agent: joyscapebot|\
Disallow: /
User-agent: jspyda|\
Disallow: /
User-agent: junut\ bot|\
Disallow: /
User-agent: justview|\
Disallow: /
User-agent: jyxobot|\
Disallow: /
User-agent: k\.s\.bot|\
Disallow: /
User-agent: kakclebot|\
Disallow: /
User-agent: kalooga|\
Disallow: /
User-agent: katatudo\-spider|\
Disallow: /
User-agent: kbeta1|\
Disallow: /
User-agent: keepni\ web\ site\ monitor|\
Disallow: /
User-agent: kenjin\.spider|\
Disallow: /
User-agent: keybot\ translation\-search\-machine|\
Disallow: /
User-agent: keywenbot|\
Disallow: /
User-agent: keyword\ density|\
Disallow: /
User-agent: keyword\.density|\
Disallow: /
User-agent: kinjabot|\
Disallow: /
User-agent: kitenga\-crawler\-bot|\
Disallow: /
User-agent: kiwistatus|\
Disallow: /
User-agent: kmbot\-|\
Disallow: /
User-agent: kmccrew\ bot\ search|\
Disallow: /
User-agent: knight|\
Disallow: /
User-agent: knowitall|\
Disallow: /
User-agent: knowledge\ engine|\
Disallow: /
User-agent: knowledge\.com|\
Disallow: /
User-agent: koepabot|\
Disallow: /
User-agent: koninklijke|\
Disallow: /
User-agent: korniki|\
Disallow: /
User-agent: krowler|\
Disallow: /
User-agent: ksbot|\
Disallow: /
User-agent: kuloko\-bot|\
Disallow: /
User-agent: kulturarw3|\
Disallow: /
User-agent: kummhttp|\
Disallow: /
User-agent: kurzor|\
Disallow: /
User-agent: kyluka\ crawl|\
Disallow: /
User-agent: l\.webis|\
Disallow: /
User-agent: labhoo|\
Disallow: /
User-agent: labourunions411|\
Disallow: /
User-agent: lachesis|\
Disallow: /
User-agent: lament|\
Disallow: /
User-agent: lamerexterminator|\
Disallow: /
User-agent: lapozzbot|\
Disallow: /
User-agent: larbin|\
Disallow: /
User-agent: lbot|\
Disallow: /
User-agent: leaptag|\
Disallow: /
User-agent: leechftp|\
Disallow: /
User-agent: leechget|\
Disallow: /
User-agent: letscrawl\.com|\
Disallow: /
User-agent: lexibot|\
Disallow: /
User-agent: lexxebot|\
Disallow: /
User-agent: lftp|\
Disallow: /
User-agent: libcrawl|\
Disallow: /
User-agent: libiviacore|\
Disallow: /
User-agent: libw|\
Disallow: /
User-agent: likse|\
Disallow: /
User-agent: linguee\ bot|\
Disallow: /
User-agent: link\ checker|\
Disallow: /
User-agent: link\ validator|\
Disallow: /
User-agent: link\_checker|\
Disallow: /
User-agent: linkalarm|\
Disallow: /
User-agent: linkbot|\
Disallow: /
User-agent: linkcheck\ by\ siteimprove\.com|\
Disallow: /
User-agent: linkcheck\ scanner|\
Disallow: /
User-agent: linkchecker|\
Disallow: /
User-agent: linkdex\.com|\
Disallow: /
User-agent: linkextractorpro|\
Disallow: /
User-agent: linklint|\
Disallow: /
User-agent: linklooker|\
Disallow: /
User-agent: linkman|\
Disallow: /
User-agent: links\ sql|\
Disallow: /
User-agent: linkscan|\
Disallow: /
User-agent: linksmanager\.com\_bot|\
Disallow: /
User-agent: linksweeper|\
Disallow: /
User-agent: linkwalker|\
Disallow: /
User-agent: litefinder|\
Disallow: /
User-agent: litlrbot|\
Disallow: /
User-agent: little\ grabber\ at\ skanktale\.com|\
Disallow: /
User-agent: livelapbot|\
Disallow: /
User-agent: lm\ harvester|\
Disallow: /
User-agent: lmqueuebot|\
Disallow: /
User-agent: lnspiderguy|\
Disallow: /
User-agent: loadtimebot|\
Disallow: /
User-agent: localcombot|\
Disallow: /
User-agent: locust|\
Disallow: /
User-agent: lolongbot|\
Disallow: /
User-agent: lookbot|\
Disallow: /
User-agent: lsearch|\
Disallow: /
User-agent: lssbot|\
Disallow: /
User-agent: lt\ scotland\ checklink|\
Disallow: /
User-agent: ltx71.com|\
Disallow: /
User-agent: lwp|\
Disallow: /
User-agent: lycos\_spider|\
Disallow: /
User-agent: lydia\ entity\ spider|\
Disallow: /
User-agent: lynnbot|\
Disallow: /
User-agent: lytranslate|\
Disallow: /
User-agent: mag\-net|\
Disallow: /
User-agent: magnet|\
Disallow: /
User-agent: magpie\-crawler|\
Disallow: /
User-agent: magus\ bot|\
Disallow: /
User-agent: mail\.ru|\
Disallow: /
User-agent: mainseek\_bot|\
Disallow: /
User-agent: mammoth|\
Disallow: /
User-agent: map\ robot|\
Disallow: /
User-agent: markwatch|\
Disallow: /
User-agent: masagool|\
Disallow: /
User-agent: masidani\_bot\_|\
Disallow: /
User-agent: mass\ downloader|\
Disallow: /
User-agent: mata\ hari|\
Disallow: /
User-agent: mata\.hari|\
Disallow: /
User-agent: matentzn\ at\ cs\ dot\ man\ dot\ ac\ dot\ uk|\
Disallow: /
User-agent: maxamine\.com\-\-robot|\
Disallow: /
User-agent: maxamine\.com\-robot|\
Disallow: /
User-agent: maxomobot|\
Disallow: /
User-agent: mcbot|\
Disallow: /
User-agent: medrabbit|\
Disallow: /
User-agent: megite|\
Disallow: /
User-agent: memacbot|\
Disallow: /
User-agent: memo|\
Disallow: /
User-agent: mendeleybot|\
Disallow: /
User-agent: mercator\-|\
Disallow: /
User-agent: mercuryboard\_user\_agent\_sql\_injection\.nasl|\
Disallow: /
User-agent: metacarta|\
Disallow: /
User-agent: metaeuro\ web\ search|\
Disallow: /
User-agent: metager2|\
Disallow: /
User-agent: metagloss|\
Disallow: /
User-agent: metal\ crawler|\
Disallow: /
User-agent: metaquerier|\
Disallow: /
User-agent: metaspider|\
Disallow: /
User-agent: metaspinner|\
Disallow: /
User-agent: metauri|\
Disallow: /
User-agent: mfcrawler|\
Disallow: /
User-agent: mfhttpscan|\
Disallow: /
User-agent: midown\ tool|\
Disallow: /
User-agent: miixpc|\
Disallow: /
User-agent: mini\-robot|\
Disallow: /
User-agent: minibot|\
Disallow: /
User-agent: minirank|\
Disallow: /
User-agent: mirror|\
Disallow: /
User-agent: missigua\ locator|\
Disallow: /
User-agent: mister\ pix|\
Disallow: /
User-agent: mister\.pix|\
Disallow: /
User-agent: miva|\
Disallow: /
User-agent: mj12bot|\
Disallow: /
User-agent: mnogosearch|\
Disallow: /
User-agent: mod\_accessibility|\
Disallow: /
User-agent: moduna\.com|\
Disallow: /
User-agent: moget|\
Disallow: /
User-agent: mojeekbot|\
Disallow: /
User-agent: monkeycrawl|\
Disallow: /
User-agent: moses|\
Disallow: /
User-agent: mowserbot|\
Disallow: /
User-agent: mqbot|\
Disallow: /
User-agent: mse360|\
Disallow: /
User-agent: msindianwebcrawl|\
Disallow: /
User-agent: msmobot|\
Disallow: /
User-agent: msnptc|\
Disallow: /
User-agent: msrbot|\
Disallow: /
User-agent: mt\-soft|\
Disallow: /
User-agent: multitext|\
Disallow: /
User-agent: my\-heritrix\-crawler|\
Disallow: /
User-agent: my\_little\_searchengine\_project|\
Disallow: /
User-agent: myapp|\
Disallow: /
User-agent: mycompanybot|\
Disallow: /
User-agent: mycrawler|\
Disallow: /
User-agent: myengines\-us\-bot|\
Disallow: /
User-agent: myfamilybot|\
Disallow: /
User-agent: myra|\
Disallow: /
User-agent: nabot|\
Disallow: /
User-agent: najdi\.si|\
Disallow: /
User-agent: nambu|\
Disallow: /
User-agent: nameprotect|\
Disallow: /
User-agent: nasa\ search|\
Disallow: /
User-agent: natchcvs|\
Disallow: /
User-agent: natweb\-bad\-link\-mailer|\
Disallow: /
User-agent: naver|\
Disallow: /
User-agent: navroad|\
Disallow: /
User-agent: nearsite|\
Disallow: /
User-agent: nec\-meshexplorer|\
Disallow: /
User-agent: neosciocrawler|\
Disallow: /
User-agent: nerdbynature\.bot|\
Disallow: /
User-agent: nerdybot|\
Disallow: /
User-agent: nerima\-crawl-|\
Disallow: /
User-agent: nessus|\
Disallow: /
User-agent: nestreader|\
Disallow: /
User-agent: net::trackback|\
Disallow: /
User-agent: net\ vampire|\
Disallow: /
User-agent: netants|\
Disallow: /
User-agent: netcarta\ cyberpilot\ pro|\
Disallow: /
User-agent: netcraft|\
Disallow: /
User-agent: netexperts|\
Disallow: /
User-agent: netid\.com\ bot|\
Disallow: /
User-agent: netmechanic|\
Disallow: /
User-agent: netprospector|\
Disallow: /
User-agent: netresearchserver|\
Disallow: /
User-agent: netseer|\
Disallow: /
User-agent: netshift=|\
Disallow: /
User-agent: netsongbot|\
Disallow: /
User-agent: netsparker|\
Disallow: /
User-agent: netspider|\
Disallow: /
User-agent: netsrcherp|\
Disallow: /
User-agent: netzip|\
Disallow: /
User-agent: newmedhunt|\
Disallow: /
User-agent: news\ bot|\
Disallow: /
User-agent: news\_search\_app|\
Disallow: /
User-agent: newsgatherer|\
Disallow: /
User-agent: newsgroupreporter|\
Disallow: /
User-agent: newstrovebot|\
Disallow: /
User-agent: nextgensearchbot|\
Disallow: /
User-agent: nextthing\.org|\
Disallow: /
User-agent: nicebot|\
Disallow: /
User-agent: nicerspro|\
Disallow: /
User-agent: niki\-bot|\
Disallow: /
User-agent: nimblecrawler|\
Disallow: /
User-agent: nimbus\-1|\
Disallow: /
User-agent: ninetowns|\
Disallow: /
User-agent: ninja|\
Disallow: /
User-agent: njuicebot|\
Disallow: /
User-agent: nlese|\
Disallow: /
User-agent: nogate|\
Disallow: /
User-agent: norbert\ the\ spider|\
Disallow: /
User-agent: noteworthybot|\
Disallow: /
User-agent: npbot|\
Disallow: /
User-agent: nrcan\ intranet\ crawler|\
Disallow: /
User-agent: nsdl\_search\_bot|\
Disallow: /
User-agent: nu\_tch|\
Disallow: /
User-agent: nuggetize\.com\ bot|\
Disallow: /
User-agent: nusearch\ spider|\
Disallow: /
User-agent: nutch|\
Disallow: /
User-agent: nwspider|\
Disallow: /
User-agent: nymesis|\
Disallow: /
User-agent: nys\-crawler|\
Disallow: /
User-agent: objectssearch|\
Disallow: /
User-agent: obot|\
Disallow: /
User-agent: obvius\ external\ linkcheck|\
Disallow: /
User-agent: ocelli|\
Disallow: /
User-agent: octopus|\
Disallow: /
User-agent: odp\ entries\ t\_st|\
Disallow: /
User-agent: oegp|\
Disallow: /
User-agent: offline\ navigator|\
Disallow: /
User-agent: offline\.explorer|\
Disallow: /
User-agent: ogspider|\
Disallow: /
User-agent: omiexplorer\_bot|\
Disallow: /
User-agent: omniexplorer|\
Disallow: /
User-agent: omnifind|\
Disallow: /
User-agent: omniweb|\
Disallow: /
User-agent: onetszukaj|\
Disallow: /
User-agent: online\ link\ validator|\
Disallow: /
User-agent: oozbot|\
Disallow: /
User-agent: openbot|\
Disallow: /
User-agent: openfind|\
Disallow: /
User-agent: openintelligencedata|\
Disallow: /
User-agent: openisearch|\
Disallow: /
User-agent: openlink\ virtuoso\ rdf\ crawler|\
Disallow: /
User-agent: opensearchserver\_bot|\
Disallow: /
User-agent: opidig|\
Disallow: /
User-agent: optidiscover|\
Disallow: /
User-agent: oracle\ secure\ enterprise\ search|\
Disallow: /
User-agent: oracle\ ultra\ search|\
Disallow: /
User-agent: orangebot|\
Disallow: /
User-agent: orisbot|\
Disallow: /
User-agent: ornl\_crawler|\
Disallow: /
User-agent: ornl\_mercury|\
Disallow: /
User-agent: osis\-project\.jp|\
Disallow: /
User-agent: oso|\
Disallow: /
User-agent: outfoxbot|\
Disallow: /
User-agent: outfoxmelonbot|\
Disallow: /
User-agent: owler\-bot|\
Disallow: /
User-agent: owsbot|\
Disallow: /
User-agent: ozelot|\
Disallow: /
User-agent: p3p\ client|\
Disallow: /
User-agent: page\_verifier|\
Disallow: /
User-agent: pagebiteshyperbot|\
Disallow: /
User-agent: pagebull|\
Disallow: /
User-agent: pagedown|\
Disallow: /
User-agent: pagefetcher|\
Disallow: /
User-agent: pagegrabber|\
Disallow: /
User-agent: pagepeeker|\
Disallow: /
User-agent: pagerank\ monitor|\
Disallow: /
User-agent: pamsnbot\.htm|\
Disallow: /
User-agent: panopy\ bot|\
Disallow: /
User-agent: panscient\.com|\
Disallow: /
User-agent: pansophica|\
Disallow: /
User-agent: papa\ foto|\
Disallow: /
User-agent: paperlibot|\
Disallow: /
User-agent: parasite|\
Disallow: /
User-agent: parsijoo|\
Disallow: /
User-agent: pathtraq|\
Disallow: /
User-agent: pattern|\
Disallow: /
User-agent: patwebbot|\
Disallow: /
User-agent: pavuk|\
Disallow: /
User-agent: paxleframework|\
Disallow: /
User-agent: pbbot|\
Disallow: /
User-agent: pcbrowser|\
Disallow: /
User-agent: pcore\-http|\
Disallow: /
User-agent: pd\-crawler|\
Disallow: /
User-agent: penthesila|\
Disallow: /
User-agent: perform\_crawl|\
Disallow: /
User-agent: perman|\
Disallow: /
User-agent: personal\ ultimate\ crawler|\
Disallow: /
User-agent: php\ version\ tracker|\
Disallow: /
User-agent: phpcrawl|\
Disallow: /
User-agent: phpdig|\
Disallow: /
User-agent: picosearch|\
Disallow: /
User-agent: pieno\ robot|\
Disallow: /
User-agent: pipbot|\
Disallow: /
User-agent: pipeliner|\
Disallow: /
User-agent: pita|\
Disallow: /
User-agent: pixfinder|\
Disallow: /
User-agent: piyushbot|\
Disallow: /
User-agent: planetwork\ bot\ search|\
Disallow: /
User-agent: plucker|\
Disallow: /
User-agent: plukkie|\
Disallow: /
User-agent: plumtree|\
Disallow: /
User-agent: pockey|\
Disallow: /
User-agent: pocohttp|\
Disallow: /
User-agent: pogodak\.ba|\
Disallow: /
User-agent: pogodak\.co\.yu|\
Disallow: /
User-agent: poirot|\
Disallow: /
User-agent: polybot|\
Disallow: /
User-agent: pompos|\
Disallow: /
User-agent: poodle\ predictor|\
Disallow: /
User-agent: popscreenbot|\
Disallow: /
User-agent: postpost|\
Disallow: /
User-agent: privacyfinder|\
Disallow: /
User-agent: projectwf\-java\-test\-crawler|\
Disallow: /
User-agent: propowerbot|\
Disallow: /
User-agent: prowebwalker|\
Disallow: /
User-agent: proxem\ websearch|\
Disallow: /
User-agent: proximic|\
Disallow: /
User-agent: proxy\ crawler|\
Disallow: /
User-agent: psbot|\
Disallow: /
User-agent: pss\-bot|\
Disallow: /
User-agent: psycheclone|\
Disallow: /
User-agent: pub\-crawler|\
Disallow: /
User-agent: pucl|\
Disallow: /
User-agent: pulsebot|\
Disallow: /
User-agent: pump|\
Disallow: /
User-agent: pwebot|\
Disallow: /
User-agent: python|\
Disallow: /
User-agent: qeavis\ agent|\
Disallow: /
User-agent: qfkbot|\
Disallow: /
User-agent: qualidade|\
Disallow: /
User-agent: qualidator\.com\ bot|\
Disallow: /
User-agent: quepasacreep|\
Disallow: /
User-agent: queryn\ metasearch|\
Disallow: /
User-agent: queryn\.metasearch|\
Disallow: /
User-agent: quest\.durato|\
Disallow: /
User-agent: quintura\-crw|\
Disallow: /
User-agent: qunarbot|\
Disallow: /
User-agent: qwantify|\
Disallow: /
User-agent: qweery\_robot\.txt\_checkbot|\
Disallow: /
User-agent: qweerybot|\
Disallow: /
User-agent: r2ibot|\
Disallow: /
User-agent: r6\_commentreader|\
Disallow: /
User-agent: r6\_feedfetcher|\
Disallow: /
User-agent: r6\_votereader|\
Disallow: /
User-agent: rabot|\
Disallow: /
User-agent: radian6|\
Disallow: /
User-agent: radiation\ retriever|\
Disallow: /
User-agent: rampybot|\
Disallow: /
User-agent: rankivabot|\
Disallow: /
User-agent: rankur|\
Disallow: /
User-agent: rational\ sitecheck|\
Disallow: /
User-agent: rcstartbot|\
Disallow: /
User-agent: realdownload|\
Disallow: /
User-agent: reaper|\
Disallow: /
User-agent: rebi\-shoveler|\
Disallow: /
User-agent: recorder|\
Disallow: /
User-agent: redbot|\
Disallow: /
User-agent: redcarpet|\
Disallow: /
User-agent: reget|\
Disallow: /
User-agent: repomonkey|\
Disallow: /
User-agent: research\ robot|\
Disallow: /
User-agent: riddler|\
Disallow: /
User-agent: riight|\
Disallow: /
User-agent: risenetbot|\
Disallow: /
User-agent: riverglassscanner\
Disallow: /
User-agent: robopal|\
Disallow: /
User-agent: robosourcer|\
Disallow: /
User-agent: robotek|\
Disallow: /
User-agent: robozilla|\
Disallow: /
User-agent: roger|\
Disallow: /
User-agent: rome\ client|\
Disallow: /
User-agent: rondello|\
Disallow: /
User-agent: rotondo|\
Disallow: /
User-agent: roverbot|\
Disallow: /
User-agent: rpt\-httpclient|\
Disallow: /
User-agent: rtgibot|\
Disallow: /
User-agent: rufusbot|\
Disallow: /
User-agent: runnk\ online\ rss\ reader|\
Disallow: /
User-agent: runnk\ rss\ aggregator|\
Disallow: /
User-agent: s2bot|\
Disallow: /
User-agent: s\~stremor\-crawler|\
Disallow: /
User-agent: safaribookmarkchecker|\
Disallow: /
User-agent: safednsbot|\
Disallow: /
User-agent: safetynet\ robot|\
Disallow: /
User-agent: saladspoon|\
Disallow: /
User-agent: sapienti|\
Disallow: /
User-agent: sapphireweb|\
Disallow: /
User-agent: sbider|\
Disallow: /
User-agent: sbl\-bot|\
Disallow: /
User-agent: scfcrawler|\
Disallow: /
User-agent: scich|\
Disallow: /
User-agent: scientificcommons\.org|\
Disallow: /
User-agent: scollspider|\
Disallow: /
User-agent: scooperbot|\
Disallow: /
User-agent: scooter|\
Disallow: /
User-agent: scoutjet|\
Disallow: /
User-agent: scrapebox|\
Disallow: /
User-agent: scrapy|\
Disallow: /
User-agent: scrawltest|\
Disallow: /
User-agent: screaming\ frog|\
Disallow: /
User-agent: scrubby|\
Disallow: /
User-agent: scspider|\
Disallow: /
User-agent: scumbot|\
Disallow: /
User-agent: search\ publisher|\
Disallow: /
User-agent: search\ x\-bot|\
Disallow: /
User-agent: search\-channel|\
Disallow: /
User-agent: search\-engine\-studio|\
Disallow: /
User-agent: search\.kumkie\.com|\
Disallow: /
User-agent: search\.updated\.com|\
Disallow: /
User-agent: search\.usgs\.gov|\
Disallow: /
User-agent: searcharoo\.net|\
Disallow: /
User-agent: searchblox|\
Disallow: /
User-agent: searchbot|\
Disallow: /
User-agent: searchengine|\
Disallow: /
User-agent: searchhippo\.com|\
Disallow: /
User-agent: searchit\-bot|\
Disallow: /
User-agent: searchmarking|\
Disallow: /
User-agent: searchmarks|\
Disallow: /
User-agent: searchmee!|\
Disallow: /
User-agent: searchmee\_v|\
Disallow: /
User-agent: searchmining|\
Disallow: /
User-agent: searchnowbot|\
Disallow: /
User-agent: searchpreview|\
Disallow: /
User-agent: searchspider\.com|\
Disallow: /
User-agent: searqubot|\
Disallow: /
User-agent: seb\ spider|\
Disallow: /
User-agent: seekbot|\
Disallow: /
User-agent: seeker\.lookseek\.com|\
Disallow: /
User-agent: seeqbot|\
Disallow: /
User-agent: seeqpod\-vertical\-crawler|\
Disallow: /
User-agent: selflinkchecker|\
Disallow: /
User-agent: semager|\
Disallow: /
User-agent: semanticdiscovery|\
Disallow: /
User-agent: semantifire|\
Disallow: /
User-agent: semisearch|\
Disallow: /
User-agent: semrushbot|\
Disallow: /
User-agent: seoengworldbot|\
Disallow: /
User-agent: seokicks|\
Disallow: /
User-agent: seznambot|\
Disallow: /
User-agent: shablastbot|\
Disallow: /
User-agent: shadowwebanalyzer|\
Disallow: /
User-agent: shareaza|\
Disallow: /
User-agent: shelob|\
Disallow: /
User-agent: sherlock|\
Disallow: /
User-agent: shim\-crawler|\
Disallow: /
User-agent: shopsalad|\
Disallow: /
User-agent: shopwiki|\
Disallow: /
User-agent: showlinks|\
Disallow: /
User-agent: showyoubot|\
Disallow: /
User-agent: siclab|\
Disallow: /
User-agent: silk|\
Disallow: /
User-agent: simplepie|\
Disallow: /
User-agent: siphon|\
Disallow: /
User-agent: sitebot|\
Disallow: /
User-agent: sitecheck|\
Disallow: /
User-agent: sitefinder|\
Disallow: /
User-agent: siteguardbot|\
Disallow: /
User-agent: siteorbiter|\
Disallow: /
User-agent: sitesnagger|\
Disallow: /
User-agent: sitesucker|\
Disallow: /
User-agent: sitesweeper|\
Disallow: /
User-agent: sitexpert|\
Disallow: /
User-agent: skimbot|\
Disallow: /
User-agent: skimwordsbot|\
Disallow: /
User-agent: skreemrbot|\
Disallow: /
User-agent: skywalker|\
Disallow: /
User-agent: sleipnir|\
Disallow: /
User-agent: slow\-crawler|\
Disallow: /
User-agent: slysearch|\
Disallow: /
User-agent: smart\-crawler|\
Disallow: /
User-agent: smartdownload|\
Disallow: /
User-agent: smarte\ bot|\
Disallow: /
User-agent: smartwit\.com|\
Disallow: /
User-agent: snake|\
Disallow: /
User-agent: snap\.com\ beta\ crawler|\
Disallow: /
User-agent: snapbot|\
Disallow: /
User-agent: snappreviewbot|\
Disallow: /
User-agent: snappy|\
Disallow: /
User-agent: snookit|\
Disallow: /
User-agent: snooper|\
Disallow: /
User-agent: snoopy|\
Disallow: /
User-agent: societyrobot|\
Disallow: /
User-agent: socscibot|\
Disallow: /
User-agent: soft411\ directory|\
Disallow: /
User-agent: sogou|\
Disallow: /
User-agent: sohu\ agent|\
Disallow: /
User-agent: sohu\-search|\
Disallow: /
User-agent: sokitomi\ crawl|\
Disallow: /
User-agent: solbot|\
Disallow: /
User-agent: sondeur|\
Disallow: /
User-agent: sootle|\
Disallow: /
User-agent: sosospider|\
Disallow: /
User-agent: space\ bison|\
Disallow: /
User-agent: space\ fung|\
Disallow: /
User-agent: spacebison|\
Disallow: /
User-agent: spankbot|\
Disallow: /
User-agent: spanner|\
Disallow: /
User-agent: spatineo\ monitor\ controller|\
Disallow: /
User-agent: spatineo\ serval\ controller|\
Disallow: /
User-agent: spatineo\ serval\ getmapbot|\
Disallow: /
User-agent: special\_archiver|\
Disallow: /
User-agent: speedy|\
Disallow: /
User-agent: sphere\ scout|\
Disallow: /
User-agent: sphider|\
Disallow: /
User-agent: spider\.terranautic\.net|\
Disallow: /
User-agent: spiderengine|\
Disallow: /
User-agent: spiderku|\
Disallow: /
User-agent: spiderman|\
Disallow: /
User-agent: spinn3r|\
Disallow: /
User-agent: spinne|\
Disallow: /
User-agent: sportcrew\-bot|\
Disallow: /
User-agent: sproose|\
Disallow: /
User-agent: spyder3\.microsys\.com|\
Disallow: /
User-agent: sq\ webscanner|\
Disallow: /
User-agent: sqlmap|\
Disallow: /
User-agent: squid\-prefetch|\
Disallow: /
User-agent: squidclamav\_redirector|\
Disallow: /
User-agent: sqworm|\
Disallow: /
User-agent: srevbot|\
Disallow: /
User-agent: sslbot|\
Disallow: /
User-agent: ssm\ agent|\
Disallow: /
User-agent: stackrambler|\
Disallow: /
User-agent: stardownloader|\
Disallow: /
User-agent: statbot|\
Disallow: /
User-agent: statcrawler|\
Disallow: /
User-agent: statedept\-crawler|\
Disallow: /
User-agent: steeler|\
Disallow: /
User-agent: stegmann\-bot|\
Disallow: /
User-agent: stero|\
Disallow: /
User-agent: stripper|\
Disallow: /
User-agent: stumbler|\
Disallow: /
User-agent: suchclip|\
Disallow: /
User-agent: sucker|\
Disallow: /
User-agent: sumeetbot|\
Disallow: /
User-agent: sumitbot|\
Disallow: /
User-agent: summizebot|\
Disallow: /
User-agent: summizefeedreader|\
Disallow: /
User-agent: sunrise\ xp|\
Disallow: /
User-agent: superbot|\
Disallow: /
User-agent: superhttp|\
Disallow: /
User-agent: superlumin\ downloader|\
Disallow: /
User-agent: superpagesbot|\
Disallow: /
User-agent: supremesearch\.net|\
Disallow: /
User-agent: supybot|\
Disallow: /
User-agent: surdotlybot|\
Disallow: /
User-agent: surf|\
Disallow: /
User-agent: surveybot|\
Disallow: /
User-agent: suzuran|\
Disallow: /
User-agent: swebot|\
Disallow: /
User-agent: swish\-e|\
Disallow: /
User-agent: sygolbot|\
Disallow: /
User-agent: synapticwalker|\
Disallow: /
User-agent: syntryx\ ant\ scout\ chassis\ pheromone|\
Disallow: /
User-agent: systemsearch\-robot|\
Disallow: /
User-agent: szukacz|\
Disallow: /
User-agent: t\-h\-u\-n\-d\-e\-r\-s\-t\-o\-n\-e|\
Disallow: /
User-agent: tailrank|\
Disallow: /
User-agent: takeout|\
Disallow: /
User-agent: talkro\ web\-shot|\
Disallow: /
User-agent: tamu\_crawler|\
Disallow: /
User-agent: tapuzbot|\
Disallow: /
User-agent: tarantula|\
Disallow: /
User-agent: targetblaster\.com|\
Disallow: /
User-agent: targetyournews\.com\ bot|\
Disallow: /
User-agent: tausdatabot|\
Disallow: /
User-agent: taxinomiabot|\
Disallow: /
User-agent: teamsoft\ wininet\ component|\
Disallow: /
User-agent: tecomi\ bot|\
Disallow: /
User-agent: teezirbot|\
Disallow: /
User-agent: teleport|\
Disallow: /
User-agent: telesoft|\
Disallow: /
User-agent: teradex\ mapper|\
Disallow: /
User-agent: teragram\_crawler|\
Disallow: /
User-agent: terrawizbot|\
Disallow: /
User-agent: testbot|\
Disallow: /
User-agent: testing\ of\ bot|\
Disallow: /
User-agent: textbot|\
Disallow: /
User-agent: thatrobotsite\.com|\
Disallow: /
User-agent: the\ dyslexalizer|\
Disallow: /
User-agent: the\ intraformant|\
Disallow: /
User-agent: the\.intraformant|\
Disallow: /
User-agent: thenomad|\
Disallow: /
User-agent: theophrastus|\
Disallow: /
User-agent: theusefulbot|\
Disallow: /
User-agent: thumbbot|\
Disallow: /
User-agent: thumbnail\.cz\ robot|\
Disallow: /
User-agent: thumbshots\-de\-bot|\
Disallow: /
User-agent: tigerbot|\
Disallow: /
User-agent: tighttwatbot|\
Disallow: /
User-agent: tineye|\
Disallow: /
User-agent: titan|\
Disallow: /
User-agent: to\-dress\_ru\_bot\_|\
Disallow: /
User-agent: to\-night\-bot|\
Disallow: /
User-agent: tocrawl|\
Disallow: /
User-agent: topicalizer|\
Disallow: /
User-agent: topicblogs|\
Disallow: /
User-agent: toplistbot|\
Disallow: /
User-agent: topserver\ php|\
Disallow: /
User-agent: topyx\-crawler|\
Disallow: /
User-agent: touche|\
Disallow: /
User-agent: tourlentascanner|\
Disallow: /
User-agent: tpsystem|\
Disallow: /
User-agent: traazi|\
Disallow: /
User-agent: transgenikbot|\
Disallow: /
User-agent: travel\-search|\
Disallow: /
User-agent: travelbot|\
Disallow: /
User-agent: travellazerbot|\
Disallow: /
User-agent: treezy|\
Disallow: /
User-agent: trendiction|\
Disallow: /
User-agent: trex|\
Disallow: /
User-agent: tridentspider|\
Disallow: /
User-agent: trovator|\
Disallow: /
User-agent: true\_robot|\
Disallow: /
User-agent: tscholarsbot|\
Disallow: /
User-agent: tsm\ translation\-search\-machine|\
Disallow: /
User-agent: tswebbot|\
Disallow: /
User-agent: tulipchain|\
Disallow: /
User-agent: turingos|\
Disallow: /
User-agent: turnitinbot|\
Disallow: /
User-agent: tutorgigbot|\
Disallow: /
User-agent: tweetedtimes\ bot|\
Disallow: /
User-agent: tweetmemebot|\
Disallow: /
User-agent: twengabot|\
Disallow: /
User-agent: twice|\
Disallow: /
User-agent: twikle|\
Disallow: /
User-agent: twinuffbot|\
Disallow: /
User-agent: twisted\ pagegetter|\
Disallow: /
User-agent: twitturls|\
Disallow: /
User-agent: twitturly|\
Disallow: /
User-agent: tygobot|\
Disallow: /
User-agent: tygoprowler|\
Disallow: /
User-agent: typhoeus|\
Disallow: /
User-agent: u\.s\.\ government\ printing\ office|\
Disallow: /
User-agent: uberbot|\
Disallow: /
User-agent: ucb\-nutch|\
Disallow: /
User-agent: udmsearch|\
Disallow: /
User-agent: ufam\-crawler\-|\
Disallow: /
User-agent: ultraseek|\
Disallow: /
User-agent: unchaos|\
Disallow: /
User-agent: unidentified|\
Disallow: /
User-agent: unisterbot|\
Disallow: /
User-agent: unitek\ uniengine|\
Disallow: /
User-agent: universalsearch|\
Disallow: /
User-agent: unwindfetchor|\
Disallow: /
User-agent: uoftdb\_experiment|\
Disallow: /
User-agent: updated|\
Disallow: /
User-agent: url\ control|\
Disallow: /
User-agent: url\-checker|\
Disallow: /
User-agent: url\_gather|\
Disallow: /
User-agent: urlappendbot|\
Disallow: /
User-agent: urlblaze|\
Disallow: /
User-agent: urlchecker|\
Disallow: /
User-agent: urlck|\
Disallow: /
User-agent: urldispatcher|\
Disallow: /
User-agent: urlspiderpro|\
Disallow: /
User-agent: urly\ warning|\
Disallow: /
User-agent: urly\.warning|\
Disallow: /
User-agent: usaf\ afkn\ k2spider|\
Disallow: /
User-agent: usasearch|\
Disallow: /
User-agent: uss\-cosmix|\
Disallow: /
User-agent: usyd\-nlp\-spider|\
Disallow: /
User-agent: vacobot|\
Disallow: /
User-agent: vacuum|\
Disallow: /
User-agent: vadixbot|\
Disallow: /
User-agent: vagabondo|\
Disallow: /
User-agent: validator|\
Disallow: /
User-agent: valkyrie|\
Disallow: /
User-agent: vbseo|\
Disallow: /
User-agent: vci\ webviewer\ vci\ webviewer\ win32|\
Disallow: /
User-agent: verbstarbot|\
Disallow: /
User-agent: vericitecrawler|\
Disallow: /
User-agent: verifactrola|\
Disallow: /
User-agent: verity\-url\-gateway|\
Disallow: /
User-agent: vermut|\
Disallow: /
User-agent: versus\ crawler|\
Disallow: /
User-agent: versus\.integis\.ch|\
Disallow: /
User-agent: viasarchivinginformation\.html|\
Disallow: /
User-agent: vipr|\
Disallow: /
User-agent: virus\-detector|\
Disallow: /
User-agent: virus\_detector|\
Disallow: /
User-agent: visbot|\
Disallow: /
User-agent: vishal\ for\ clia|\
Disallow: /
User-agent: visweb|\
Disallow: /
User-agent: vital\ search'n\ urchin|\
Disallow: /
User-agent: vlad|\
Disallow: /
User-agent: vlsearch|\
Disallow: /
User-agent: vmbot|\
Disallow: /
User-agent: vocusbot|\
Disallow: /
User-agent: voideye|\
Disallow: /
User-agent: voilabot|\
Disallow: /
User-agent: voil|\
Disallow: /
User-agent: vortex|\
Disallow: /
User-agent: voyager|\
Disallow: /
User-agent: vspider|\
Disallow: /
User-agent: w3c\-webcon|\
Disallow: /
User-agent: w3c\_unicorn|\
Disallow: /
User-agent: w3search|\
Disallow: /
User-agent: wacbot|\
Disallow: /
User-agent: wanadoo|\
Disallow: /
User-agent: wastrix|\
Disallow: /
User-agent: water\ conserve\ portal|\
Disallow: /
User-agent: water\ conserve\ spider|\
Disallow: /
User-agent: watzbot|\
Disallow: /
User-agent: wauuu|\
Disallow: /
User-agent: wavefire|\
Disallow: /
User-agent: waypath|\
Disallow: /
User-agent: wazzup|\
Disallow: /
User-agent: wbdbot|\
Disallow: /
User-agent: web\ ceo\ online\ robot|\
Disallow: /
User-agent: web\ crawler|\
Disallow: /
User-agent: web\ downloader|\
Disallow: /
User-agent: web\ image\ collector|\
Disallow: /
User-agent: web\ link\ validator|\
Disallow: /
User-agent: web\ magnet|\
Disallow: /
User-agent: web\ site\ downloader|\
Disallow: /
User-agent: web\ sucker|\
Disallow: /
User-agent: web\-agent|\
Disallow: /
User-agent: web\-sniffer|\
Disallow: /
User-agent: web\.image\.collector|\
Disallow: /
User-agent: webaltbot|\
Disallow: /
User-agent: webauto|\
Disallow: /
User-agent: webbot|\
Disallow: /
User-agent: webbul\-bot|\
Disallow: /
User-agent: webcapture|\
Disallow: /
User-agent: webcheck|\
Disallow: /
User-agent: webclipping\.com|\
Disallow: /
User-agent: webcollage|\
Disallow: /
User-agent: webcopier|\
Disallow: /
User-agent: webcopy|\
Disallow: /
User-agent: webcorp|\
Disallow: /
User-agent: webcrawl\.net|\
Disallow: /
User-agent: webcrawler|\
Disallow: /
User-agent: webdatacentrebot|\
Disallow: /
User-agent: webdownloader\ for\ x|\
Disallow: /
User-agent: webdup|\
Disallow: /
User-agent: webemailextrac|\
Disallow: /
User-agent: webenhancer|\
Disallow: /
User-agent: webfetch|\
Disallow: /
User-agent: webgather|\
Disallow: /
User-agent: webgo\ is|\
Disallow: /
User-agent: webgobbler|\
Disallow: /
User-agent: webimages|\
Disallow: /
User-agent: webinator\-search2|\
Disallow: /
User-agent: webinator\-wbi|\
Disallow: /
User-agent: webindex|\
Disallow: /
User-agent: weblayers|\
Disallow: /
User-agent: webleacher|\
Disallow: /
User-agent: weblexbot|\
Disallow: /
User-agent: weblinker|\
Disallow: /
User-agent: weblyzard|\
Disallow: /
User-agent: webmastercoffee|\
Disallow: /
User-agent: webmasterworld\ extractor|\
Disallow: /
User-agent: webmasterworldforumbot|\
Disallow: /
User-agent: webminer|\
Disallow: /
User-agent: webmoose|\
Disallow: /
User-agent: webot|\
Disallow: /
User-agent: webpix|\
Disallow: /
User-agent: webreaper|\
Disallow: /
User-agent: webripper|\
Disallow: /
User-agent: websauger|\
Disallow: /
User-agent: webscan|\
Disallow: /
User-agent: websearchbench|\
Disallow: /
User-agent: website|\
Disallow: /
User-agent: webspear|\
Disallow: /
User-agent: websphinx|\
Disallow: /
User-agent: webspider|\
Disallow: /
User-agent: webster|\
Disallow: /
User-agent: webstripper|\
Disallow: /
User-agent: webtrafficexpress|\
Disallow: /
User-agent: webtrends\ link\ analyzer|\
Disallow: /
User-agent: webvac|\
Disallow: /
User-agent: webwalk|\
Disallow: /
User-agent: webwasher|\
Disallow: /
User-agent: webwatch|\
Disallow: /
User-agent: webwhacker|\
Disallow: /
User-agent: webxm|\
Disallow: /
User-agent: webzip|\
Disallow: /
User-agent: weddings\.info|\
Disallow: /
User-agent: wenbin|\
Disallow: /
User-agent: wep\ search|\
Disallow: /
User-agent: wepa|\
Disallow: /
User-agent: werelatebot|\
Disallow: /
User-agent: wget|\
Disallow: /
User-agent: whacker|\
Disallow: /
User-agent: whirlpool\ web\ engine|\
Disallow: /
User-agent: whowhere\ robot|\
Disallow: /
User-agent: widow|\
Disallow: /
User-agent: wikiabot|\
Disallow: /
User-agent: wikio|\
Disallow: /
User-agent: wikiwix\-bot\-|\
Disallow: /
User-agent: winhttp|\
Disallow: /
User-agent: wire|\
Disallow: /
User-agent: wisebot|\
Disallow: /
User-agent: wisenutbot|\
Disallow: /
User-agent: wish\-la|\
Disallow: /
User-agent: wish\-project|\
Disallow: /
User-agent: wisponbot|\
Disallow: /
User-agent: wmcai\-robot|\
Disallow: /
User-agent: wminer|\
Disallow: /
User-agent: wmsbot|\
Disallow: /
User-agent: woriobot|\
Disallow: /
User-agent: worldshop|\
Disallow: /
User-agent: worqmada|\
Disallow: /
User-agent: wotbox|\
Disallow: /
User-agent: wume\_crawler|\
Disallow: /
User-agent: www\ collector|\
Disallow: /
User-agent: www\-collector\-e|\
Disallow: /
User-agent: www\-mechanize|\
Disallow: /
User-agent: wwwoffle|\
Disallow: /
User-agent: wwwrobot|\
Disallow: /
User-agent: wwwster|\
Disallow: /
User-agent: wwwwanderer|\
Disallow: /
User-agent: wwwxref|\
Disallow: /
User-agent: wysigot|\
Disallow: /
User-agent: x\-clawler|\
Disallow: /
User-agent: x\-crawler|\
Disallow: /
User-agent: xaldon|\
Disallow: /
User-agent: xenu|\
Disallow: /
User-agent: xerka\ metabot|\
Disallow: /
User-agent: xerka\ webbot|\
Disallow: /
User-agent: xget|\
Disallow: /
User-agent: xirq|\
Disallow: /
User-agent: xmarksfetch|\
Disallow: /
User-agent: xqrobot|\
Disallow: /
User-agent: y!j|\
Disallow: /
User-agent: yacy\.net|\
Disallow: /
User-agent: yacybot|\
Disallow: /
User-agent: yanga\ worldsearch\ bot|\
Disallow: /
User-agent: yarienavoir\.net|\
Disallow: /
User-agent: yasaklibot|\
Disallow: /
User-agent: yats\ crawler|\
Disallow: /
User-agent: ybot|\
Disallow: /
User-agent: yebolbot|\
Disallow: /
User-agent: yellowjacket|\
Disallow: /
User-agent: yeti|\
Disallow: /
User-agent: yolinkbot|\
Disallow: /
User-agent: yooglifetchagent|\
Disallow: /
User-agent: yoono|\
Disallow: /
User-agent: yottacars\_bot|\
Disallow: /
User-agent: yourls|\
Disallow: /
User-agent: z\-add\ link\ checker|\
Disallow: /
User-agent: zagrebin|\
Disallow: /
User-agent: zao|\
Disallow: /
User-agent: zedzo\.validate|\
Disallow: /
User-agent: zermelo|\
Disallow: /
User-agent: zeus|\
Disallow: /
User-agent: zibber\-v|\
Disallow: /
User-agent: zimeno|\
Disallow: /
User-agent: zing-bottabot|\
Disallow: /
User-agent: zipppbot|\
Disallow: /
User-agent: zongbot|\
Disallow: /
User-agent: zoomspider|\
Disallow: /
User-agent: zotag\ search|\
Disallow: /
User-agent: zsebot|\
Disallow: /
User-agent: zuibot|\
Disallow: /
User-agent: zyborg|\
Disallow: /
User-agent: zyte\
Disallow: /
User-agent: *
Crawl-delay: 999.00
Disallow: /banners
Disallow: /cgi-bin
Disallow: /conanrpg
Disallow: /forum_comics
Disallow: /forum_old
Disallow: /gallery
Disallow: /l4d
Disallow: /map-
Disallow: /map/JQuery
Disallow: /map/leaflet
Disallow: /map/markers
Disallow: /map/tiles
Disallow: /map2
Disallow: /misc
Disallow: /murmur
Disallow: /mysqldumper
Disallow: /ventsrv
Disallow: /*.gif$
Disallow: /*.Gif$
Disallow: /*.GIF$
Disallow: /*.jpg$
Disallow: /*.Jpg$
Disallow: /*.JPG$
Disallow: /*.jpeg$
Disallow: /*.Jpeg$
Disallow: /*.JPEG$
Disallow: /*.pdf$
Disallow: /*.Pdf$
Disallow: /*.PDF$
Disallow: /*.png$
Disallow: /*.zip$
Disallow: /*.Zip$
Disallow: /*.ZIP$
Россия - 82.179.246.9
State Institute of Information Technologies and
University Telecommunications Ltd.
HTTP/1.1 200 OK
Date: Sun, 26 May 2019 04:05:25 GMT
Server: Apache
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-control: private, max-age=0
X-Frame-Options: SAMEORIGIN
Set-Cookie: xf_session=f9a4acffade0ed5c225098ea1e757055; path=/; httponly
Last-Modified: Sun, 26 May 2019 04:05:25 GMT
Content-Length: 40279
Vary: Accept-Encoding
Content-Type: text/html; charset=UTF-8
Кнопка для анализа сайта в один клик, для установки перетащите ссылку на "Панель закладок"