PRLOG

Title

SPAR SI

Keywords

n/a

Description

n/a

Кодировка

UTF-8

Размер страницы

655.96 КБ

Количество слов

2 500

Длина текста на странице

27 793 симв.

Длина текста без пробелов

23 766 симв.

<H1>

<H2>

<H3>

• Restavracija Interspar
• Odkrijte mlečni pravljici na naših policah
• Vedno popoln žar

Анализ текста

• 3.40% ( 85 ) spar
• 1.40% ( 35 ) solato
• 0.92% ( 23 ) plus
• 0.72% ( 18 ) pečen
• 0.64% ( 16 ) kartice
• 0.64% ( 16 ) nasvet
• 0.64% ( 16 ) zelenjava
• 0.60% ( 15 ) juha
• 0.60% ( 15 ) omaki
• 0.60% ( 15 ) sadje
• 0.60% ( 15 ) veganz
• 0.52% ( 13 ) izdelki
• 0.52% ( 13 ) lončku
• 0.48% ( 12 ) file
• 0.48% ( 12 ) kmetija
• 0.48% ( 12 ) paradižnikov
• 0.48% ( 12 ) recept
• 0.44% ( 11 ) buče
• 0.44% ( 11 ) telečja
• 0.40% ( 10 ) brez

Данные предоставлены сервисом semrush

Сайт	Общие фразы	PR	тИЦ	Alexa Rank	Alexa Country
youtube.com	2	9	0	2	2
tchibo.com	2	5	10	321480	37628
spar.ie	2	5	0	1361965	Нет данных
mojazaposlitev.si	2	5	0	139096	164
lidl.si	2	5	10	159520	248
krka.si	2	6	80	808517	1928
eduscho.at	2	6	0	54933	235
dm-drogeriemarkt.si	1	4	10	597545	1370
delo.si	1	7	50	29182	29
amazon.com	1	8	0	10	4
Еще 40 сайтов после авторизации

Данные предоставлены сервисом semrush

domain: spar.si
registrar: Softnet d.o.o.
registrar-url: http://www.softnet.si/sl/Registracija_domen/
nameserver: woody.softnet.si (212.103.128.66,2a02:800:0:3::66)
nameserver: buzz.softnet.si (212.103.128.67,2a02:800:0:3::67)
registrant: G32918
status: ok
created: 1999-02-19
expire: 2016-06-06
source: ARNES

Domain holder:
NOT DISCLOSED

Tech:
NOT DISCLOSED

# For all robots
User-agent: *

# Block access to specific groups of pages
Disallow: /online/cart
Disallow: /online/checkout
Disallow: /online/my-account

Request-rate: 1/10 # maximum rate is one page every 10 seconds
Crawl-delay: 10 # 10 seconds between page requests
Visit-time: 0400-0845 # only visit between 04:00 and 08:45 UTC

# Allow search crawlers to discover the sitemap
Sitemap: /online/sitemap.xml

# Block CazoodleBot as it does not present correct accept content headers
User-agent: CazoodleBot
Disallow: /

# Block MJ12bot as it is just noise
User-agent: MJ12bot
Disallow: /

# Block dotbot as it cannot parse base urls properly
User-agent: dotbot/1.0
Disallow: /

# Block Gigabot
User-agent: Gigabot
Disallow: /

# Block MegaIndex.ru
User-agent: MegaIndex.ru/2.0
Disallow: /

# Block MegaIndex
User-agent: MegaIndex.ru
Disallow: /

User-agent: megaindex.com
Disallow: /

# Block others
User-agent: AboutUsBot
Disallow: /
User-agent: Amfibibot
Disallow: /
User-agent: antibot
Disallow: /
User-agent: Aqua_Products
Disallow: /
User-agent: asterias
Disallow: /
User-agent: b2w/0.1
Disallow: /
User-agent: BackDoorBot
Disallow: /
User-agent: BackDoorBot/1.0
Disallow: /
User-agent: BecomeBot
Disallow: /
User-agent: BlowFish
Disallow: /
User-agent: BlowFish/1.0
Disallow: /
User-agent: Bookmark search tool
Disallow: /
User-agent: BotALot
Disallow: /
User-agent: BruinBot
Disallow: /
User-agent: btbot
Disallow: /
User-agent: BuiltBotTough
Disallow: /
User-agent: Bullseye
Disallow: /
User-agent: Bullseye/1.0
Disallow: /
User-agent: BunnySlippers
Disallow: /
User-agent: cfetch
Disallow: /
User-agent: CheeseBot
Disallow: /
User-agent: CherryPicker
Disallow: /
User-agent: CherryPickerElite/1.0
Disallow: /
User-agent: CherryPickerSE/1.0
Disallow: /
User-agent: CoolBot
Disallow: /
User-agent: CopyRightCheck
Disallow: /
User-agent: cosmos
Disallow: /
User-agent: Cowbot
Disallow: /
User-agent: Crescent
Disallow: /
User-agent: CydralSpider
Disallow: /
User-agent: Dumbot
Disallow: /
User-agent: EmailCollector
Disallow: /
User-agent: EmailSiphon
Disallow: /
User-agent: EmailWolf
Disallow: /
User-agent: EroCrawler
Disallow: /
User-agent: Exabot
Disallow: /
User-agent: ExtractorPro
Disallow: /
User-agent: FairAd Client
Disallow: /
User-agent: Fetch
Disallow: /
User-agent: FindLinks
Disallow: /
User-agent: Flaming AttackBot
Disallow: /
User-agent: Foobot
Disallow: /
User-agent: Gaisbot
Disallow: /
User-agent: GetRight
Disallow: /
User-agent: GetRight/4.2
Disallow: /
User-agent: grub
Disallow: /
User-agent: grub-client
Disallow: /
User-agent: Harvest
Disallow: /
User-agent: Harvest/1.5
Disallow: /
User-agent: HenryTheMiragoRobot
Disallow: /
User-agent: Heritrix
Disallow: /
User-agent: hloader
Disallow: /
User-agent: HooWWWer
Disallow: /
User-agent: httplib
Disallow: /
User-agent: HTTrack
Disallow: /
User-agent: humanlinks
Disallow: /
User-agent: InfoNaviRobot
Disallow: /
User-agent: IRLbot
Disallow: /
User-agent: Iron33
Disallow: /
User-agent: Iron33/1.0.2
Disallow: /
User-agent: JemmaTheTourist
Disallow: /
User-agent: JennyBot
Disallow: /
User-agent: Jetbot
Disallow: /
User-agent: Jetbot/1.0
Disallow: /
User-agent: Kenjin Spider
Disallow: /
User-agent: Lachesis
Disallow: /
User-agent: larbin
Disallow: /
User-agent: LexiBot
Disallow: /
User-agent: LinkextractorPro
Disallow: /
User-agent: LinkWalker
Disallow: /
User-agent: LNSpiderguy
Disallow: /
User-agent: LocalcomBot
Disallow: /
User-agent: Mata Hari
Disallow: /
User-agent: Mister PiX
Disallow: /
User-agent: MMCrawler
Disallow: /
User-agent: moget
Disallow: /
User-agent: moget*
Disallow: /
User-agent: moget/2.1
Disallow: /
User-agent: NaverBot
Disallow: /
User-agent: NetAnts
Disallow: /
User-agent: NetMechanic
Disallow: /
User-agent: NICErsPRO
Disallow: /
User-agent: NimbleCrawler
Disallow: /
User-agent: NPBot
Disallow: /
User-agent: Nutch
Disallow: /
User-agent: ObjectsSearch
Disallow: /
User-agent: Offline Explorer
Disallow: /
User-agent: Openbot
Disallow: /
User-agent: Openfind
Disallow: /
User-agent: Oracle Ultra Search
Disallow: /
User-agent: penthesilea*
Disallow: /
User-agent: PerMan
Disallow: /
User-agent: PhpDig*
Disallow: /
User-agent: ProPowerBot
Disallow: /
User-agent: ProPowerBot/2.14
Disallow: /
User-agent: ProWebWalker
Disallow: /
User-agent: psbot
Disallow: /
User-agent: Python-urllib
Disallow: /
User-agent: QuepasaCreep
Disallow: /
User-agent: QueryN Metasearch
Disallow: /
User-agent: Radiation Retriever
Disallow: /
User-agent: Radiation Retriever 1.1
Disallow: /
User-agent: RepoMonkey
Disallow: /
User-agent: RepoMonkey Bait & Tackle/v1.01
Disallow: /
User-agent: RepoMonkey*
Disallow: /
User-agent: SBIder
Disallow: /
User-agent: searchpreview
Disallow: /
User-agent: Searchspider
Disallow: /
User-agent: Shim-Crawler
Disallow: /
User-agent: SiteSnagger
Disallow: /
User-agent: The Intraformant
Disallow: /
User-agent: TheNomad
Disallow: /
User-agent: thesubot
Disallow: /
User-agent: thumbshots-de-bot
Disallow: /
User-agent: toCrawl/UrlDispatcher
Disallow: /
User-agent: True_Robot
Disallow: /
User-agent: True_Robot/1.0
Disallow: /
User-agent: turingos
Disallow: /
User-agent: Ultraseek
Disallow: /
User-agent: WebCapture
Disallow: /
User-agent: WebCopier
Disallow: /
User-agent: WebEnhancer
Disallow: /
User-agent: WebmasterWorldForumBot
Disallow: /
User-agent: WebSauger
Disallow: /
User-agent: Webster Pro
Disallow: /
User-agent: WebStripper
Disallow: /
User-agent: Webwhacker
Disallow: /
User-agent: 12soso
Disallow: /
User-agent: 192.comagent
Disallow: /
User-agent: 1noonbot
Disallow: /
User-agent: 1on1searchbot
Disallow: /
User-agent: 3de_search2
Disallow: /
User-agent: 3d_search
Disallow: /
User-agent: 3g bot
Disallow: /
User-agent: 3gse
Disallow: /
User-agent: 50.nu
Disallow: /
User-agent: a1 sitemap generator
Disallow: /
User-agent: a1 website download
Disallow: /
User-agent: a6-indexer
Disallow: /
User-agent: aasp
Disallow: /
User-agent: abachobot
Disallow: /
User-agent: abonti
Disallow: /
User-agent: abotemailsearch
Disallow: /
User-agent: aboundex
Disallow: /
User-agent: aboutusbot
Disallow: /
User-agent: accmonitor compliance server
Disallow: /
User-agent: accoon
Disallow: /
User-agent: achulkov.net page walker
Disallow: /
User-agent: acme.spider
Disallow: /
User-agent: acoonbot
Disallow: /
User-agent: acquia-crawler
Disallow: /
User-agent: activetouristbot
Disallow: /
User-agent: ad muncher
Disallow: /
User-agent: adamm bot
Disallow: /
User-agent: adbeat_bot
Disallow: /
User-agent: adminshop.com
Disallow: /
User-agent: advanced email extractor
Disallow: /
User-agent: aesop_com_spiderman
Disallow: /
User-agent: aespider
Disallow: /
User-agent: af knowledge now verity spider
Disallow: /
User-agent: aggregator:vocus
Disallow: /
User-agent: ah-ha.com crawler
Disallow: /
User-agent: ahrefs
Disallow: /
User-agent: aibot
Disallow: /
User-agent: aidu
Disallow: /
User-agent: aihitbot
Disallow: /
User-agent: aipbot
Disallow: /
User-agent: aisiid
Disallow: /
User-agent: aitcsrobot/1.1
Disallow: /
User-agent: ajsitemap
Disallow: /
User-agent: akamai-sitesnapshot
Disallow: /
User-agent: alexawebsearchplatform
Disallow: /
User-agent: alexfdownload
Disallow: /
User-agent: alexibot
Disallow: /
User-agent: alkalinebot
Disallow: /
User-agent: all acronyms bot
Disallow: /
User-agent: alpha search agent
Disallow: /
User-agent: amerla search bot
Disallow: /
User-agent: amfibibot
Disallow: /
User-agent: ampmppc.com
Disallow: /
User-agent: amznkassocbot
Disallow: /
User-agent: anemone
Disallow: /
User-agent: anonymous
Disallow: /
User-agent: anotherbot
Disallow: /
User-agent: answerbot
Disallow: /
User-agent: answerbus
Disallow: /
User-agent: answerchase prove
Disallow: /
User-agent: antbot
Disallow: /
User-agent: antibot
Disallow: /
User-agent: antisantyworm
Disallow: /
User-agent: antro.net
Disallow: /
User-agent: aonde-spider
Disallow: /
User-agent: aport
Disallow: /
User-agent: appengine-google
Disallow: /
User-agent: appid: s~stremor-crawler-
Disallow: /
User-agent: aqua_products
Disallow: /
User-agent: arabot
Disallow: /
User-agent: arachmo
Disallow: /
User-agent: arachnophilia
Disallow: /
User-agent: archive.org_bot
Disallow: /
User-agent: aria equalizer
Disallow: /
User-agent: arianna.libero.it
Disallow: /
User-agent: arikus_spider
Disallow: /
User-agent: art-online.com
Disallow: /
User-agent: artavisbot
Disallow: /
User-agent: artera
Disallow: /
User-agent: asaha search engine turkey
Disallow: /
User-agent: ask
Disallow: /
User-agent: aspider
Disallow: /
User-agent: aspseek
Disallow: /
User-agent: asterias
Disallow: /
User-agent: astrofind
Disallow: /
User-agent: athenusbot
Disallow: /
User-agent: atlocalbot
Disallow: /
User-agent: atomic_email_hunter
Disallow: /
User-agent: attach
Disallow: /
User-agent: attrakt
Disallow: /
User-agent: attributor
Disallow: /
User-agent: augurfind
Disallow: /
User-agent: auresys
Disallow: /
User-agent: autobaron crawler
Disallow: /
User-agent: autoemailspider
Disallow: /
User-agent: autowebdir
Disallow: /
User-agent: avsearch-
Disallow: /
User-agent: axfeedsbot
Disallow: /
User-agent: axonize-bot
Disallow: /
User-agent: ayna
Disallow: /
User-agent: b2w
Disallow: /
User-agent: backdoorbot
Disallow: /
User-agent: backrub
Disallow: /
User-agent: backstreet browser
Disallow: /
User-agent: backweb
Disallow: /
User-agent: baidu
Disallow: /
User-agent: bandit
Disallow: /
User-agent: batchftp
Disallow: /
User-agent: baypup
Disallow: /
User-agent: bdfetch
Disallow: /
User-agent: becomebot
Disallow: /
User-agent: becomejpbot
Disallow: /
User-agent: beetlebot
Disallow: /
User-agent: bender
Disallow: /
User-agent: besserscheitern-crawl
Disallow: /
User-agent: betabot
Disallow: /
User-agent: big brother
Disallow: /
User-agent: big data
Disallow: /
User-agent: bigado.com
Disallow: /
User-agent: bigcliquebot
Disallow: /
User-agent: bigfoot
Disallow: /
User-agent: biglotron
Disallow: /
User-agent: bilbo
Disallow: /
User-agent: bilgibetabot
Disallow: /
User-agent: bilgibot
Disallow: /
User-agent: bintellibot
Disallow: /
User-agent: bitlybot
Disallow: /
User-agent: bitvouseragent
Disallow: /
User-agent: bizbot003
Disallow: /
User-agent: bizbot04
Disallow: /
User-agent: bizworks retriever
Disallow: /
User-agent: black hole
Disallow: /
User-agent: black.hole
Disallow: /
User-agent: blackbird
Disallow: /
User-agent: blackmask.net search engine
Disallow: /
User-agent: blackwidow
Disallow: /
User-agent: bladder fusion
Disallow: /
User-agent: blaiz-bee
Disallow: /
User-agent: blexbot
Disallow: /
User-agent: blinkx
Disallow: /
User-agent: blitzbot
Disallow: /
User-agent: blog conversation project
Disallow: /
User-agent: blogmyway
Disallow: /
User-agent: blogpulselive
Disallow: /
User-agent: blogrefsbot
Disallow: /
User-agent: blogscope
Disallow: /
User-agent: blogslive
Disallow: /
User-agent: bloobybot
Disallow: /
User-agent: blowfish
Disallow: /
User-agent: blt
Disallow: /
User-agent: bnf.fr_bot
Disallow: /
User-agent: boaconstrictor
Disallow: /
User-agent: boardreader
Disallow: /
User-agent: boia-scan-agent
Disallow: /
User-agent: boia.org
Disallow: /
User-agent: boitho
Disallow: /
User-agent: boi_crawl_00
Disallow: /
User-agent: bookmark buddy bookmark checker
Disallow: /
User-agent: bookmark search tool
Disallow: /
User-agent: bosug
Disallow: /
User-agent: bot apoena
Disallow: /
User-agent: botalot
Disallow: /
User-agent: botrighthere
Disallow: /
User-agent: botswana
Disallow: /
User-agent: bottybot
Disallow: /
User-agent: bpbot
Disallow: /
User-agent: braintime_search
Disallow: /
User-agent: brokenlinkcheck.com
Disallow: /
User-agent: browseremulator
Disallow: /
User-agent: browsermob
Disallow: /
User-agent: bruinbot
Disallow: /
User-agent: bsearchr&d
Disallow: /
User-agent: bspider
Disallow: /
User-agent: btbot
Disallow: /
User-agent: btsearch
Disallow: /
User-agent: bubing
Disallow: /
User-agent: buddy
Disallow: /
User-agent: buibui
Disallow: /
User-agent: buildcms crawler
Disallow: /
User-agent: builtbottough
Disallow: /
User-agent: bullseye
Disallow: /
User-agent: bumblebee
Disallow: /
User-agent: bunnyslippers
Disallow: /
User-agent: buscadorclarin
Disallow: /
User-agent: buscaplus robi
Disallow: /
User-agent: butterfly
Disallow: /
User-agent: buyhawaiibot
Disallow: /
User-agent: buzzbot
Disallow: /
User-agent: byindia
Disallow: /
User-agent: byspider
Disallow: /
User-agent: byteserver
Disallow: /
User-agent: bzbot
Disallow: /
User-agent: c r a w l 3 r
Disallow: /
User-agent: cacheblaster
Disallow: /
User-agent: caddbot
Disallow: /
User-agent: cafi
Disallow: /
User-agent: camcrawler
Disallow: /
User-agent: camelstampede
Disallow: /
User-agent: canon-webrecord
Disallow: /
User-agent: careerbot
Disallow: /
User-agent: cataguru
Disallow: /
User-agent: catchbot
Disallow: /
User-agent: cazoodle
Disallow: /
User-agent: ccbot
Disallow: /
User-agent: ccgcrawl
Disallow: /
User-agent: ccubee
Disallow: /
User-agent: cd-preload
Disallow: /
User-agent: ce-preload
Disallow: /
User-agent: cegbfeieh
Disallow: /
User-agent: cerberian drtrs
Disallow: /
User-agent: cert figleafbot
Disallow: /
User-agent: cfetch
Disallow: /
User-agent: cfnetwork
Disallow: /
User-agent: chameleon
Disallow: /
User-agent: charlotte
Disallow: /
User-agent: check&get
Disallow: /
User-agent: checkbot
Disallow: /
User-agent: checklinks
Disallow: /
User-agent: cheesebot
Disallow: /
User-agent: chemiede-nodebot
Disallow: /
User-agent: cherrypicker
Disallow: /
User-agent: chilkat
Disallow: /
User-agent: chinaclaw
Disallow: /
User-agent: cipinetbot
Disallow: /
User-agent: cis455crawler
Disallow: /
User-agent: citeseerxbot
Disallow: /
User-agent: cizilla
Disallow: /
User-agent: clariabot
Disallow: /
User-agent: climate ark
Disallow: /
User-agent: climateark spider
Disallow: /
User-agent: clshttp
Disallow: /
User-agent: clushbot
Disallow: /
User-agent: coast scan engine
Disallow: /
User-agent: coast webmaster pro
Disallow: /
User-agent: coccoc
Disallow: /
User-agent: collapsarweb
Disallow: /
User-agent: collector
Disallow: /
User-agent: colocrossing
Disallow: /
User-agent: combine
Disallow: /
User-agent: connectsearch
Disallow: /
User-agent: conpilot
Disallow: /
User-agent: contentsmartz
Disallow: /
User-agent: contextad bot
Disallow: /
User-agent: contype
Disallow: /
User-agent: cookienet
Disallow: /
User-agent: coolbot
Disallow: /
User-agent: coolcheck
Disallow: /
User-agent: copernic
Disallow: /
User-agent: copier
Disallow: /
User-agent: copyrightcheck
Disallow: /
User-agent: core-project
Disallow: /
User-agent: cosmos
Disallow: /
User-agent: covario-ids
Disallow: /
User-agent: cowbot-
Disallow: /
User-agent: cowdog bot
Disallow: /
User-agent: crabbybot
Disallow: /
User-agent: craftbot@yahoo.com
Disallow: /
User-agent: crawler.kpricorn.org
Disallow: /
User-agent: crawler43.ejupiter.com
Disallow: /
User-agent: crawler4j
Disallow: /
User-agent: crawler@
Disallow: /
User-agent: crawler_for_infomine
Disallow: /
User-agent: crawly
Disallow: /
User-agent: crawl_application
Disallow: /
User-agent: creativecommons
Disallow: /
User-agent: crescent
Disallow: /
User-agent: cs-crawler
Disallow: /
User-agent: cse html validator
Disallow: /
User-agent: cshttpclient
Disallow: /
User-agent: cuasarbot
Disallow: /
User-agent: culsearch
Disallow: /
User-agent: curl
Disallow: /
User-agent: custo
Disallow: /
User-agent: cvaulev
Disallow: /
User-agent: cyberdog
Disallow: /
User-agent: cybernavi_webget
Disallow: /
User-agent: cyberpatrol sitecat webbot
Disallow: /
User-agent: cyberspyder
Disallow: /
User-agent: cydralspider
Disallow: /
User-agent: d1garabicengine
Disallow: /
User-agent: datacha0s
Disallow: /
User-agent: datafountains
Disallow: /
User-agent: dataparksearch
Disallow: /
User-agent: dataprovider.com
Disallow: /
User-agent: datascape robot
Disallow: /
User-agent: dataspearspiderbot
Disallow: /
User-agent: dataspider
Disallow: /
User-agent: dattatec.com
Disallow: /
User-agent: daumoa
Disallow: /
User-agent: dblbot
Disallow: /
User-agent: dcpbot
Disallow: /
User-agent: declumbot
Disallow: /
User-agent: deepindex
Disallow: /
User-agent: deepnet crawler
Disallow: /
User-agent: deeptrawl
Disallow: /
User-agent: dejan
Disallow: /
User-agent: del.icio.us-thumbnails
Disallow: /
User-agent: deltascan
Disallow: /
User-agent: delvubot
Disallow: /
User-agent: der gro§e bildersauger
Disallow: /
User-agent: der große bildersauger
Disallow: /
User-agent: deusu
Disallow: /
User-agent: dfs-fetch
Disallow: /
User-agent: diagem
Disallow: /
User-agent: diamond
Disallow: /
User-agent: dibot
Disallow: /
User-agent: didaxusbot
Disallow: /
User-agent: digext
Disallow: /
User-agent: digger
Disallow: /
User-agent: digi-rssbot
Disallow: /
User-agent: digitalarchivesbot
Disallow: /
User-agent: digout4u
Disallow: /
User-agent: diibot
Disallow: /
User-agent: dillo
Disallow: /
User-agent: dir_snatch.exe
Disallow: /
User-agent: disco
Disallow: /
User-agent: distilled-reputation-monitor
Disallow: /
User-agent: djangotraineebot
Disallow: /
User-agent: dkimrepbot
Disallow: /
User-agent: dmoz downloader
Disallow: /
User-agent: docomo
Disallow: /
User-agent: dof-verify
Disallow: /
User-agent: domaincrawler
Disallow: /
User-agent: domainscan
Disallow: /
User-agent: domainwatcher bot
Disallow: /
User-agent: dotbot
Disallow: /
User-agent: dotspotsbot
Disallow: /
User-agent: dow jones searchbot
Disallow: /
User-agent: download
Disallow: /
User-agent: doy
Disallow: /
User-agent: dragonfly
Disallow: /
User-agent: drip
Disallow: /
User-agent: drone
Disallow: /
User-agent: dtaagent
Disallow: /
User-agent: dtsearchspider
Disallow: /
User-agent: dumbot
Disallow: /
User-agent: dwaar
Disallow: /
User-agent: dxseeker
Disallow: /
User-agent: e-societyrobot
Disallow: /
User-agent: eah
Disallow: /
User-agent: earth platform indexer
Disallow: /
User-agent: earth science educator robot
Disallow: /
User-agent: easydl
Disallow: /
User-agent: ebingbong
Disallow: /
User-agent: ec2linkfinder
Disallow: /
User-agent: ecairn-grabber
Disallow: /
User-agent: ecatch
Disallow: /
User-agent: echoosebot
Disallow: /
User-agent: edisterbot
Disallow: /
User-agent: edugovsearch
Disallow: /
User-agent: egothor
Disallow: /
User-agent: eidetica.com
Disallow: /
User-agent: eirgrabber
Disallow: /
User-agent: elblindo the blind bot
Disallow: /
User-agent: elisabot
Disallow: /
User-agent: ellerdalebot
Disallow: /
User-agent: email exractor
Disallow: /
User-agent: emailcollector
Disallow: /
User-agent: emailleach
Disallow: /
User-agent: emailsiphon
Disallow: /
User-agent: emailwolf
Disallow: /
User-agent: emeraldshield
Disallow: /
User-agent: empas_robot
Disallow: /
User-agent: enabot
Disallow: /
User-agent: endeca
Disallow: /
User-agent: enigmabot
Disallow: /
User-agent: enswer neuro bot
Disallow: /
User-agent: enter user-agent
Disallow: /
User-agent: entitycubebot
Disallow: /
User-agent: erocrawler
Disallow: /
User-agent: estylesearch
Disallow: /
User-agent: esyndicat bot
Disallow: /
User-agent: eurosoft-bot
Disallow: /
User-agent: evaal
Disallow: /
User-agent: eventware
Disallow: /
User-agent: everest-vulcan inc.
Disallow: /
User-agent: exabot
Disallow: /
User-agent: exactsearch
Disallow: /
User-agent: exactseek
Disallow: /
User-agent: exooba
Disallow: /
User-agent: exploder
Disallow: /
User-agent: express webpictures
Disallow: /
User-agent: extractor
Disallow: /
User-agent: eyenetie
Disallow: /
User-agent: ez-robot
Disallow: /
User-agent: ezooms
Disallow: /
User-agent: f-bot test pilot
Disallow: /
User-agent: factbot
Disallow: /
User-agent: fairad client
Disallow: /
User-agent: falcon
Disallow: /
User-agent: fast data search document retriever
Disallow: /
User-agent: fast esp
Disallow: /
User-agent: fast-search-engine
Disallow: /
User-agent: fastbot crawler
Disallow: /
User-agent: fastbot.de crawler
Disallow: /
User-agent: fatbot
Disallow: /
User-agent: favcollector
Disallow: /
User-agent: faviconizer
Disallow: /
User-agent: favorites sweeper
Disallow: /
User-agent: fdm
Disallow: /
User-agent: fdse robot
Disallow: /
User-agent: fedcontractorbot
Disallow: /
User-agent: fembot
Disallow: /
User-agent: fetch api request
Disallow: /
User-agent: fetch_ici
Disallow: /
User-agent: fgcrawler
Disallow: /
User-agent: filangy
Disallow: /
User-agent: filehound
Disallow: /
User-agent: findanisp.com_isp_finder
Disallow: /
User-agent: findlinks
Disallow: /
User-agent: findweb
Disallow: /
User-agent: firebat
Disallow: /
User-agent: firstgov.gov search
Disallow: /
User-agent: flaming attackbot
Disallow: /
User-agent: flamingo_searchengine
Disallow: /
User-agent: flashcapture
Disallow: /
User-agent: flashget
Disallow: /
User-agent: flickysearchbot
Disallow: /
User-agent: fluffy the spider
Disallow: /
User-agent: flunky
Disallow: /
User-agent: focused_crawler
Disallow: /
User-agent: followsite
Disallow: /
User-agent: foobot
Disallow: /
User-agent: fooooo_web_video_crawl
Disallow: /
User-agent: fopper
Disallow: /
User-agent: formulafinderbot
Disallow: /
User-agent: forschungsportal
Disallow: /
User-agent: francis
Disallow: /
User-agent: freewebmonitoring sitechecker
Disallow: /
User-agent: freshcrawler
Disallow: /
User-agent: freshdownload
Disallow: /
User-agent: freshlinks.exe
Disallow: /
User-agent: friendfeedbot
Disallow: /
User-agent: frodo.at
Disallow: /
User-agent: froggle
Disallow: /
User-agent: frontpage
Disallow: /
User-agent: froola bot
Disallow: /
User-agent: fr_crawler
Disallow: /
User-agent: fu-nbi
Disallow: /
User-agent: full_breadth_crawler
Disallow: /
User-agent: funnelback
Disallow: /
User-agent: furlbot
Disallow: /
User-agent: g10-bot
Disallow: /
User-agent: gaisbot
Disallow: /
User-agent: galaxybot
Disallow: /
User-agent: gazz
Disallow: /
User-agent: gbplugin
Disallow: /
User-agent: generate_infomine_category_classifiers
Disallow: /
User-agent: genevabot
Disallow: /
User-agent: geniebot
Disallow: /
User-agent: genieo
Disallow: /
User-agent: geomaxenginebot
Disallow: /
User-agent: geometabot
Disallow: /
User-agent: geonabot
Disallow: /
User-agent: geovisu
Disallow: /
User-agent: germcrawler
Disallow: /
User-agent: gethtmlcontents
Disallow: /
User-agent: getleft
Disallow: /
User-agent: getright
Disallow: /
User-agent: getsmart
Disallow: /
User-agent: geturl.rexx
Disallow: /
User-agent: getweb!
Disallow: /
User-agent: giant
Disallow: /
User-agent: gigablastopensource
Disallow: /
User-agent: gigabot
Disallow: /
User-agent: girafabot
Disallow: /
User-agent: gleamebot
Disallow: /
User-agent: gnome-vfs
Disallow: /
User-agent: go!zilla
Disallow: /
User-agent: go-ahead-got-it
Disallow: /
User-agent: go-http-client
Disallow: /
User-agent: goforit.com
Disallow: /
User-agent: goforitbot
Disallow: /
User-agent: gold crawler
Disallow: /
User-agent: goldfire server
Disallow: /
User-agent: golem
Disallow: /
User-agent: goodjelly
Disallow: /
User-agent: gordon-college-google-mini
Disallow: /
User-agent: goroam
Disallow: /
User-agent: goseebot
Disallow: /
User-agent: gotit
Disallow: /
User-agent: govbot
Disallow: /
User-agent: gpu p2p crawler
Disallow: /
User-agent: grabber
Disallow: /
User-agent: grabnet
Disallow: /
User-agent: grafula
Disallow: /
User-agent: grapefx
Disallow: /
User-agent: grapeshot
Disallow: /
User-agent: grbot
Disallow: /
User-agent: greenyogi
Disallow: /
User-agent: gromit
Disallow: /
User-agent: grub
Disallow: /
User-agent: gsa
Disallow: /
User-agent: gslfbot
Disallow: /
User-agent: gulliver
Disallow: /
User-agent: gulperbot
Disallow: /
User-agent: gurujibot
Disallow: /
User-agent: gvc business crawler
Disallow: /
User-agent: gvc crawler
Disallow: /
User-agent: gvc search bot
Disallow: /
User-agent: gvc web crawler
Disallow: /
User-agent: gvc weblink crawler
Disallow: /
User-agent: gvc world links
Disallow: /
User-agent: gvcbot.com
Disallow: /
User-agent: happyfunbot
Disallow: /
User-agent: harvest
Disallow: /
User-agent: hatena antenna
Disallow: /
User-agent: hawler
Disallow: /
User-agent: hcat
Disallow: /
User-agent: hclsreport-crawler
Disallow: /
User-agent: hd nutch agent
Disallow: /
User-agent: header_test_client
Disallow: /
User-agent: healia
Disallow: /
User-agent: helix
Disallow: /
User-agent: here will be link to crawler site
Disallow: /
User-agent: heritrix
Disallow: /
User-agent: hiscan
Disallow: /
User-agent: hisoftware accmonitor server
Disallow: /
User-agent: hisoftware accverify
Disallow: /
User-agent: hitcrawler
Disallow: /
User-agent: hivabot
Disallow: /
User-agent: hloader
Disallow: /
User-agent: hmsebot
Disallow: /
User-agent: hmview
Disallow: /
User-agent: hoge
Disallow: /
User-agent: holmes
Disallow: /
User-agent: homepagesearch
Disallow: /
User-agent: hooblybot-image
Disallow: /
User-agent: hoowwwer
Disallow: /
User-agent: hostcrawler
Disallow: /
User-agent: hsft - link scanner
Disallow: /
User-agent: hsft - lvu scanner
Disallow: /
User-agent: hslide
Disallow: /
User-agent: ht://check
Disallow: /
User-agent: htdig
Disallow: /
User-agent: html link validator
Disallow: /
User-agent: htmlparser
Disallow: /
User-agent: httplib
Disallow: /
User-agent: httrack
Disallow: /
User-agent: huaweisymantecspider
Disallow: /
User-agent: hul-wax
Disallow: /
User-agent: humanlinks
Disallow: /
User-agent: hyperestraier
Disallow: /
User-agent: hyperix
Disallow: /
User-agent: iaarchiver-
Disallow: /
User-agent: ia_archiver
Disallow: /
User-agent: ibuena
Disallow: /
User-agent: icab
Disallow: /
User-agent: icds-ingestion
Disallow: /
User-agent: ichiro
Disallow: /
User-agent: icopyright conductor
Disallow: /
User-agent: ieautodiscovery
Disallow: /
User-agent: iecheck
Disallow: /
User-agent: ihwebchecker
Disallow: /
User-agent: iiitbot
Disallow: /
User-agent: iim_405
Disallow: /
User-agent: ilsebot
Disallow: /
User-agent: iltrovatore
Disallow: /
User-agent: image stripper
Disallow: /
User-agent: image sucker
Disallow: /
User-agent: image-fetcher
Disallow: /
User-agent: imagebot
Disallow: /
User-agent: imagefortress
Disallow: /
User-agent: imageshereimagesthereimageseverywhere
Disallow: /
User-agent: imagevisu
Disallow: /
User-agent: imds_monitor
Disallow: /
User-agent: imo-google-robot-intelink
Disallow: /
User-agent: inagist.com url crawler
Disallow: /
User-agent: indexer
Disallow: /
User-agent: industry cortex webcrawler
Disallow: /
User-agent: indy library
Disallow: /
User-agent: indylabs_marius
Disallow: /
User-agent: inelabot
Disallow: /
User-agent: inet32 ctrl
Disallow: /
User-agent: inetbot
Disallow: /
User-agent: info seeker
Disallow: /
User-agent: infolink
Disallow: /
User-agent: infomine
Disallow: /
User-agent: infonavirobot
Disallow: /
User-agent: informant
Disallow: /
User-agent: infoseek sidewinder
Disallow: /
User-agent: infotekies
Disallow: /
User-agent: infousabot
Disallow: /
User-agent: ingrid
Disallow: /
User-agent: inktomi
Disallow: /
User-agent: insightscollector
Disallow: /
User-agent: insightsworksbot
Disallow: /
User-agent: inspirebot
Disallow: /
User-agent: insumascout
Disallow: /
User-agent: intelix
Disallow: /
User-agent: intelliseek
Disallow: /
User-agent: interget
Disallow: /
User-agent: internet ninja
Disallow: /
User-agent: internet radio crawler
Disallow: /
User-agent: internetlinkagent
Disallow: /
User-agent: interseek
Disallow: /
User-agent: ioi
Disallow: /
User-agent: ip-web-crawler.com
Disallow: /
User-agent: ipadd bot
Disallow: /
User-agent: ipselonbot
Disallow: /
User-agent: ips-agent
Disallow: /
User-agent: iria
Disallow: /
User-agent: irlbot
Disallow: /
User-agent: iron33
Disallow: /
User-agent: isara
Disallow: /
User-agent: isearch
Disallow: /
User-agent: isilox
Disallow: /
User-agent: istellabot
Disallow: /
User-agent: its-learning crawler
Disallow: /
User-agent: iu_csci_b659_class_crawler
Disallow: /
User-agent: ivia
Disallow: /
User-agent: jadynave
Disallow: /
User-agent: java
Disallow: /
User-agent: jbot
Disallow: /
User-agent: jemmathetourist
Disallow: /
User-agent: jennybot
Disallow: /
User-agent: jetbot
Disallow: /
User-agent: jetbrains omea pro
Disallow: /
User-agent: jetcar
Disallow: /
User-agent: jim
Disallow: /
User-agent: jobo
Disallow: /
User-agent: jobspider_ba
Disallow: /
User-agent: joc
Disallow: /
User-agent: joedog
Disallow: /
User-agent: joyscapebot
Disallow: /
User-agent: jspyda
Disallow: /
User-agent: junut bot
Disallow: /
User-agent: justview
Disallow: /
User-agent: jyxobot
Disallow: /
User-agent: k.s.bot
Disallow: /
User-agent: kakclebot
Disallow: /
User-agent: kalooga
Disallow: /
User-agent: katatudo-spider
Disallow: /
User-agent: kbeta1
Disallow: /
User-agent: keepni web site monitor
Disallow: /
User-agent: kenjin.spider
Disallow: /
User-agent: keybot translation-search-machine
Disallow: /
User-agent: keywenbot
Disallow: /
User-agent: keyword density
Disallow: /
User-agent: keyword.density
Disallow: /
User-agent: kinjabot
Disallow: /
User-agent: kitenga-crawler-bot
Disallow: /
User-agent: kiwistatus
Disallow: /
User-agent: kmbot-
Disallow: /
User-agent: kmccrew bot search
Disallow: /
User-agent: knight
Disallow: /
User-agent: knowitall
Disallow: /
User-agent: knowledge engine
Disallow: /
User-agent: knowledge.com
Disallow: /
User-agent: koepabot
Disallow: /
User-agent: koninklijke
Disallow: /
User-agent: korniki
Disallow: /
User-agent: krowler
Disallow: /
User-agent: ksbot
Disallow: /
User-agent: kuloko-bot
Disallow: /
User-agent: kulturarw3
Disallow: /
User-agent: kummhttp
Disallow: /
User-agent: kurzor
Disallow: /
User-agent: kyluka crawl
Disallow: /
User-agent: l.webis
Disallow: /
User-agent: labhoo
Disallow: /
User-agent: labourunions411
Disallow: /
User-agent: lachesis
Disallow: /
User-agent: lament
Disallow: /
User-agent: lamerexterminator
Disallow: /
User-agent: lapozzbot
Disallow: /
User-agent: larbin
Disallow: /
User-agent: lbot
Disallow: /
User-agent: leaptag
Disallow: /
User-agent: leechftp
Disallow: /
User-agent: leechget
Disallow: /
User-agent: letscrawl.com
Disallow: /
User-agent: lexibot
Disallow: /
User-agent: lexxebot
Disallow: /
User-agent: lftp
Disallow: /
User-agent: libcrawl
Disallow: /
User-agent: libiviacore
Disallow: /
User-agent: libw
Disallow: /
User-agent: likse
Disallow: /
User-agent: linguee bot
Disallow: /
User-agent: link checker
Disallow: /
User-agent: link validator
Disallow: /
User-agent: linkalarm
Disallow: /
User-agent: linkbot
Disallow: /
User-agent: linkcheck by siteimprove.com
Disallow: /
User-agent: linkcheck scanner
Disallow: /
User-agent: linkchecker
Disallow: /
User-agent: linkdex.com
Disallow: /
User-agent: linkextractorpro
Disallow: /
User-agent: linklint
Disallow: /
User-agent: linklooker
Disallow: /
User-agent: linkman
Disallow: /
User-agent: links sql
Disallow: /
User-agent: linkscan
Disallow: /
User-agent: linksmanager.com_bot
Disallow: /
User-agent: linksweeper
Disallow: /
User-agent: linkwalker
Disallow: /
User-agent: link_checker
Disallow: /
User-agent: litefinder
Disallow: /
User-agent: litlrbot
Disallow: /
User-agent: little grabber at skanktale.com
Disallow: /
User-agent: livelapbot
Disallow: /
User-agent: lm harvester
Disallow: /
User-agent: lmqueuebot
Disallow: /
User-agent: lnspiderguy
Disallow: /
User-agent: loadtimebot
Disallow: /
User-agent: localcombot
Disallow: /
User-agent: locust
Disallow: /
User-agent: lolongbot
Disallow: /
User-agent: lookbot
Disallow: /
User-agent: lsearch
Disallow: /
User-agent: lssbot
Disallow: /
User-agent: lt scotland checklink
Disallow: /
User-agent: ltx71.com
Disallow: /
User-agent: lwp
Disallow: /
User-agent: lycos_spider
Disallow: /
User-agent: lydia entity spider
Disallow: /
User-agent: lynnbot
Disallow: /
User-agent: lytranslate
Disallow: /
User-agent: mag-net
Disallow: /
User-agent: magnet
Disallow: /
User-agent: magpie-crawler
Disallow: /
User-agent: magus bot
Disallow: /
User-agent: mail.ru
Disallow: /
User-agent: mainseek_bot
Disallow: /
User-agent: mammoth
Disallow: /
User-agent: map robot
Disallow: /
User-agent: markwatch
Disallow: /
User-agent: masagool
Disallow: /
User-agent: masidani_bot_
Disallow: /
User-agent: mass downloader
Disallow: /
User-agent: mata hari
Disallow: /
User-agent: mata.hari
Disallow: /
User-agent: matentzn at cs dot man dot ac dot uk
Disallow: /
User-agent: maxamine.com--robot
Disallow: /
User-agent: maxamine.com-robot
Disallow: /
User-agent: maxomobot
Disallow: /
User-agent: mcbot
Disallow: /
User-agent: medrabbit
Disallow: /
User-agent: megite
Disallow: /
User-agent: memacbot
Disallow: /
User-agent: memo
Disallow: /
User-agent: mendeleybot
Disallow: /
User-agent: mercator-
Disallow: /
User-agent: mercuryboard_user_agent_sql_injection.nasl
Disallow: /
User-agent: metacarta
Disallow: /
User-agent: metaeuro web search
Disallow: /
User-agent: metager2
Disallow: /
User-agent: metagloss
Disallow: /
User-agent: metal crawler
Disallow: /
User-agent: metaquerier
Disallow: /
User-agent: metaspider
Disallow: /
User-agent: metaspinner
Disallow: /
User-agent: metauri
Disallow: /
User-agent: mfcrawler
Disallow: /
User-agent: mfhttpscan
Disallow: /
User-agent: midown tool
Disallow: /
User-agent: miixpc
Disallow: /
User-agent: mini-robot
Disallow: /
User-agent: minibot
Disallow: /
User-agent: minirank
Disallow: /
User-agent: mirror
Disallow: /
User-agent: missigua locator
Disallow: /
User-agent: mister pix
Disallow: /
User-agent: mister.pix
Disallow: /
User-agent: miva
Disallow: /
User-agent: mj12bot
Disallow: /
User-agent: mnogosearch
Disallow: /
User-agent: moduna.com
Disallow: /
User-agent: mod_accessibility
Disallow: /
User-agent: moget
Disallow: /
User-agent: mojeekbot
Disallow: /
User-agent: monkeycrawl
Disallow: /
User-agent: moses
Disallow: /
User-agent: mowserbot
Disallow: /
User-agent: mqbot
Disallow: /
User-agent: mse360
Disallow: /
User-agent: msindianwebcrawl
Disallow: /
User-agent: msmobot
Disallow: /
User-agent: msnptc
Disallow: /
User-agent: msrbot
Disallow: /
User-agent: mt-soft
Disallow: /
User-agent: multitext
Disallow: /
User-agent: my-heritrix-crawler
Disallow: /
User-agent: myapp
Disallow: /
User-agent: mycompanybot
Disallow: /
User-agent: mycrawler
Disallow: /
User-agent: myengines-us-bot
Disallow: /
User-agent: myfamilybot
Disallow: /
User-agent: myra
Disallow: /
User-agent: my_little_searchengine_project
Disallow: /
User-agent: nabot
Disallow: /
User-agent: najdi.si
Disallow: /
User-agent: nambu
Disallow: /
User-agent: nameprotect
Disallow: /
User-agent: nasa search
Disallow: /
User-agent: natchcvs
Disallow: /
User-agent: natweb-bad-link-mailer
Disallow: /
User-agent: naver
Disallow: /
User-agent: navroad
Disallow: /
User-agent: nearsite
Disallow: /
User-agent: nec-meshexplorer
Disallow: /
User-agent: neosciocrawler
Disallow: /
User-agent: nerdbynature.bot
Disallow: /
User-agent: nerdybot
Disallow: /
User-agent: nerima-crawl-
Disallow: /
User-agent: nessus
Disallow: /
User-agent: nestreader
Disallow: /
User-agent: net vampire
Disallow: /
User-agent: net::trackback
Disallow: /
User-agent: netants
Disallow: /
User-agent: netcarta cyberpilot pro
Disallow: /
User-agent: netcraft
Disallow: /
User-agent: netexperts
Disallow: /
User-agent: netid.com bot
Disallow: /
User-agent: netmechanic
Disallow: /
User-agent: netprospector
Disallow: /
User-agent: netresearchserver
Disallow: /
User-agent: netseer
Disallow: /
User-agent: netshift=
Disallow: /
User-agent: netsongbot
Disallow: /
User-agent: netsparker
Disallow: /
User-agent: netspider
Disallow: /
User-agent: netsrcherp
Disallow: /
User-agent: netzip
Disallow: /
User-agent: newmedhunt
Disallow: /
User-agent: news bot
Disallow: /
User-agent: newsgatherer
Disallow: /
User-agent: newsgroupreporter
Disallow: /
User-agent: newstrovebot
Disallow: /
User-agent: news_search_app
Disallow: /
User-agent: nextgensearchbot
Disallow: /
User-agent: nextthing.org
Disallow: /
User-agent: nicebot
Disallow: /
User-agent: nicerspro
Disallow: /
User-agent: niki-bot
Disallow: /
User-agent: nimblecrawler
Disallow: /
User-agent: nimbus-1
Disallow: /
User-agent: ninetowns
Disallow: /
User-agent: ninja
Disallow: /
User-agent: njuicebot
Disallow: /
User-agent: nlese
Disallow: /
User-agent: nogate
Disallow: /
User-agent: norbert the spider
Disallow: /
User-agent: noteworthybot
Disallow: /
User-agent: npbot
Disallow: /
User-agent: nrcan intranet crawler
Disallow: /
User-agent: nsdl_search_bot
Disallow: /
User-agent: nuggetize.com bot
Disallow: /
User-agent: nusearch spider
Disallow: /
User-agent: nutch
Disallow: /
User-agent: nu_tch
Disallow: /
User-agent: nwspider
Disallow: /
User-agent: nymesis
Disallow: /
User-agent: nys-crawler
Disallow: /
User-agent: objectssearch
Disallow: /
User-agent: obot
Disallow: /
User-agent: obvius external linkcheck
Disallow: /
User-agent: ocelli
Disallow: /
User-agent: octopus
Disallow: /
User-agent: odp entries t_st
Disallow: /
User-agent: oegp
Disallow: /
User-agent: offline navigator
Disallow: /
User-agent: offline.explorer
Disallow: /
User-agent: ogspider
Disallow: /
User-agent: omiexplorer_bot
Disallow: /
User-agent: omniexplorer
Disallow: /
User-agent: omnifind
Disallow: /
User-agent: omniweb
Disallow: /
User-agent: onetszukaj
Disallow: /
User-agent: online link validator
Disallow: /
User-agent: oozbot
Disallow: /
User-agent: openbot
Disallow: /
User-agent: openfind
Disallow: /
User-agent: openintelligencedata
Disallow: /
User-agent: openisearch
Disallow: /
User-agent: openlink virtuoso rdf crawler
Disallow: /
User-agent: opensearchserver_bot
Disallow: /
User-agent: opidig
Disallow: /
User-agent: optidiscover
Disallow: /
User-agent: oracle secure enterprise search
Disallow: /
User-agent: oracle ultra search
Disallow: /
User-agent: orangebot
Disallow: /
User-agent: orisbot
Disallow: /
User-agent: ornl_crawler
Disallow: /
User-agent: ornl_mercury
Disallow: /
User-agent: osis-project.jp
Disallow: /
User-agent: oso
Disallow: /
User-agent: outfoxbot
Disallow: /
User-agent: outfoxmelonbot
Disallow: /
User-agent: owler-bot
Disallow: /
User-agent: owsbot
Disallow: /
User-agent: ozelot
Disallow: /
User-agent: p3p client
Disallow: /
User-agent: pagebiteshyperbot
Disallow: /
User-agent: pagebull
Disallow: /
User-agent: pagedown
Disallow: /
User-agent: pagefetcher
Disallow: /
User-agent: pagegrabber
Disallow: /
User-agent: pagepeeker
Disallow: /
User-agent: pagerank monitor
Disallow: /
User-agent: page_verifier
Disallow: /
User-agent: pamsnbot.htm
Disallow: /
User-agent: panopy bot
Disallow: /
User-agent: panscient.com
Disallow: /
User-agent: pansophica
Disallow: /
User-agent: papa foto
Disallow: /
User-agent: paperlibot
Disallow: /
User-agent: parasite
Disallow: /
User-agent: parsijoo
Disallow: /
User-agent: pathtraq
Disallow: /
User-agent: pattern
Disallow: /
User-agent: patwebbot
Disallow: /
User-agent: pavuk
Disallow: /
User-agent: paxleframework
Disallow: /
User-agent: pbbot
Disallow: /
User-agent: pcbrowser
Disallow: /
User-agent: pcore-http
Disallow: /
User-agent: pd-crawler
Disallow: /
User-agent: penthesila
Disallow: /
User-agent: perform_crawl
Disallow: /
User-agent: perman
Disallow: /
User-agent: personal ultimate crawler
Disallow: /
User-agent: php version tracker
Disallow: /
User-agent: phpcrawl
Disallow: /
User-agent: phpdig
Disallow: /
User-agent: picosearch
Disallow: /
User-agent: pieno robot
Disallow: /
User-agent: pipbot
Disallow: /
User-agent: pipeliner
Disallow: /
User-agent: pita
Disallow: /
User-agent: pixfinder
Disallow: /
User-agent: piyushbot
Disallow: /
User-agent: planetwork bot search
Disallow: /
User-agent: plucker
Disallow: /
User-agent: plukkie
Disallow: /
User-agent: plumtree
Disallow: /
User-agent: pockey
Disallow: /
User-agent: pocohttp
Disallow: /
User-agent: pogodak.ba
Disallow: /
User-agent: pogodak.co.yu
Disallow: /
User-agent: poirot
Disallow: /
User-agent: polybot
Disallow: /
User-agent: pompos
Disallow: /
User-agent: poodle predictor
Disallow: /
User-agent: popscreenbot
Disallow: /
User-agent: postpost
Disallow: /
User-agent: privacyfinder
Disallow: /
User-agent: projectwf-java-test-crawler
Disallow: /
User-agent: propowerbot
Disallow: /
User-agent: prowebwalker
Disallow: /
User-agent: proxem websearch
Disallow: /
User-agent: proximic
Disallow: /
User-agent: proxy crawler
Disallow: /
User-agent: psbot
Disallow: /
User-agent: pss-bot
Disallow: /
User-agent: psycheclone
Disallow: /
User-agent: pub-crawler
Disallow: /
User-agent: pucl
Disallow: /
User-agent: pulsebot
Disallow: /
User-agent: pump
Disallow: /
User-agent: pwebot
Disallow: /
User-agent: python
Disallow: /
User-agent: qeavis agent
Disallow: /
User-agent: qfkbot
Disallow: /
User-agent: qualidade
Disallow: /
User-agent: qualidator.com bot
Disallow: /
User-agent: quepasacreep
Disallow: /
User-agent: queryn metasearch
Disallow: /
User-agent: queryn.metasearch
Disallow: /
User-agent: quest.durato
Disallow: /
User-agent: quintura-crw
Disallow: /
User-agent: qunarbot
Disallow: /
User-agent: qwantify
Disallow: /
User-agent: qweerybot
Disallow: /
User-agent: qweery_robot.txt_checkbot
Disallow: /
User-agent: r2ibot
Disallow: /
User-agent: r6_commentreader
Disallow: /
User-agent: r6_feedfetcher
Disallow: /
User-agent: r6_votereader
Disallow: /
User-agent: rabot
Disallow: /
User-agent: radian6
Disallow: /
User-agent: radiation retriever
Disallow: /
User-agent: rampybot
Disallow: /
User-agent: rankivabot
Disallow: /
User-agent: rankur
Disallow: /
User-agent: rational sitecheck
Disallow: /
User-agent: rcstartbot
Disallow: /
User-agent: realdownload
Disallow: /
User-agent: reaper
Disallow: /
User-agent: rebi-shoveler
Disallow: /
User-agent: recorder
Disallow: /
User-agent: redbot
Disallow: /
User-agent: redcarpet
Disallow: /
User-agent: reget
Disallow: /
User-agent: repomonkey
Disallow: /
User-agent: research robot
Disallow: /
User-agent: riddler
Disallow: /
User-agent: riight
Disallow: /
User-agent: risenetbot
Disallow: /
User-agent: riverglassscanner
Disallow: /
User-agent: robopal
Disallow: /
User-agent: robosourcer
Disallow: /
User-agent: robotek
Disallow: /
User-agent: robozilla
Disallow: /
User-agent: roger
Disallow: /
User-agent: rome client
Disallow: /
User-agent: rondello
Disallow: /
User-agent: rotondo
Disallow: /
User-agent: roverbot
Disallow: /
User-agent: rpt-httpclient
Disallow: /
User-agent: rtgibot
Disallow: /
User-agent: rufusbot
Disallow: /
User-agent: runnk online rss reader
Disallow: /
User-agent: runnk rss aggregator
Disallow: /
User-agent: s2bot
Disallow: /
User-agent: safaribookmarkchecker
Disallow: /
User-agent: safednsbot
Disallow: /
User-agent: safetynet robot
Disallow: /
User-agent: saladspoon
Disallow: /
User-agent: sapienti
Disallow: /
User-agent: sapphireweb
Disallow: /
User-agent: sbider
Disallow: /
User-agent: sbl-bot
Disallow: /
User-agent: scfcrawler
Disallow: /
User-agent: scich
Disallow: /
User-agent: scientificcommons.org
Disallow: /
User-agent: scollspider
Disallow: /
User-agent: scooperbot
Disallow: /
User-agent: scooter
Disallow: /
User-agent: scoutjet
Disallow: /
User-agent: scrapebox
Disallow: /
User-agent: scrapy
Disallow: /
User-agent: scrawltest
Disallow: /
User-agent: screaming frog
Disallow: /
User-agent: scrubby
Disallow: /
User-agent: scspider
Disallow: /
User-agent: scumbot
Disallow: /
User-agent: search publisher
Disallow: /
User-agent: search x-bot
Disallow: /
User-agent: search-channel
Disallow: /
User-agent: search-engine-studio
Disallow: /
User-agent: search.kumkie.com
Disallow: /
User-agent: search.updated.com
Disallow: /
User-agent: search.usgs.gov
Disallow: /
User-agent: searcharoo.net
Disallow: /
User-agent: searchblox
Disallow: /
User-agent: searchbot
Disallow: /
User-agent: searchengine
Disallow: /
User-agent: searchhippo.com
Disallow: /
User-agent: searchit-bot
Disallow: /
User-agent: searchmarking
Disallow: /
User-agent: searchmarks
Disallow: /
User-agent: searchmee!
Disallow: /
User-agent: searchmee_v
Disallow: /
User-agent: searchmining
Disallow: /
User-agent: searchnowbot
Disallow: /
User-agent: searchpreview
Disallow: /
User-agent: searchspider.com
Disallow: /
User-agent: searqubot
Disallow: /
User-agent: seb spider
Disallow: /
User-agent: seekbot
Disallow: /
User-agent: seeker.lookseek.com
Disallow: /
User-agent: seeqbot
Disallow: /
User-agent: seeqpod-vertical-crawler
Disallow: /
User-agent: selflinkchecker
Disallow: /
User-agent: semager
Disallow: /
User-agent: semanticdiscovery
Disallow: /
User-agent: semantifire
Disallow: /
User-agent: semisearch
Disallow: /
User-agent: semrushbot
Disallow: /
User-agent: seoengworldbot
Disallow: /
User-agent: seokicks
Disallow: /
User-agent: seznambot
Disallow: /
User-agent: shablastbot
Disallow: /
User-agent: shadowwebanalyzer
Disallow: /
User-agent: shareaza
Disallow: /
User-agent: shelob
Disallow: /
User-agent: sherlock
Disallow: /
User-agent: shim-crawler
Disallow: /
User-agent: shopsalad
Disallow: /
User-agent: shopwiki
Disallow: /
User-agent: showlinks
Disallow: /
User-agent: showyoubot
Disallow: /
User-agent: siclab
Disallow: /
User-agent: silk
Disallow: /
User-agent: simplepie
Disallow: /
User-agent: siphon
Disallow: /
User-agent: sitebot
Disallow: /
User-agent: sitecheck
Disallow: /
User-agent: sitefinder
Disallow: /
User-agent: siteguardbot
Disallow: /
User-agent: siteorbiter
Disallow: /
User-agent: sitesnagger
Disallow: /
User-agent: sitesucker
Disallow: /
User-agent: sitesweeper
Disallow: /
User-agent: sitexpert
Disallow: /
User-agent: skimbot
Disallow: /
User-agent: skimwordsbot
Disallow: /
User-agent: skreemrbot
Disallow: /
User-agent: skywalker
Disallow: /
User-agent: sleipnir
Disallow: /
User-agent: slow-crawler
Disallow: /
User-agent: slysearch
Disallow: /
User-agent: smart-crawler
Disallow: /
User-agent: smartdownload
Disallow: /
User-agent: smarte bot
Disallow: /
User-agent: smartwit.com
Disallow: /
User-agent: snake
Disallow: /
User-agent: snap.com beta crawler
Disallow: /
User-agent: snapbot
Disallow: /
User-agent: snappreviewbot
Disallow: /
User-agent: snappy
Disallow: /
User-agent: snookit
Disallow: /
User-agent: snooper
Disallow: /
User-agent: snoopy
Disallow: /
User-agent: societyrobot
Disallow: /
User-agent: socscibot
Disallow: /
User-agent: soft411 directory
Disallow: /
User-agent: sogou
Disallow: /
User-agent: sohu agent
Disallow: /
User-agent: sohu-search
Disallow: /
User-agent: sokitomi crawl
Disallow: /
User-agent: solbot
Disallow: /
User-agent: sondeur
Disallow: /
User-agent: sootle
Disallow: /
User-agent: sosospider
Disallow: /
User-agent: space bison
Disallow: /
User-agent: space fung
Disallow: /
User-agent: spacebison
Disallow: /
User-agent: spankbot
Disallow: /
User-agent: spanner
Disallow: /
User-agent: spatineo monitor controller
Disallow: /
User-agent: spatineo serval controller
Disallow: /
User-agent: spatineo serval getmapbot
Disallow: /
User-agent: special_archiver
Disallow: /
User-agent: speedy
Disallow: /
User-agent: sphere scout
Disallow: /
User-agent: sphider
Disallow: /
User-agent: spider.terranautic.net
Disallow: /
User-agent: spiderengine
Disallow: /
User-agent: spiderku
Disallow: /
User-agent: spiderman
Disallow: /
User-agent: spinn3r
Disallow: /
User-agent: spinne
Disallow: /
User-agent: sportcrew-bot
Disallow: /
User-agent: sproose
Disallow: /
User-agent: spyder3.microsys.com
Disallow: /
User-agent: sq webscanner
Disallow: /
User-agent: sqlmap
Disallow: /
User-agent: squid-prefetch
Disallow: /
User-agent: squidclamav_redirector
Disallow: /
User-agent: sqworm
Disallow: /
User-agent: srevbot
Disallow: /
User-agent: sslbot
Disallow: /
User-agent: ssm agent
Disallow: /
User-agent: stackrambler
Disallow: /
User-agent: stardownloader
Disallow: /
User-agent: statbot
Disallow: /
User-agent: statcrawler
Disallow: /
User-agent: statedept-crawler
Disallow: /
User-agent: steeler
Disallow: /
User-agent: stegmann-bot
Disallow: /
User-agent: stero
Disallow: /
User-agent: stripper
Disallow: /
User-agent: stumbler
Disallow: /
User-agent: suchclip
Disallow: /
User-agent: sucker
Disallow: /
User-agent: sumeetbot
Disallow: /
User-agent: sumitbot
Disallow: /
User-agent: summizebot
Disallow: /
User-agent: summizefeedreader
Disallow: /
User-agent: sunrise xp
Disallow: /
User-agent: superbot
Disallow: /
User-agent: superhttp
Disallow: /
User-agent: superlumin downloader
Disallow: /
User-agent: superpagesbot
Disallow: /
User-agent: supremesearch.net
Disallow: /
User-agent: supybot
Disallow: /
User-agent: surdotlybot
Disallow: /
User-agent: surf
Disallow: /
User-agent: surveybot
Disallow: /
User-agent: suzuran
Disallow: /
User-agent: swebot
Disallow: /
User-agent: swish-e
Disallow: /
User-agent: sygolbot
Disallow: /
User-agent: synapticwalker
Disallow: /
User-agent: syntryx ant scout chassis pheromone
Disallow: /
User-agent: systemsearch-robot
Disallow: /
User-agent: szukacz
Disallow: /
User-agent: s~stremor-crawler
Disallow: /
User-agent: t-h-u-n-d-e-r-s-t-o-n-e
Disallow: /
User-agent: tailrank
Disallow: /
User-agent: takeout
Disallow: /
User-agent: talkro web-shot
Disallow: /
User-agent: tamu_crawler
Disallow: /
User-agent: tapuzbot
Disallow: /
User-agent: tarantula
Disallow: /
User-agent: targetblaster.com
Disallow: /
User-agent: targetyournews.com bot
Disallow: /
User-agent: tausdatabot
Disallow: /
User-agent: taxinomiabot
Disallow: /
User-agent: teamsoft wininet component
Disallow: /
User-agent: tecomi bot
Disallow: /
User-agent: teezirbot
Disallow: /
User-agent: teleport
Disallow: /
User-agent: telesoft
Disallow: /
User-agent: teradex mapper
Disallow: /
User-agent: teragram_crawler
Disallow: /
User-agent: terrawizbot
Disallow: /
User-agent: testbot
Disallow: /
User-agent: testing of bot
Disallow: /
User-agent: textbot
Disallow: /
User-agent: thatrobotsite.com
Disallow: /
User-agent: the dyslexalizer
Disallow: /
User-agent: the intraformant
Disallow: /
User-agent: the.intraformant
Disallow: /
User-agent: thenomad
Disallow: /
User-agent: theophrastus
Disallow: /
User-agent: theusefulbot
Disallow: /
User-agent: thumbbot
Disallow: /
User-agent: thumbnail.cz robot
Disallow: /
User-agent: thumbshots-de-bot
Disallow: /
User-agent: tigerbot
Disallow: /
User-agent: tighttwatbot
Disallow: /
User-agent: tineye
Disallow: /
User-agent: titan
Disallow: /
User-agent: to-dress_ru_bot_
Disallow: /
User-agent: to-night-bot
Disallow: /
User-agent: tocrawl
Disallow: /
User-agent: topicalizer
Disallow: /
User-agent: topicblogs
Disallow: /
User-agent: toplistbot
Disallow: /
User-agent: topserver php
Disallow: /
User-agent: topyx-crawler
Disallow: /
User-agent: touche
Disallow: /
User-agent: tourlentascanner
Disallow: /
User-agent: tpsystem
Disallow: /
User-agent: traazi
Disallow: /
User-agent: transgenikbot
Disallow: /
User-agent: travel-search
Disallow: /
User-agent: travelbot
Disallow: /
User-agent: travellazerbot
Disallow: /
User-agent: treezy
Disallow: /
User-agent: trendiction
Disallow: /
User-agent: trex
Disallow: /
User-agent: tridentspider
Disallow: /
User-agent: trovator
Disallow: /
User-agent: true_robot
Disallow: /
User-agent: tscholarsbot
Disallow: /
User-agent: tsm translation-search-machine
Disallow: /
User-agent: tswebbot
Disallow: /
User-agent: tulipchain
Disallow: /
User-agent: turingos
Disallow: /
User-agent: turnitinbot
Disallow: /
User-agent: tutorgigbot
Disallow: /
User-agent: tweetedtimes bot
Disallow: /
User-agent: tweetmemebot
Disallow: /
User-agent: twengabot
Disallow: /
User-agent: twice
Disallow: /
User-agent: twikle
Disallow: /
User-agent: twinuffbot
Disallow: /
User-agent: twisted pagegetter
Disallow: /
User-agent: twitturls
Disallow: /
User-agent: twitturly
Disallow: /
User-agent: tygobot
Disallow: /
User-agent: tygoprowler
Disallow: /
User-agent: typhoeus
Disallow: /
User-agent: u.s. government printing office
Disallow: /
User-agent: uberbot
Disallow: /
User-agent: ucb-nutch
Disallow: /
User-agent: udmsearch
Disallow: /
User-agent: ufam-crawler-
Disallow: /
User-agent: ultraseek
Disallow: /
User-agent: unchaos
Disallow: /
User-agent: unisterbot
Disallow: /
User-agent: unidentified
Disallow: /
User-agent: unitek uniengine
Disallow: /
User-agent: universalsearch
Disallow: /
User-agent: unwindfetchor
Disallow: /
User-agent: uoftdb_experiment
Disallow: /
User-agent: updated
Disallow: /
User-agent: url control
Disallow: /
User-agent: url-checker
Disallow: /
User-agent: urlappendbot
Disallow: /
User-agent: urlblaze
Disallow: /
User-agent: urlchecker
Disallow: /
User-agent: urlck
Disallow: /
User-agent: urldispatcher
Disallow: /
User-agent: urlspiderpro
Disallow: /
User-agent: urly warning
Disallow: /
User-agent: urly.warning
Disallow: /
User-agent: url_gather
Disallow: /
User-agent: usaf afkn k2spider
Disallow: /
User-agent: usasearch
Disallow: /
User-agent: uss-cosmix
Disallow: /
User-agent: usyd-nlp-spider
Disallow: /
User-agent: vacobot
Disallow: /
User-agent: vacuum
Disallow: /
User-agent: vadixbot
Disallow: /
User-agent: vagabondo
Disallow: /
User-agent: validator
Disallow: /
User-agent: valkyrie
Disallow: /
User-agent: vbseo
Disallow: /
User-agent: vci webviewer vci webviewer win32
Disallow: /
User-agent: verbstarbot
Disallow: /
User-agent: vericitecrawler
Disallow: /
User-agent: verifactrola
Disallow: /
User-agent: verity-url-gateway
Disallow: /
User-agent: vermut
Disallow: /
User-agent: versus crawler
Disallow: /
User-agent: versus.integis.ch
Disallow: /
User-agent: viasarchivinginformation.html
Disallow: /
User-agent: vipr
Disallow: /
User-agent: virus-detector
Disallow: /
User-agent: virus_detector
Disallow: /
User-agent: visbot
Disallow: /
User-agent: vishal for clia
Disallow: /
User-agent: visweb
Disallow: /
User-agent: vital search'n urchin
Disallow: /
User-agent: vlad
Disallow: /
User-agent: vlsearch
Disallow: /
User-agent: voilabot
Disallow: /
User-agent: vmbot
Disallow: /
User-agent: vocusbot
Disallow: /
User-agent: voideye
Disallow: /
User-agent: voil
Disallow: /
User-agent: vortex
Disallow: /
User-agent: voyager
Disallow: /
User-agent: vspider
Disallow: /
User-agent: w3c-webcon
Disallow: /
User-agent: w3c_unicorn
Disallow: /
User-agent: w3search
Disallow: /
User-agent: wacbot
Disallow: /
User-agent: wanadoo
Disallow: /
User-agent: wastrix
Disallow: /
User-agent: water conserve portal
Disallow: /
User-agent: water conserve spider
Disallow: /
User-agent: watzbot
Disallow: /
User-agent: wauuu
Disallow: /
User-agent: wavefire
Disallow: /
User-agent: waypath
Disallow: /
User-agent: wazzup
Disallow: /
User-agent: wbdbot
Disallow: /
User-agent: web ceo online robot
Disallow: /
User-agent: web crawler
Disallow: /
User-agent: web downloader
Disallow: /
User-agent: web image collector
Disallow: /
User-agent: web link validator
Disallow: /
User-agent: web magnet
Disallow: /
User-agent: web site downloader
Disallow: /
User-agent: web sucker
Disallow: /
User-agent: web-agent
Disallow: /
User-agent: web-sniffer
Disallow: /
User-agent: web.image.collector
Disallow: /
User-agent: webaltbot
Disallow: /
User-agent: webauto
Disallow: /
User-agent: webbot
Disallow: /
User-agent: webbul-bot
Disallow: /
User-agent: webcapture
Disallow: /
User-agent: webcheck
Disallow: /
User-agent: webclipping.com
Disallow: /
User-agent: webcollage
Disallow: /
User-agent: webcopier
Disallow: /
User-agent: webcopy
Disallow: /
User-agent: webcorp
Disallow: /
User-agent: webcrawl.net
Disallow: /
User-agent: webcrawler
Disallow: /
User-agent: webdatacentrebot
Disallow: /
User-agent: webdownloader for x
Disallow: /
User-agent: webdup
Disallow: /
User-agent: webemailextrac
Disallow: /
User-agent: webenhancer
Disallow: /
User-agent: webfetch
Disallow: /
User-agent: webgather
Disallow: /
User-agent: webgo is
Disallow: /
User-agent: webgobbler
Disallow: /
User-agent: webimages
Disallow: /
User-agent: webinator-search2
Disallow: /
User-agent: webinator-wbi
Disallow: /
User-agent: webindex
Disallow: /
User-agent: weblayers
Disallow: /
User-agent: webleacher
Disallow: /
User-agent: weblexbot
Disallow: /
User-agent: weblinker
Disallow: /
User-agent: weblyzard
Disallow: /
User-agent: webmastercoffee
Disallow: /
User-agent: webmasterworld extractor
Disallow: /
User-agent: webmasterworldforumbot
Disallow: /
User-agent: webminer
Disallow: /
User-agent: webmoose
Disallow: /
User-agent: webot
Disallow: /
User-agent: webpix
Disallow: /
User-agent: webreaper
Disallow: /
User-agent: webripper
Disallow: /
User-agent: websauger
Disallow: /
User-agent: webscan
Disallow: /
User-agent: websearchbench
Disallow: /
User-agent: website
Disallow: /
User-agent: webspear
Disallow: /
User-agent: websphinx
Disallow: /
User-agent: webspider
Disallow: /
User-agent: webster
Disallow: /
User-agent: webstripper
Disallow: /
User-agent: webtrafficexpress
Disallow: /
User-agent: webtrends link analyzer
Disallow: /
User-agent: webvac
Disallow: /
User-agent: webwalk
Disallow: /
User-agent: webwasher
Disallow: /
User-agent: webwatch
Disallow: /
User-agent: webwhacker
Disallow: /
User-agent: webxm
Disallow: /
User-agent: webzip
Disallow: /
User-agent: weddings.info
Disallow: /
User-agent: wenbin
Disallow: /
User-agent: wep search
Disallow: /
User-agent: wepa
Disallow: /
User-agent: werelatebot
Disallow: /
User-agent: wget
Disallow: /
User-agent: whacker
Disallow: /
User-agent: whirlpool web engine
Disallow: /
User-agent: whowhere robot
Disallow: /
User-agent: widow
Disallow: /
User-agent: wikiabot
Disallow: /
User-agent: wikio
Disallow: /
User-agent: wikiwix-bot-
Disallow: /
User-agent: winhttp
Disallow: /
User-agent: wire
Disallow: /
User-agent: wisebot
Disallow: /
User-agent: wisenutbot
Disallow: /
User-agent: wish-la
Disallow: /
User-agent: wish-project
Disallow: /
User-agent: wisponbot
Disallow: /
User-agent: wmcai-robot
Disallow: /
User-agent: wminer
Disallow: /
User-agent: wmsbot
Disallow: /
User-agent: woriobot
Disallow: /
User-agent: worldshop
Disallow: /
User-agent: worqmada
Disallow: /
User-agent: wotbox
Disallow: /
User-agent: wume_crawler
Disallow: /
User-agent: www collector
Disallow: /
User-agent: www-collector-e
Disallow: /
User-agent: www-mechanize
Disallow: /
User-agent: wwwoffle
Disallow: /
User-agent: wwwrobot
Disallow: /
User-agent: wwwster
Disallow: /
User-agent: wwwwanderer
Disallow: /
User-agent: wwwxref
Disallow: /
User-agent: wysigot
Disallow: /
User-agent: x-clawler
Disallow: /
User-agent: x-crawler
Disallow: /
User-agent: xaldon
Disallow: /
User-agent: xenu
Disallow: /
User-agent: xerka metabot
Disallow: /
User-agent: xerka webbot
Disallow: /
User-agent: xget
Disallow: /
User-agent: xirq
Disallow: /
User-agent: xmarksfetch
Disallow: /
User-agent: xqrobot
Disallow: /
User-agent: y!j
Disallow: /
User-agent: yacy.net
Disallow: /
User-agent: yacybot
Disallow: /
User-agent: yanga worldsearch bot
Disallow: /
User-agent: yarienavoir.net
Disallow: /
User-agent: yasaklibot
Disallow: /
User-agent: yats crawler
Disallow: /
User-agent: ybot
Disallow: /
User-agent: yebolbot
Disallow: /
User-agent: yellowjacket
Disallow: /
User-agent: yeti
Disallow: /
User-agent: yolinkbot
Disallow: /
User-agent: yooglifetchagent
Disallow: /
User-agent: yoono
Disallow: /
User-agent: yottacars_bot
Disallow: /
User-agent: yourls
Disallow: /
User-agent: z-add link checker
Disallow: /
User-agent: zagrebin
Disallow: /
User-agent: zao
Disallow: /
User-agent: zedzo.validate
Disallow: /
User-agent: zermelo
Disallow: /
User-agent: zeus
Disallow: /
User-agent: zibber-v
Disallow: /
User-agent: zimeno
Disallow: /
User-agent: zing-bottabot
Disallow: /
User-agent: zipppbot
Disallow: /
User-agent: zongbot
Disallow: /
User-agent: zoomspider
Disallow: /
User-agent: zotag search
Disallow: /
User-agent: zsebot
Disallow: /
User-agent: zuibot
Disallow: /
User-agent: zyborg
Disallow: /
User-agent: zyte
Disallow: /

IP адреса

США - 72.21.194.1

США - 72.21.211.176

США - 72.21.214.128

Организация

Amazon.com

Интернет провайдер

Amazon.com

HTTP/1.1 200 OK
Date: Sun, 28 Jul 2019 17:44:18 GMT
Server: Apache/2.2.15 (Red Hat)
Set-Cookie: OAMAuthnCookie_www.spar.si:443=ai8OyR6XKEzqWWD0LC1fWfwlwGTl0X0KkdAM2cwcyTsUBtiIDhysU2yX0Vjv3EIWmOBQDHuIGGZzN6Ql3Pnlr0srS9LmXVI1kgkRQMn37OF30PZIB2thDXzFCHd92yU9Wf8USL5AaFasM88X7Vjli1EeE%2FyNzYGtwnoR7SkJKUZnGqszxuwGPrcMoOO1R41oTOLXASGXrFKkh6e%2BS1Y%2BV%2FXjH5qEKkrdaPEqErVRetfOF3KfRx8LDbMsv1OKKIrAQ%2Fv9KrQ5vwbeb7CCm2Lpa11oojjxcStaiRoyF7vLA0OOMEPKlKc9XSR73mGez5Mb5dX9tv4WpM8gQojb4R%2BoJL9t%2B%2FDHteijuKRfu9jfJe%2BDmG4FcGs%2FfmtUcSvScKMEXRT6GGV5qXvj4VdmmAhchA%3D%3D; httponly; secure; path=/
Cache-Control: no-cache
Pragma: no-cache
Last-Modified: Sun, 28 Jul 2019 08:15:23 GMT
ETag: "7c03df-a87c1-58eb9608f7d52"
Accept-Ranges: bytes
Cteonnt-Length: 690113
Content-Type: text/html; charset=UTF-8
Cneonction: close
Set-Cookie: NSC_bumc-bfnxxxq_dmvtufs1=ffffffffaf171cdc45525d5f4f58455e445a4a423660;expires=Sun, 28-Jul-2019 18:14:18 GMT;path=/;httponly
Transfer-Encoding: chunked
Content-Language: en
Set-Cookie: NSC_wt_qpsubm_pbn_tj-q-7826=ffffffffaf171f1645525d5f4f58455e445a4a4228a2;expires=Sun, 28-Jul-2019 18:14:18 GMT;path=/;secure;httponly

Проиндексирован

• Mail.ru
• Yahoo.com
• Bing.com
• Картинки Google
• Картинки Yandex