| Возраст домена | n/a |
| Дата окончания | n/a |
| ИКС | n/a |
| Страниц в Google | n/a |
| Страниц в Яндексе | n/a |
| Dmoz | n/a |
| Яндекс Каталог | n/a |
| Alexa Traffic Rank | n/a |
| Alexa Country | n/a |
| История изменения показателей | Авторизация |
Идет сбор информации... Обновить
Mhd Arjunanta
web development, seo, digital marketing, mhd arjunanta
Artikel, tutorial, dan insight seputar web development, SEO, digital marketing, dan teknologi.
UTF-8
265.52 КБ
1 003
7 590 симв.
6 455 симв.
Идет сбор информации... Обновить
Идет сбор информации... Обновить

Идет сбор информации... Обновить
Идет сбор информации... Обновить
# ============================================================
# robots.txt - Mhd Arjunanta
# https://mhdarjunanta.com/
# Last Updated: 2026-10-19
# Auto-Generated by robots.php
# Fitur: Auto Crawl, Auto Index, Auto SEO, Auto Schema
# ============================================================
# ------------------------------------------------------------
# SEMUA CRAWLER - Aturan Default
# ------------------------------------------------------------
User-agent: *
Allow: /
Allow: /sitemap.xml
Allow: /sitemap_index.xml
Allow: /robots.txt
Allow: /favicon.ico
Allow: /manifest.json
Allow: /manifest.webmanifest
Allow: /site.webmanifest
Allow: /browserconfig.xml
Allow: /security.txt
Allow: /ads.txt
Allow: /humans.txt
Allow: /apple-touch-icon.png
Allow: /apple-touch-icon-precomposed.png
Allow: /rss
Allow: /rss.xml
Allow: /feed
Allow: /feed.xml
Allow: /atom.xml
Allow: /feed.json
Allow: /json-feed
Allow: /blog
Allow: /blog/
Allow: /projects
Allow: /projects/
Allow: /about
Allow: /about/
Allow: /contact
Allow: /contact/
Allow: /faq
Allow: /faq/
Allow: /location
Allow: /location/
Allow: /promo-2026
Allow: /experience
Allow: /tags
Allow: /tags/
Allow: /archive
Allow: /archive/
Allow: /category/
Allow: /tag/
Allow: /author/
Allow: /*.css$
Allow: /*.js$
Allow: /*.mjs$
Allow: /*.png$
Allow: /*.jpg$
Allow: /*.jpeg$
Allow: /*.webp$
Allow: /*.avif$
Allow: /*.gif$
Allow: /*.svg$
Allow: /*.ico$
Allow: /*.woff$
Allow: /*.woff2$
Allow: /*.ttf$
Allow: /*.otf$
Allow: /*.eot$
# --- SITEMAP EXCEPTION ---
Allow: /sitemap.xml
Allow: /sitemap_index.xml
Allow: /image-sitemap.xml
Allow: /news-sitemap.xml
Allow: /video-sitemap.xml
Allow: /news-sitemap.php
Allow: /robots.txt
# ------------------------------------------------------------
# Halaman & folder yang tidak perlu diindeks
# ------------------------------------------------------------
Disallow: /404
Disallow: /404.html
Disallow: /404.php
Disallow: /search
Disallow: /search/
Disallow: /search?
Disallow: /login
Disallow: /login.php
Disallow: /register
Disallow: /register.php
Disallow: /logout
Disallow: /logout.php
Disallow: /member/
Disallow: /members/
Disallow: /account/
Disallow: /dashboard/
Disallow: /cgi-bin/
Disallow: /tmp/
Disallow: /temp/
Disallow: /cache/
Disallow: /backup/
Disallow: /backups/
Disallow: /private/
Disallow: /secret/
Disallow: /hidden/
Disallow: /admin/
Disallow: /administrator/
Disallow: /phpmyadmin/
Disallow: /myadmin/
Disallow: /pma/
Disallow: /cpanel/
Disallow: /webmail/
Disallow: /logs/
Disallow: /log/
Disallow: /debug/
Disallow: /test/
Disallow: /testing/
Disallow: /staging/
Disallow: /dev/
Disallow: /core/
Disallow: /vendor/
Disallow: /node_modules/
Disallow: /database/
Disallow: /db/
Disallow: /config/
Disallow: /configs/
Disallow: /install/
Disallow: /setup/
Disallow: /upgrade/
Disallow: /api/
Disallow: /api/v1/
Disallow: /api/v2/
Disallow: /graphql
Disallow: /webhook/
Disallow: /webhooks/
Disallow: /cron/
Disallow: /jobs/
Disallow: /queue/
Disallow: /workers/
# Query string yang tidak perlu diindeks
Disallow: /*?s=
Disallow: /*?q=
Disallow: /*?search=
Disallow: /*?utm_
Disallow: /*?fbclid=
Disallow: /*?gclid=
Disallow: /*?ref=
Disallow: /*?aff=
Disallow: /*?affiliate=
Disallow: /*?session=
Disallow: /*?sid=
Disallow: /*?token=
Disallow: /*?preview=
Disallow: /*?draft=
Disallow: /*?print=
Disallow: /*?amp=
Disallow: /*?replytocom=
Disallow: /*?filter=
Disallow: /*?sort=
Disallow: /*?order=
Disallow: /*?page=
Disallow: /*?p=
Disallow: /*?pid=
Disallow: /*?id=
Disallow: /*?cat=
Disallow: /*?tag=
Disallow: /*?author=
Disallow: /*?month=
Disallow: /*?year=
# File & folder sensitif (dotfiles)
Disallow: /.*$
Disallow: /.env
Disallow: /.env.*
Disallow: /.git/
Disallow: /.gitignore
Disallow: /.gitattributes
Disallow: /.svn/
Disallow: /.hg/
Disallow: /.htaccess
Disallow: /.htpasswd
Disallow: /.user.ini
Disallow: /composer.json
Disallow: /composer.lock
Disallow: /package.json
Disallow: /package-lock.json
Disallow: /yarn.lock
Disallow: /pnpm-lock.yaml
Disallow: /Dockerfile
Disallow: /docker-compose.yml
Disallow: /.dockerignore
# Blokir ekstensi file sensitif
Disallow: /*.log$
Disallow: /*.env$
Disallow: /*.ini$
Disallow: /*.conf$
Disallow: /*.config$
Disallow: /*.cfg$
Disallow: /*.bak$
Disallow: /*.backup$
Disallow: /*.old$
Disallow: /*.orig$
Disallow: /*.save$
Disallow: /*.swp$
Disallow: /*.swo$
Disallow: /*.swn$
Disallow: /*.sql$
Disallow: /*.db$
Disallow: /*.sqlite$
Disallow: /*.sqlite3$
Disallow: /*.lock$
Disallow: /*.yml$
Disallow: /*.yaml$
Disallow: /*.dist$
Disallow: /*.sh$
Disallow: /*.bash$
Disallow: /*.zsh$
Disallow: /*.exe$
Disallow: /*.dll$
Disallow: /*.bat$
Disallow: /*.cmd$
Disallow: /*.ps1$
Disallow: /*.jar$
Disallow: /*.war$
Disallow: /*.pyc$
Disallow: /*.pyo$
Disallow: /*.php.bak$
Disallow: /*.php.old$
Disallow: /*.php~
Disallow: /*~
# ------------------------------------------------------------
# MESIN PENCARI UTAMA - Google
# ------------------------------------------------------------
User-agent: Googlebot
Allow: /
Allow: /*.json$
Allow: /*.xml$
Allow: /sitemap.xml
Allow: /news-sitemap.xml
Allow: /rss
Allow: /feed
Disallow: /404
Disallow: /404.html
Disallow: /search
Disallow: /admin/
Disallow: /api/
User-agent: Googlebot-Image
Allow: /
Allow: /*.png$
Allow: /*.jpg$
Allow: /*.jpeg$
Allow: /*.webp$
Allow: /*.avif$
Allow: /*.svg$
Allow: /*.gif$
Disallow: /admin/
Disallow: /private/
User-agent: Googlebot-News
Allow: /
Allow: /blog/
Allow: /category/
Allow: /tag/
Allow: /author/
Allow: /archive/
Disallow: /admin/
Disallow: /api/
User-agent: Googlebot-Video
Allow: /
Allow: /*.mp4$
Allow: /*.webm$
Allow: /*.mov$
Disallow: /admin/
User-agent: Google-InspectionTool
Allow: /
User-agent: Google-Extended
Allow: /
User-agent: APIs-Google
Allow: /
User-agent: Mediapartners-Google
Allow: /
User-agent: AdsBot-Google
Allow: /
User-agent: AdsBot-Google-Mobile
Allow: /
User-agent: FeedFetcher-Google
Allow: /
Allow: /rss
Allow: /feed
Allow: /atom.xml
Allow: /feed.json
User-agent: Storebot-Google
Allow: /
User-agent: Chrome-Lighthouse
Allow: /
# ------------------------------------------------------------
# MESIN PENCARI UTAMA - Bing & Microsoft
# ------------------------------------------------------------
User-agent: Bingbot
Allow: /
Allow: /*.json$
Allow: /*.xml$
Allow: /sitemap.xml
Allow: /news-sitemap.xml
Allow: /rss
Disallow: /404
Disallow: /404.html
Disallow: /search
User-agent: msnbot
Allow: /
User-agent: adidxbot
Allow: /
User-agent: BingPreview
Allow: /
User-agent: MicrosoftPreview
Allow: /
# ------------------------------------------------------------
# MESIN PENCARI UTAMA - Yandex
# ------------------------------------------------------------
User-agent: YandexBot
Allow: /
Allow: /sitemap.xml
Allow: /news-sitemap.xml
Allow: /rss
Crawl-delay: 1
User-agent: YandexImages
Allow: /
User-agent: YandexMobileBot
Allow: /
User-agent: YandexMetrika
Allow: /
User-agent: YandexNews
Allow: /
User-agent: YandexVideo
Allow: /
# ------------------------------------------------------------
# MESIN PENCARI LAIN - Baidu, DuckDuckGo, Apple, dll.
# ------------------------------------------------------------
User-agent: Baiduspider
Allow: /
Allow: /sitemap.xml
User-agent: Baiduspider-image
Allow: /
User-agent: Baiduspider-video
Allow: /
User-agent: Baiduspider-news
Allow: /
User-agent: DuckDuckBot
Allow: /
User-agent: DuckDuckGo-Favicons-Bot
Allow: /
User-agent: Applebot
Allow: /
User-agent: Applebot-Extended
Allow: /
User-agent: Slurp
Allow: /
User-agent: Sogou web spider
Allow: /
User-agent: Sogou inst spider
Allow: /
User-agent: Exabot
Allow: /
User-agent: ia_archiver
Allow: /
User-agent: SeznamBot
Allow: /
User-agent: Naver
Allow: /
User-agent: Yeti
Allow: /
User-agent: Daum
Allow: /
User-agent: Daumoa
Allow: /
User-agent: Qwantify
Allow: /
User-agent: MojeekBot
Allow: /
User-agent: Gigabot
Allow: /
# ------------------------------------------------------------
# SOCIAL MEDIA & PREVIEW BOTS
# ------------------------------------------------------------
User-agent: facebookexternalhit
Allow: /
User-agent: FacebookBot
Allow: /
User-agent: meta-externalagent
Allow: /
User-agent: Twitterbot
Allow: /
User-agent: LinkedInBot
Allow: /
User-agent: Pinterest
Allow: /
User-agent: Pinterestbot
Allow: /
User-agent: Slackbot
Allow: /
User-agent: Slackbot-Links
Allow: /
User-agent: TelegramBot
Allow: /
User-agent: WhatsApp
Allow: /
User-agent: Discordbot
Allow: /
User-agent: Embedly
Allow: /
User-agent: Iframely
Allow: /
User-agent: W3C_Validator
Allow: /
User-agent: Validator.W3.org
Allow: /
User-agent: FeedValidator
Allow: /
# ------------------------------------------------------------
# TOOLS MONITORING
# ------------------------------------------------------------
User-agent: UptimeRobot
Allow: /
User-agent: StatusCake
Allow: /
User-agent: Pingdom
Allow: /
User-agent: GTmetrix
Allow: /
User-agent: Site24x7
Allow: /
User-agent: Better Uptime
Allow: /
User-agent: Freshping
Allow: /
# ------------------------------------------------------------
# SEO TOOLS
# ------------------------------------------------------------
User-agent: AhrefsBot
Allow: /
User-agent: SemrushBot
Allow: /
User-agent: Screaming Frog SEO Spider
Allow: /
User-agent: Majestic
Allow: /
User-agent: Moz
Allow: /
User-agent: rogerbot
Allow: /
# ------------------------------------------------------------
# AI CRAWLERS - DIIZINKAN
# ------------------------------------------------------------
User-agent: GPTBot
Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: Claude-Web
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: Claude-SearchBot
Allow: /
User-agent: Claude-User
Allow: /
User-agent: anthropic-ai
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Perplexity-User
Allow: /
User-agent: cohere-ai
Allow: /
User-agent: cohere-training-data-crawler
Allow: /
User-agent: CCBot
Allow: /
User-agent: Bytespider
Allow: /
User-agent: Meta-ExternalAgent
Allow: /
User-agent: meta-externalfetcher
Allow: /
User-agent: Applebot-Extended
Allow: /
User-agent: Google-Extended
Allow: /
User-agent: YouBot
Allow: /
User-agent: Diffbot
Allow: /
User-agent: DeepSeekBot
Allow: /
User-agent: QwenBot
Allow: /
User-agent: GrokBot
Allow: /
User-agent: xAI-Crawler
Allow: /
User-agent: DuckAssistBot
Allow: /
User-agent: Omgilibot
Allow: /
User-agent: ImagesiftBot
Allow: /
# ------------------------------------------------------------
# BAD BOTS & SCRAPERS - DIBLOKIR
# ------------------------------------------------------------
User-agent: MJ12bot
Disallow: /
User-agent: DotBot
Disallow: /
User-agent: MegaIndex
Disallow: /
User-agent: BLEXBot
Disallow: /
User-agent: blexbot
Disallow: /
User-agent: SEOkicks
Disallow: /
User-agent: Rogerbot
Disallow: /
User-agent: Scrapy
Disallow: /
User-agent: HTTrack
Disallow: /
User-agent: python-requests
Disallow: /
User-agent: python-urllib
Disallow: /
User-agent: Go-http-client
Disallow: /
User-agent: curl
Disallow: /
User-agent: Wget
Disallow: /
User-agent: wget
Disallow: /
User-agent: Nikto
Disallow: /
User-agent: sqlmap
Disallow: /
User-agent: Sqlmap
Disallow: /
User-agent: Acunetix
Disallow: /
User-agent: Netsparker
Disallow: /
User-agent: Nessus
Disallow: /
User-agent: masscan
Disallow: /
User-agent: ZmEu
Disallow: /
User-agent: Morfeus
Disallow: /
User-agent: PetalBot
Disallow: /
User-agent: petalbot-mobile
Disallow: /
User-agent: DataForSeoBot
Disallow: /
User-agent: SerpstatBot
Disallow: /
User-agent: SiteAuditBot
Disallow: /
User-agent: LinkpadBot
Disallow: /
User-agent: SemrushBot-SA
Disallow: /
User-agent: SemrushBot-BA
Disallow: /
User-agent: AhrefsBot-Image
Disallow: /
# ------------------------------------------------------------
# SITEMAP & FEED AUTO-DISCOVERY
# ------------------------------------------------------------
Sitemap: https://mhdarjunanta.com/sitemap.xml
Sitemap: https://mhdarjunanta.com/sitemap_index.xml
Sitemap: https://mhdarjunanta.com/image-sitemap.xml
Sitemap: https://mhdarjunanta.com/news-sitemap.xml
Sitemap: https://mhdarjunanta.com/video-sitemap.xml
# ------------------------------------------------------------
# AUTO SCHEMA: Feed Auto-Discovery
# ------------------------------------------------------------
# RSS: https://mhdarjunanta.com/rss
# Atom: https://mhdarjunanta.com/rss?format=atom
# JSON Feed: https://mhdarjunanta.com/rss?format=json
# robots.txt - Mhd Arjunanta
# Last Updated: 2026-10-19
# Auto-Generated by robots.php
User-agent: *
Allow: /
# AI CRAWLERS - DIIZINKAN
User-agent: GPTBot
Allow: /
User-agent: ChatGPT-User
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: PerplexityBot
Allow: /
Content-Signal: search=yes, ai-train=no
# ============================================================
# SELESAI - robots.txt Mhd Arjunanta
# Auto-Generated by robots.php | https://mhdarjunanta.com/robots.php
# ============================================================
США - Финикс - 191.101.104.95
США - Финикс - 191.96.144.102
Digital Energy Technologies Chile SpA
Digital Energy Technologies Chile SpA
HTTP/2 200
date: Fri, 06 Nov 2026 11:48:46 GMT
content-type: text/html; charset=UTF-8
vary: Accept-Encoding
set-cookie: mhdarjunanta_member=5obp76h65orr46qtjrgmlsto1k; path=/; secure; HttpOnly; SameSite=Lax
expires: Thu, 19 Nov 1981 08:52:00 GMT
cache-control: public, max-age=300, stale-while-revalidate=86400
pragma: no-cache
x-xss-protection: 1; mode=block
permissions-policy: geolocation=(), camera=(), microphone=(), payment=()
cross-origin-opener-policy: same-origin
cross-origin-resource-policy: same-origin
strict-transport-security: max-age=63072000; includeSubDomains; preload
content-security-policy: default-src 'self' https:; script-src 'self' 'unsafe-inline' 'unsafe-eval' https:; style-src 'self' 'unsafe-inline' https:; font-src 'self' https: data:; img-src 'self' data: https:; connect-src 'self' https:; media-src 'self' https:; object-src 'none'; base-uri 'self'; form-action 'self';
content-security-policy: upgrade-insecure-requests
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
referrer-policy: strict-origin-when-cross-origin
platform: hostinger
panel: hpanel
link: ; rel=\"stylesheet\"; type=\"text/xsl\"
link: ; rel=preload; as=video; type=video/mp4
link: ; rel=preconnect; crossorigin
link: ; rel=preconnect; crossorigin
x-robots-tag: index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1
x-auto-crawl: enabled
x-auto-index: ping-enabled
server: hcdn
x-hcdn-request-id: 530ce6f34e8e031e8f643cbe249abfef-imm-edge10
x-hcdn-cache-status: DYNAMIC
x-hcdn-upstream-rt: 0.302
Кнопка для анализа сайта в один клик, для установки перетащите ссылку на "Панель закладок"