User-agent: Magus Bot 1.0 Disallow: / User-agent: MagusBot Disallow: / User-agent: Magus Bot Disallow: / User-agent: Magus Disallow: / User-agent: Ezooms Disallow: / User-agent: Magus+Bot+1.0 Disallow: / User-agent: 008 Disallow: / User-agent: AhrefsBot Disallow: / User-agent: SISTRIX Crawler Disallow: / User-agent: Sosospider Disallow: / User-agent: ScoutJet Disallow: / User-agent: Blekkobot Disallow: / User-agent: panscient.com Disallow: / User-agent: b2w/0.1 Disallow: / User-agent: BotRightHere Disallow: / User-agent: DittoSpyder Disallow: / User-agent: Download Ninja Disallow: / User-agent: EmailCollector Disallow: / User-agent: EmailSiphon Disallow: / User-agent: EmailWolf Disallow: / User-agent: ExtractorPro Disallow: / User-agent: Fetch Disallow: / User-agent: Frontpage Disallow: / User-agent: grub-client Disallow: / User-agent: HTTrack Disallow: / User-agent: ieautodiscovery Disallow: / User-agent: k2spider Disallow: / User-agent: larbin Disallow: / User-agent: libwww Disallow: / User-agent: linko Disallow: / User-agent: Microsoft.URL.Control Disallow: / User-agent: Morfeus Fucking Scanner Disallow: / User-agent: MJ12bot Disallow: / User-agent: MSIECrawler Disallow: / User-agent: NPBot Disallow: / User-agent: Offline Explorer Disallow: / User-agent: ProWebWalker Disallow: / User-Agent: psbot Disallow: / User-agent: sitecheck.Internetseer.com Disallow: / User-agent: SiteSnagger Disallow: / User-agent: Teleport Disallow: / User-agent: TeleportPro Disallow: / User-agent: Teleport Pro/1.29 Disallow: / User-agent: Telesoft Disallow: / User-agent: vscooter Disallow: / User-agent: WebBandit Disallow: / User-agent: WebCopier Disallow: / User-agent: WebCopier v4.0 Disallow: / User-agent: Web Downloader/6.9 Disallow: / User-agent: WebReaper Disallow: / User-agent: WebStripper Disallow: / User-agent: WebZip Disallow: / User-agent: WebZip/4.0 Disallow: / User-agent: wget Disallow: / User-agent: ZyBORG Disallow: / User-agent: Baiduspider+(+http://www.baidu.com/search/spider.htm) Disallow: / User-agent: Baiduspider+(+http: Disallow: / User-agent: Baiduspider+ Disallow: / User-agent: Baiduspider Disallow: / User-agent: Baidu Disallow: / User-agent: Btsearch Disallow: / User-agent: ConveraCrawler Disallow: / User-agent: DA 7.0 Disallow: / User-agent: PHP version tracker Disallow: / User-agent: favorstarbot Disallow: / User-agent: findlinks Disallow: / User-agent: Jakarta Commons-HttpClient Disallow: / User-agent: PlantyNet_WebRobot_V1.9 Disallow: / User-agent: SecretBrowser Disallow: / User-agent: Pete-Spider Light Disallow: / User-agent: Python-urllib Disallow: / User-agent: Jobs Crawler Disallow: / User-agent: LWP::Simple Disallow: / User-agent: Poirot Disallow: / User-agent: larbin_2.6.3 Disallow: / User-agent: SeznamBot Disallow: / User-agent: ichiro Disallow: / User-agent: LapozzBot Disallow: / User-agent: ConveraMultiMediaCrawler Disallow: / User-agent: LinksManager.com Disallow: / User-agent: lwp-trivial Disallow: / User-agent: htdig Disallow: / User-agent: Gaisbot Disallow: / User-agent: MJ12bot Disallow: / User-agent: NG Disallow: / User-agent: NRSBOT 5.0 Disallow: / User-agent: noxtrumbot Disallow: / User-agent: RDBA.net URLCHK Disallow: / User-agent: Sirketcebot Disallow: / User-agent: StackRambler Disallow: / User-agent: Szukacz Disallow: / User-agent: Magus Bot 1.0 Disallow: / User-agent: TCF Disallow: / User-agent: Twiceler Disallow: / User-agent: MJ12bot Disallow: / User-agent: Teleport Pro Disallow: / User-agent: OutfoxBot/0.5 Disallow: / User-agent: Web Scraper Plus+ Disallow: / User-agent: Wget Disallow: / User-agent: Wget/1.10.2 Disallow: / User-agent: Website eXtractor Disallow: / User-agent: Yandex/1.01.001 Disallow: / User-agent: psbot Disallow: / User-agent: WebAlta Crawler Disallow: / User-agent: Yandex Disallow: / User-agent: GurujiBot Disallow: / User-agent: GurujiBot/1.0 Disallow: / User-agent: OpenAcoon v4.0.3 Disallow: / User-agent: VoilaBot BETA 1.2 Disallow: / User-agent: OpenAcoon v4.0.5 (www.openacoon.de) Disallow: / User-agent: DBLBot/1.0 Disallow: / User-agent: * Allow: /arg Allow: /bol Allow: /bra Allow: /2011card Allow: /can Allow: /chi Allow: /clap Allow: /col Disallow: /connex Allow: /director Allow: /disasters Allow: /dor Allow: /ecu Allow: /els Allow: /english/gov/ Allow: /English/SHA/coredata/tabulator Allow: /Spanish/SHA/coredata/tabulator Allow: /gut Allow: /guy Allow: /hai Allow: /hon Allow: /hq Allow: /ict4health Allow: /jam Allow: /journal Allow: /mex Allow: /nic Allow: /nutricionydesarrollo Allow: /PAHO-USAID Allow: /pan Allow: /panaftosa Allow: /par Allow: /per Allow: /prfep Allow: /pur Allow: /sur Allow: /trt Allow: /uru Allow: /us Allow: /ven Allow: /world-health-day-2014 Disallow: *