The spiders that visit your site consumes much of your bandwidth? If you want to control the robots or spiders that visit your site, robots-txt generator is the solution. It is very common that the webmaster of each site spends a lot of
RoboGen is a visual editor for Robot Exclusion Files.
"Linking The Web" is a free utility that regularly monitors all backlinks to your sites. If some of the links don't exist, contain wrong anchor, are covered by a "nofollow" flag or hidden by robots.txt then you will be immediately notified by email. Also
WebSite Auditor Enterprise is professional SEO software for detailed site audits, landing page reviews, robots.txt creation, Sitemap generation, easy client reporting, and more! Check out convenient delivery options for our responsive-design reports!
Crawl a website with ActiveX Spidering component. Advanced features include caching, 'avoid' patterns, and robots.txt compliance. Spider ActiveX Component for Crawling the Web: -Cache pages so future crawls can fetch from cache; -Robots.
Indexing of the title, description, keywords and full content of all HTML, ASP and TXT documents using a recursive call to all pages in a web site and sub folders excluding those folders listed in the robots.txt file. Simple search of the index for
A desktop SEO Audit Tool, a Visual approach to make SEO tasks quick and easy. Crawl, Spot on-site issues at a glance, Take page screenshots and see where the fold lays, dominate robots.txt and XML Sitemaps. A comprehensive SEO suite, all Visual!
Sitemap Generator to generate html or txt sitemap. HTML Sitemap can be a valuable page to your website for search engines, vistors, and web masters.
Acquiring your potential customers e-mail addresses is a cornerstone of successful e-marketing. This innovative software allows extracting e-mail addresses from your target-audience relevant web sites! Stop wasting money! Download free trial now!
Create text, HTML, RSS and XML sitemaps for search engines like Google and Yahoo. Rich template support for HTML sitemaps. Supports splitting and compressing XML sitemaps. Many website crawl options. View all broken and redirected links in website.
Sitemap Writer Pro is an easy- to -use program, fast and efficient which offers creation of sitemaps with millions of URLs and 6 different types of sitemaps (Standard Sitemap, Google News, Google Video, Google Code Search, Google Geo, Google Mobile).