Completly free spider or crawler for websites you will find many on the World Wide Web but very few of them provide both, a basic analysis functionality as well as a clear presentation of the data. The seoBOXX WebsiteSpider offers you completely free very fast and efficient spider-/crawl functionality and determines a lot of information of your domain in a very short time. Here both, the basic data of the domain and the data of each URL of the selected domain are analyzed and the data obtained is stored in a database file. The efficient MTWS-CoreEngine gives the WebsiteSpider the opportunety to run multiple analyzes in parallel and to produce the required analysis results quickly and efficiently.
The problems of crawling or spidering lies firstly in the crawl speed and the other in the detail of the data obtained and their storage and optical presentation. The basis of the site is the WebsiteSpider is the MTWS-CoreEngine (MultiThreadedWebsiteSpider), the basic component of the WebsiteSpider and most of the other products. The MTWS-CoreEngine uses multiple threads to download the URLs source, analyses it, stores the collected data in a database and fills the URL list of the domain with the new found URLs. This fundamental component works very quickly and efficiently with its optimized source code and includes several algorithms for error correction for the accesses to the web servers, which include more and more mechanisms to quickly and consecutive regard multiple hits from an IP address as an attack.
The WebsiteSpider generates three different report types. The contents of the reports are not variable in this free version. The website spider generates, just as the WebsiteAnalyser, the following reports/report types:
seoBOXX
Andreas Soraru
Wagenfeldstr. 1
59379 Selm
Deutschland
Web: http://seoboxx.com
EMail:
Kontakt: Kontaktformular