Spider Website

We want to scan the blogcatalog . com site and create a database of all the blog url's and their categories and/or tags. The spider would be set-up to go against the website - blogcatalog . com and scan through all the sites in the catalog. The directory is located at: <[login to view URL]> The spider would go into each of these directories and each blog in the category. The data that would be extracted for each blog would include: Blog Name, Blog url, All the categories, All the tags, Added date, Country, Language, Description, Total Views, Total Hits The output file would be the full blogcatalog. Additionally, if you could provide a program or describe a method that I could update this database/query on a regular basis, that would be worth extra funds. ## Deliverables 1) Run a spider against [login to view URL] 2) Pull all sites down from the catalog 3) Visit the site detail page in the spider and extract the information about every website in the catalog. 3a) Information includes: Blog Name, Blog url, All the categories, All the tags, Added date, Country, Language, Description, Total Views, Total Hits 4) Output all data to a tab delimited flat file 5) QA the file to make sure all the catalog has been output to the file. 6) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement). ## Platform Internet Explorer or Firefox. Windows or Unix.