Application needed to crawl & gather website data

We need a Windows application that we can leave running on a PC that will crawl a business directory on the web We need to be able to specify limits (e.g. just gather .[login to view URL] domains, or just a specific category in the directory) and compile the following data. It needs to log the page name, URL, and results for the following criteria.... 1) Does the site use frames (y/n) 2) Does the page have a Doctype declaration (y/n) + what is it (html, xhtml, strict etc) 3) Does the page code validate (can integrate an open source validator) 4) What is the google page rank of the page (this is optional) 5) What was the directory category 6) A way of determining (looking for ideas) of whether it is built using CSS+divs, or tables for layout. Please state as much detail as possible about how you will achieve this, how the logged data can be accessed (ideally as an exportable excel spreadsheet from the app.), and post examples if you have done similar things.