A small python script that use scrapy to web scrap and output xml/csv files with media pipeline options that can be stored in s3 is wanted.
The script with run in ssh on ubuntu server that has been configured and is working fine.
The job is about an hr.
See scrapy documentation for the map.