Scrapy Jobs

I need the full textual content of a set of websites extracted and handed back to me in a clean, machine-readable dataset. The task is limited to scraping website content (no product feeds or user information), but within that scope I want every publicly available page: headings, body copy, meta tags, image alt text—anything that helps recreate or analyse the site’s structure. Here is what I’m expecting: • A repeatable script or notebook (Python with BeautifulSoup, Scrapy, Selenium, or a comparable tool is fine) that I can rerun later on new URLs. • The scraped data delivered in a single file per run—CSV, Excel, or JSON all work for me, so use whichever format your pipeline handles best; just keep it consistent. • Respectful crawling: follow , a...

I’m looking for a fast, reliable way to pull company contact details straight from the official exchange sites — and nseindia.com. The goal is to capture every listed company’s publicly annoucements and save them into a database. The script should run everyday and collect below: The scripcode from NSE and BSE Categoryname of the annoucement The weblink to the announcement bseindia and nseindia may have their own api which can be used. Links to NSEIndia: (SME tab) Link To BSEIndia Annoucements: Deliverables • A runnable Python script which gives insert SQLs to to updated in DB. I’ll verify the work by cross-checking a random sample of companies against the live pages, so accuracy and field completeness matter more than raw speed.

I need a one-time extraction of contact details from a UK-based website. The pages are publicly accessible, but they sit behind basic pagination and a search filter, so the process may require a crawler that can navigate multiple result pages before harvesting the data. Please capture the following items and export them to a single, tidy Excel workbook (.xlsx): • Email addresses • Phone numbers (normalised to +44 format) • Postal addresses A clean header row, one record per line, and the removal of duplicates are essential. Basic validation—such as checking email syntax and trimming whitespace—will save me time later. I’m comfortable if you choose Python with BeautifulSoup, Scrapy, or Selenium, or another language you prefer, provided the result...

I need a small application that checks a chosen info website every hour, captures any freshly posted content, and immediately publishes those findings inside a password-protected web portal. The exact data points we collect—new arrestee, charge , bond amounts, or something else—can be finalised together once development begins, but the program must recognise what is new since the previous run and avoid duplicates. The workflow I have in mind is straightforward: 1. An automated scraper (Python with BeautifulSoup, Scrapy, or a comparable solution is fine) runs on an hourly schedule. 2. Scraped items are stored in a lightweight database so they can be compared against previous runs and displayed chronologically. 3. A simple web interface, secured by login credentials, sho...

I need all publicly visible text, images, and product-specific details extracted from a single website and handed back to me in a clean CSV file. This is a one-off job, not an ongoing crawl, so the script only needs to run once—though I’d like to keep the code for my records in case I decide to repeat the process later. The CSV should include columns for every piece of information that appears on the product pages (titles, descriptions, prices, image URLs, SKU or ID codes, category paths, and any other attributes you’re able to capture). All referenced images must be downloaded as files and organised in a folder structure that mirrors the CSV paths. Key points • Accuracy matters: fields should align consistently across rows. • No duplicates or missing reco...