Find Jobs
Hire Freelancers

A bot/spider to crawl the web for flash and non-mobile optimized websites

$30-250 USD

Cerrado
Publicado hace más de 7 años

$30-250 USD

Pagado a la entrega
We'd like a spider/bot to crawl the web for old sites that need an update. So we'd like the spider to crawl and filter sites according to some criteria: - 100% flash sites - Sites that don't have a mobile-friendly version (maybe looking for viewport media queries in css? ) - Sites that have an old date on the footer , like © 2014 (or lower, to be configurable) IT should output the results to a spreadsheet. So the results should look like: [login to view URL] Flash [login to view URL] No mobile version [login to view URL] older than XXXX Some important things: - Each of the three options should toggle on/off - This should be cascading, meaning first it checks if the site is flash. If it is, it adds the domain to the database and continue crawling. If not flash, it runs the second rule, if has a mobile version. And so forth. - Limit the results according to a domain filter (example: only .[login to view URL] domains, for targeting australian sites only) In your proposals please specify which language are you going to use, how the interface is going to be and provide as much information as possible. Similar projects that you've done in the past are welcome as well. Thanks M+W
ID del proyecto: 10832617

Información sobre el proyecto

10 propuestas
Proyecto remoto
Activo hace 8 años

¿Buscas ganar dinero?

Beneficios de presentar ofertas en Freelancer

Fija tu plazo y presupuesto
Cobra por tu trabajo
Describe tu propuesta
Es gratis registrarse y presentar ofertas en los trabajos
10 freelancers están ofertando un promedio de $248 USD por este trabajo
Avatar del usuario
Hi I work towards providing reliable, relevant and robust IT solutions at most competitive prices to my customers. I ensure 100% customer satisfaction so lets start Thanks
$237 USD en 10 días
4,8 (322 comentarios)
8,0
8,0
Avatar del usuario
Olá, Meu nome é Rui Pimentel, mais conhecido recentemente pelo meu envolvimento em projetos de crawling a sites com netflix, nike, supremenewyork e tantos outros trending neste momento. Eu li completamente a descrição do seu projeto e percebo perfeitamente do que necessita, no entanto tenho algumas preocupações que gostava de discutir consigo. Se fosse você a fazer input dos domínios de websites seria um simples approach, mas pelo que percebo que você procura, teremos de recorrer a pesquisas no google, e temo que tal possa ser complicado se não for possível a utilização de vários IPS ou proxies, ou na pior das hipóteses, a necessidade de remover captchas manualmente à medida que vão aparecendo no processo de pesquisa. Aguardo um contacto da sua parte, espero sinceramente poder ajudá-lo a concretizar este projeto, tenho a certeza que ficará bastante satisfeito com o resultado. Cumprimentos, Rui
$147 USD en 3 días
4,9 (35 comentarios)
5,8
5,8
Avatar del usuario
Hi, I have read your specification and can definitely help you with your requirements. Please open a line of communication to discuss the project so that i can make it as per your vision and I am ready to start this project. waiting for your quick and positive response. Best regards
$155 USD en 10 días
4,8 (48 comentarios)
5,5
5,5
Avatar del usuario
I went through the description and found the project interesting. I can make a spider for you to run checks on websites (three as per your specs: flash, mobile support and old footer date) and output you a spreadsheet with the list of sites that fail them. I can code that using python (scrapy or anything else) or ruby as per further discussion. The settings can either be command line, given via inputs, via text files, in code or whatever other way suits you. I have previously written crawlers in scrapy and raw python/ruby, most of it is closed source so can't give you a reference.
$250 USD en 7 días
4,7 (5 comentarios)
4,4
4,4
Avatar del usuario
I am expert in python for Web scraping , API integration and lexical analysis. Also a good hand in web application designing using dijango and flask. I am a Oracle Certified professional (OCP) with experience in Oracle,MySQL, SQL Server and MongoDB. I can help you with your requirements. Please initiate online chat.. so we can proceed with discussion
$245 USD en 7 días
4,5 (10 comentarios)
4,2
4,2
Avatar del usuario
Hey. I got acquainted with you provided Info. The project is very interested. I understand perfectly the essence of the task. There is a lot of work experience, have experience in hozhih projects. I would be happy to implement your project into reality. There is a great desire to work. I propose to discuss all the details of your project in more detail, deadlines, budget. I would like to look at the detail technical task. I look forward to our cooperation. I'll wait for your answer in the chat.
$555 USD en 14 días
5,0 (7 comentarios)
3,7
3,7
Avatar del usuario
Please see my reviews
$250 USD en 5 días
5,0 (3 comentarios)
3,5
3,5
Avatar del usuario
Hi, I'm a java developer with big experience in web and console application development. Can make this bot using java that will parse pages for that info.
$122 USD en 4 días
5,0 (2 comentarios)
2,7
2,7

Sobre este cliente

Bandera de BRAZIL
São Paulo, Brazil
5,0
1
Miembro desde jul 18, 2014

Verificación del cliente

¡Gracias! Te hemos enviado un enlace para reclamar tu crédito gratuito.
Algo salió mal al enviar tu correo electrónico. Por favor, intenta de nuevo.
Usuarios registrados Total de empleos publicados
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Cargando visualización previa
Permiso concedido para Geolocalización.
Tu sesión de acceso ha expirado y has sido desconectado. Por favor, inica sesión nuevamente.