Java Selenium Junior task: create simple page crawler as graph

Your task is to develop a web page crawler in selenium, which takes the given URL as starting page and crawls all URLs of same second&top-level domain of the website.

Means crawl all pages and fetch:

- image links

- links itself

follow recursively all links and buttons

fetch of each of the given links and images:

- the link-url

- all html-attributes

so basically you need to

- collect all URLs

- iterate over all, ensure not to do the crawling-work multiple times

- use PageObject pattern and other recommended from seleniumHQ

Your input of the function will be a URL and a limiting regex to limit the found URLs for a given regex.

The Result of the function will be a jgrapht graph with model classes, which represent the graph-structure of the website.

What is NOT needed:

- a UI (not required)

- a service architecture (like spring or JEE)

- persistency

simply a function, which executes selenium and collects the given details

To execute the test, create a JUnit-runner, which runs the application

What are our requirements?

- your code passes checkstyle, pmd and findbugs (we will share you a git repo with eclipse settings)

- JDK11

- create a model class representing the input of your function

- create a service class implementing the logic

- create a unit test, which tests the service class

- we do NOT need a UI, we only need the model + service method to access the logic via API or via JUnit

- if you need libs beside of selenium or jgrapht, apache commons are fine. Other libs NEED prior clearance

- the runtime is JRE (no JavaEE nor Spring-container)

What are our requirement?

- before we hire you, we need the signed NDA: [login to view URL]

Outlook

- if you do a good job on supporting our team, we are open to integrate you into regular work

What proposals do we SKIP?

- you are unwilling to sign the NDA: [login to view URL]

What is our budget?

we do not disclose our budget nor planned hourly rate. Offer us your best bid.

This job is created with a placeholder budget/rate. So ignore our rates/budgets and post your best bid.

Eclipse Java JUnit Selenium Webdriver Desarrollo de software

Nº del proyecto: #23802959

Sobre el proyecto