I need an Hadoop Professional

Job Description:

I want the following

1. Read the attached paper. (Earlier stage for straggler detection and handling using combined CPU test and LATE methodology).

a) Implement the methodology used in the paper.

b) i) Write a program to detect/identify intermediate results (from the map phase) above 64Mb/128Mb and compare with (a) using same metrics to show which is a better method.

OR

ii) Improvement on the methodology used in (a) that will produce a better result.

2. Find a suitable paper on replication of data in hadoop mapreduce framework.

a) Implement the methodology used in the paper

b) i) Write a program to split identified intermediate results from (1 b(i)) appropriately into 64Mb/128Mb and compare with 2(a) using same metrics to show which is a better method.

OR

ii) Improvement on the methodology used in 2(a) that will produce a better result

3. Find a suitable paper on allocation strategy of data/tasks to nodes in Hadoop Mapreduce framework.

a) Implement the methodology used in the paper

b) i) Write a program to reallocate the splits from (2 (b(i)) above to nodes by considering the capability (probably the memory size) of the nodes to handle the splits allocated to them particularly in a heterogeneous environment where the capabilities of the machines are not the same and compare the result with 3(a) using similar metrics to show which is a better method.

OR

ii) Improvement on the methodology used in 3(a) that will produce a better result.

4. A certified dataset used to test the effectiveness of these programs.

5. A detailed explanation on the implementation and writeup on 1, 2, 3 that can be used as thesis objectives with graphical illustrations.

Habilidades: Hadoop, Big Data Sales, Java, Map Reduce, Python

Nº del proyecto: #34258767