Project 1

A Java application ETL batch processor. Data should be parsed from a CSV or JSON file, sent to an Apache Spark cluster using the Java Spark RDD library for analysis, and its result persisted to a SQL database. Prior queries should be presented over a Java Servlet using HTTP.

Features

• Documentation
• Unit testing
• Logging
• Batch Processing
• SQL Data Persistance
• HTTP Server API
• Containerization
• Cloud Hosting

Tech Stack

• JavaSE 8:
  • Functional Programming
  • Streams
• JavaEE 7:
  • HttpServlet
• Apache Spark RDD
• Maven 3
• JUnit 5
• Docker
• Amazon Web Services:
  • EC2
  • RDS
  • S3
  • EMR
• Git SCM (on GitHub)

Presentation

• 2 minute slide deck
• 5-10 minute live demonstration