Job Description
Essential Job Responsibilities
The SkillBridge intern will train as a Data Engineer with Enlighten, reporting to a designated Enlighten Supervisor, with the goal of learning Enlighten’s Enterprise approach to design, build, and optimize Big Data Platform (BDP) systems for data collection, storage, access, and analytics at scale. The intern will be assigned special projects as needed.
Desired End State (3-4 month target)
-
At the end of four months, the intern will be able to understand basic data engineer concepts. Data engineers, as part of enterprise data analytics teams, manage, optimize, oversee, and monitor data retrieval, storage, and distribution throughout the BDP enclaves.
-
The intern will develop a good understanding of all critical tasks to successfully conduct data engineer objectives:
-
Data acquisition
-
Develop data set processes (Extract, transform, load (ETL))
-
Use programming language and tools
-
Identify ways to improve data reliability, efficiency, and quality
-
Prepare data for predictive and prescriptive modeling
-
Use data to discover tasks that can be automated
-
- Enlighten will benefit from the military background of the SkillBridge intern and considers the SkillBridge internship an overall positive experience.
Assumptions/Restrictions
- If candidate is selected and approved, SkillBridge Intern can travel in conjunction with this internship; Enlighten will fund all travel costs.
- SkillBridge Intern will be available during core hours for critical meetings and training.
Training Plan
Phase 1 Basics
- Week 1: In processing, Introductions/office familiarization
- Week 2: Intro to the Big Data Platform (BDP) technologies
- Week 3: Intro to BDP analytics and applications
- Week 4: Intro to Army and Air Force Cyber data ETL process
Phase 2 Parser Overview
- Week 5: Data Acquisition, Taxonomy/Schema
- Week 6: Review existing parsers and deployment flows; understand existing transforms available for use in parsers.
- Weeks 7-8: BDP technologies - Kafka and Nifi workflows; Establish a development environment for BDP parser development
Phase 3 Parser Development, Deployment, and Testing
- Weeks 9-11: Modify Kafka and Nifi workflows; modify existing parser schema and transforms to alter ingest pipeline.
- Weeks 12-13: Understand how to access logs to identify errors; understand common parser development errors; troubleshoot errors.
Additional Goals
- Project management