Job Description
Everforth ECS is seeking a highly skilled Senior Data Engineer with extensive experience designing, building, and maintaining data platforms and pipelines for large-scale, enterprise applications in support of the Department of Homeland Security. Tools necessary for excelling in this role include SQL (PostgreSQL or MySQL preferred), Elasticsearch/OpenSearch, data modeling and transformation frameworks, and DevSecOps practices in the Cloud.
This role will focus heavily on modernizing legacy data architectures and re-architecting the application’s data layer, shifting from Elasticsearch to PostgreSQL or MariaDB as the system of record for transactional data. You will play a critical role in evolving scoring models, intelligence modifiers, and capability gap logic while ensuring alignment and consistency between the new relational database and Elasticsearch.
Key Responsibilities:
- Design and architect robust, scalable data models, schemas, indexes, and stored procedures in a relational database to support transactional workloads and complex business logic.
- Lead the migration of the system of record from Elasticsearch to a relational database, including data modeling, transformation, validation, and cutover strategies.
- Develop and maintain data pipelines and integration processes to keep the database and Elasticsearch aligned for search and analytics use cases.
- Implement and evolve scoring models, intelligence modifiers, and capability gap logic in the data layer to support mission-critical decision support.
- Optimize database performance, including indexing strategies, query tuning, and partitioning to support high-volume, low-latency operations.
- Define and enforce data quality, data validation, and data governance standards across legacy and modernized components.
- Collaborate with application engineers to integrate APIs and services with the underlying data models and stored procedures, ensuring seamless functionality and consistent data semantics.
- Provide technical guidance and solutions for legacy data system modernization while minimizing downtime and operational risk during migration.
- Identify and remediate data-related technical debt in older systems while transitioning to modern data engineering patterns, tools, and frameworks.
- Integrate and operate CI/CD pipelines used for building, testing, and deploying data services and schema changes in cloud environments.