Back to jobs4 to 5 years’ experience ‘hands-on’ leading highly technical data-centric software engineers, preferably in financial services, in support of Data Lake houses (e.g. AWS, Databricks, Snowflake), data virtualizations solutions (e.g. Starburst), modern data catalogs & data quality capabilities (e.g. Collibra), and Master Data Management (e.g. Informatica)
Proven AWS expertise (S3, EKS, Lambda, Glue, EMR, Redshift, S3, RDS/Aurora, Valkey/Redis, Step Functions, MSK/Kafka, Event Bridge).
Experience applying AI/ML to boost engineering productivity (e.g., GitHub Copilot, Amazon Q).
Strong Python for data engineering (Pandas, Spark/PySpark) and API Development (FastAPI or similar).
Design and build high-performance REST/GraphQL APIs with optimized query patterns, pagination, caching, and schema design to ensure low-latency, performant reads at scale.
Build ETL/ELT pipelines for batch and streaming; Kafka/MSK with schemas (Avro/JSON) and delivery semantics.
Optimize performance of object-based and relational datastores
Design applications with microservices and containerization mindset with ‘decoupling’ architectural principles
Caching with Valkey/Redis: TTL strategies, invalidation, key design.
CI/CD and IaC with GitHub, Git/GitHub workflows, ArgoCD.
Security and reliability: IAM, KMS, secrets, VPC networking, observability (OpenTelemetry), SLOs, cost governance.
Ability to solve complex data-driven requirements and triage towards defects and production bugs
Job Description
- AWS Certification strongly preferred