Design and build scalable, cloud-native data platforms from greenfield to production
Implement near-real-time ingestion pipelines using event-driven patterns
Define and enforce platform standards, including Data Lake / Lakehouse principles, medallion architecture, and data contracts
Refactor and optimise existing Spark and PySpark scripts for performance and maintainability
Introduce best practices for code quality, testing, and CI/CD across data pipelines
Drive adoption of AI tooling and agentic workflows within the data engineering team
Ensure data quality, observability, and reliability across all pipelines and platforms
Develop self-service tooling and microservices to simplify platform usage for other teams

5+ years of professional experience in Data Engineering
Strong Python and SQL development skills for pipeline development and optimisation
Proficiency in Apache Spark / PySpark, including query optimisation and performance tuning
Hands-on experience with Databricks (preferred) or Snowflake
Experience with at least one major cloud provider: Azure (preferred), AWS, or GCP
Experience with stream processing technologies (Kafka, Spark Structured Streaming)
Solid understanding of ETL/ELT patterns, data modelling (dimensional, Data Vault), and data warehousing
Experience with orchestration tools (Apache Airflow, Azure Data Factory, or equivalent)
Knowledge of Infrastructure as Code (Terraform or equivalent)
Understanding of production-grade system requirements: reliability, scalability, observability, and performance
Upper-Intermediate English level

WILL BE A PLUS

Familiarity with RAG pipeline design and LLM integration patterns
Knowledge of data governance frameworks and tools (Unity Catalog, Apache Atlas, or similar)
Experience with dbt for data transformation and modelling
Familiarity with MLflow, Feature Stores, or ML platform integration

PERSONAL PROFILE

Senior Data Engineer

Job Description