Senior Data Engineer (Databricks + Snowflake)

Job Summary

We are seeking a highly skilled Senior Data Engineer with strong expertise in Databricks, Snowflake, and modern data engineering practices. The ideal candidate should have hands-on experience in building scalable batch and streaming data pipelines, optimizing Spark jobs, and working with large-scale distributed data systems.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines (batch & streaming)
• Build and optimize data processing workflows using PySpark and SQL
• Work with Databricks for data processing, cluster management, and job orchestration
• Implement and manage data pipelines using Delta Lake architecture
• Integrate data from multiple sources including Kafka and cloud storage systems
• Ensure data quality, governance, and reliability across pipelines
• Collaborate with cross-functional teams for data platform enhancements
• Troubleshoot and debug Spark jobs for performance optimization

Required Skills

• Strong experience in Databricks and Snowflake
• Proficiency in Python, PySpark, and SQL
• Experience with Apache Airflow for workflow orchestration
• Hands-on experience with Kafka (real-time streaming pipelines)
• Experience with GitHub-based CI/CD deployments
• Strong understanding of Spark internals, optimization techniques, and debugging
• Experience in Databricks cluster (DBX) management
• Hands-on experience with Delta Lake pipelines
• Knowledge of Databricks Asset Bundles (DAB)

Good to Have

• Experience with cloud platforms (AWS / Azure / GCP)
• Knowledge of data lake architectures and medallion architecture
• Experience in handling large-scale data (TB/PB level)

Soft Skills

• Strong communication and collaboration skills
• Ability to work in a fast-paced environment
• Problem-solving mindset

Qualifications

• Bachelor’s degree in Computer Science, Engineering, or related field

Notes

• Candidate should be comfortable working on both batch and streaming data pipelines
• Strong focus on performance tuning, scalability, and reliability is expected