Back to jobs
Job Description
What you will be doing
- Manage and orchestrate complex data pipelines using Azure Data Factory (ADF).
- Design, implement, and optimize data transformation processes using Python libraries (Pandas, Numpy, PySpark).
- Develop, maintain, and troubleshoot scalable data workflows on Databricks.
- Utilize Azure Data Explorer (ADX) for real-time data analysis and complex queries.
- Create and optimize SQL queries to extract, transform, and load data.
- Automate routine data tasks with Automation Runbooks and Azure Functions.
- Design and manage scalable data storage solutions using Azure Data Lake.
- Write and execute unit tests to ensure the integrity and quality of data.
- Utilize CI/CD pipelines using Azure DevOps for seamless integration and deployment.
- Optimize Azure usage to manage and reduce costs effectively.
- Ensure to work in compliance with Hydro Quality system, HSE regulations, policies and standardized operating procedures.
- Perform all other tasks upon the instructions of the superior in charge which may be necessitated by the operations of the related unit and which do not conflict with any applicable laws, statutory provisions and company rules.
- Compliance with area-specific customer requirement
What will make you successful?
- 3+ years of experience in data engineering or a similar role.
- Microsoft Certified: Azure Data Engineer Associate or similar certification is a plus.
- Strong problem-solving skills and attention to detail
- Excellent communication and teamwork abilities.
- Master/Bachelor’s degree in Computer Science, Information Technology, or a related field.
- Fluent in English is mandatory. Any other language is a plus
- Proficiency in using Azure Data Factory to create and manage data pipelines.
- Proven experience in data transformation using Python (Pandas, Numpy, PySpark).
- Advanced knowledge of SQL and query optimization.
- Understanding of Azure Data Explorer (ADX) for data analysis.
- Hands-on experience with Databricks for big data processing and collaborative data science workflows.
- Expertise in designing and managing data storage with Azure Data Lake.
- Experience in automation with Runbooks and Azure Functions.
- Familiarity with CI/CD pipelines using Azure DevOps.
- Experience in writing and executing unit tests for data quality assurance
- Experience using GitHub for version control, collaboration, and code review.
- Knowledge of Infrastructure as Code (IaC) tools (e.g., Terraform, ARM templates) is a plus.
- Awareness of Azure pricing models and cost management practices is a plus.
What we offer you
- Working at the world’s only fully integrated aluminum and leading renewable energy company
- Diverse, global teams
- Flexible work environment/home office
- We provide you the freedom to be creative and to learn from experts
- Possibility to grow with the company, gain new certificates
- Attractive benefit package