Requirements

Expected technologies

Microsoft Azure

Python

PySpark

Data Lake

Databricks

Our requirements

• Hands-on experience with Azure cloud services, especially Data Lake and Databricks
• Proficiency in Python and PySpark for large-scale data processing
• Ability to develop Python applications, write automation tasks
• Experience with version control (e.g., Git) and CI/CD practices
• Good command of English

Your responsibilities

• Designing, developing, and maintaining data pipelines using Azure Databricks and Delta Lake
• Implement data application in Python with transformations and aggregations using PySpark: ingestion, data historization, integration, filtering
• Developing a data quality framework, observability, and monitoring tools
• Collaborate with stakeholders to understand data requirements and ensure data quality
• Optimize performance and cost-efficiency of data workflows in Azure Databricks