Requirements

Expected technologies

Apache Spark

Apache Kafka

SQL

Microservices

CI/CD

Java

Data

Databricks

Optional technologies

Azure

DDD

Our requirements

• Have 4+ years of Apache Spark experience and have faced various data engineering challenges in batch or streaming
• Have an interest in stream processing with Apache Spark Structured Streaming on top of Apache Kafka
• Have experience leading technical solution designs
• Have experience with distributed systems on a cloud platform
• Have experience with large-scale systems in a microservice architecture
• Are familiar with Git and CI/CD practices and can design or implement the deployment process for your data pipelines
• Possess a proactive approach and can-do attitude
• Are excellent in English and Polish, both written and spoken
• Have a higher education in computer science or a related field
• Are a team player with strong communication skills

Optional

• Apache Spark Structured Streaming
• Azure
• Domain Driven Development
• Docker containers and Kubernetes
• Message brokers (i.e. Kafka) and event-driven architecture
• Agile/Scrum

Your responsibilities

• Develop and maintain the leading IT solution for the energy market using Apache Spark, Databricks, Delta Lake, and Apache Kafka
• Have end-to-end responsibility for the full lifecycle of features you develop
• Design technical solutions for business requirements from the product roadmap
• Maintain alignment with architectural principles defined on the project and organizational level
• Ensure optimal performance through continuous monitoring and code optimization.
• Refactor existing code and enhance system architecture to improve maintainability and scalability.
• Design and evolve the test automation strategy, including technology stack and solution architecture.
• Prepare reviews, participate in retrospectives, estimate user stories, and refine features ensuring their readiness for development.