Data Engineer/Consultant (Senior/Staff)

We are #VLteam – tech enthusiasts constantly striving for growth. The team is our foundation, that’s why we care the most about the friendly atmosphere, a lot of self-development opportunities and good working conditions. Trust and autonomy are two essential qualities that drive our performance. We simply believe in the idea of ​​“measuring outcomes, not hours”. Join us & see for yourself!About the roleThe majority of these roles will be at the forefront of client collaboration and building VL positions in the industry (spearheading projects).You will work closely and directly with a different specialist from the client side. Collaborate with stakeholders to define requirements, develop data pipelines and data quality metrics.You will participate in defining the requirements and architecture for the new platform, implement the solution, and remain involved in its operations and maintenance post-launchYour work will also introduce data governance and management, laying the foundation for accurate and comprehensive reporting that was previously impossible.Build data ingestion & processing pipelines. All of the above with a strong focus on the customer’s needs.Flexibility in action and the ability to overcome obstacles are highly valued in this role.View available projects:ProjectJetBrainsProjectt scopeThe client is introducing Atlan as a new internal Data Catalogue solution and uses Glean as a company-wide unified search platform for thousands of employees.To ensure a smooth transition from our existing Knowledge Base and OpenMetadata setup, we need to index Atlan assets into Glean so that metadata for databases, tables, metrics, and reports is easily discoverable through search.Tech stackPython,  System & Data Integration, Kubernetes, System design, Infrastructure mindsetSkillsWe’re looking for a Data Platform Engineer with experience in data platforms and system design at scale. We expect a track record in designing integration architectures for external systems and streamlining data migration/ingestion.As a Data Platform Engineer, you will design and implement a solution that: Periodically indexes Atlan metadata assets into Glean, runs on a configurable schedule (hourly/daily), is production-ready, observable, and maintainable by our DevOps team after handover.Moreover, ensure compliance and data governance at the appropriate level in line with the company’s standards.What we expect in generalA proactive approach and flexibility in action were a mustVery good command of English (written and spoken)Hands-on experience with PythonProven experience with data warehouse solutions (e.g., BigQuery, Redshift, Snowflake)Experience with Databricks or data lakehouse platformsStrong background in data modelling, data catalogue concepts, data formats, and data pipelines/ETL design, implementation and maintenanceAbility to thrive in an Agile environment, collaborating with team members to solve complex problems with transparencyExperience with AWS/GCP/Azure cloud services, including: GCS/S3/ABS, EMR/Dataproc, MWAA/Composer or Microsoft Fabric, ADF/AWS GlueExperience in ecosystems requiring improvements and the drive to implement best practices as a long-term processExperience with Infrastructure as Code practices, particularly Terraform, is an advantageProactive approachDon’t worry if you don’t meet all the requirements. What matters most is your passion and willingness to develop. Apply and find out!A few perks of being with usBuilding tech communityFlexible hybrid work modelHome office reimbursementLanguage lessonsMyBenefit pointsPrivate healthcareTraining PackageVirtusity / in-house trainingAnd a lot more!Apply now