Join our Team

Senior Data Engineer

Data Team/Full-time

Amsterdam, Netherlands or Valletta, Malta

Company Description:
Etpa is a designated European Power Exchange for the short-term electricity market. Our state-of-the-art platform is a key enabler for the transition to renewable energy. We provide access to the spot-trading to many new players in the markets, including battery companies, Energy management systems, asset owners and utilities. We outperform our competitors in terms of speed, cash management, scalability, cost and IT connectivity. Our current portfolio of exchanges includes Intraday (XBID), Congestion management (GOPACS) and day after trading (expost). We are rapidly expanding our geography and products, introducing XBID to Germany, Austria and Belgium this year.

Job Description:
We are seeking a highly skilled and motivated pragmatic Data Engineer to join our data team. As a Data Engineer, you will be responsible for designing, developing, and maintaining our data architecture and infrastructure. The ideal candidate should have a strong background in Python, RDBMS, Apache Spark, Apache Hadoop (or a Hadoop-based tool such as EMR), AWS Athena, and experience with tools like Dagster (or similar), Superset (or similar), Kubernetes, and a proven track record in building robust (but pragmatic) data pipelines, data modelling, and data integration solutions. Additionally, expertise in AWS, Git, infrastructure management, and a focus on optimizing and monitoring data pipelines for performance are essential for success in this role.

Responsibilities:

• Design pragmatical solution on the axis of (quick) business needs vs technological optimised (data) challenges.
• Manage and execute the projects of the data team and be your own PO facilitating your stakeholders.
• Design, develop, and maintain scalable and efficient data architecture and infrastructure.
• Implement and optimize data pipelines for the extraction, transformation, and loading (ETL) of data.
• Collaborate with cross-functional teams and stakeholders to understand data requirements and deliver effective solutions.
• Develop and implement data modelling strategies to support business analytics and reporting needs.
• Utilize Python for scripting and programming tasks related to data engineering.
• Implement and manage data security measures in compliance with industry standards.
• Utilize Apache Spark, Hadoop-based technologies (e.g., EMR), and AWS Athena for distributed data processing.
• Work with workflow management tools like Dagster for orchestrating complex data pipelines.
• Utilize visualization tools like Superset for data exploration and analysis.
• Deploy and manage data pipelines in the AWS cloud environment.
• Work with containerization and orchestration platforms like Kubernetes.
• Monitor and optimize data pipelines for performance and efficiency.
• Provide expertise in infrastructure management related to data engineering processes.
• Version control using Git for codebase management.
• Stay updated on industry trends and technologies, contributing to continuous learning within the team.

Qualifications:

• Bachelor's degree in Computer Science, Information Technology, or a related field.
• Proven experience as a Data Engineer or similar role with a strong pragmatic drive to delivery projects quick and step-by-step (good is good enough approach).
• Strong programming skills, particularly in Python.
• Proficiency in relational databases (e.g., MySQL, PostgreSQL) and experience with data modelling.
• Hands-on experience with Apache Spark and AWS Athena.
• Expertise in designing and implementing data integration solutions.
• Experience with AWS cloud services.
• Experience with workflow management tools like Dagster or similar.
• Experience with visualization tools like Superset or similar.
• Experience with containerization and orchestration platforms like Kubernetes.
• Ability to optimize and monitor data pipelines for performance.
• Understanding of data security and compliance requirements.
• Knowledge of data warehousing concepts and experience with solutions like AWS Redshift is a plus.
• Strong collaboration and communication skills and able to manage a project on your own.
• Someone who know the difference between “being right and getting it right”.
• Continuous learning mindset and adaptability to emerging technologies.
• Authorised to work in the EU in case of Amsterdam or Malta.

 What can you expect from us:

• We offer a competitive salary for position in Malta or Amsterdam.
• A unique chance to collaborate closely with motivated professionals, while actively contributing to driving the energy transition forward within a startup setting.
• Acquiring significant responsibilities within a dynamic, international environment.
• We have an office in Valletta or Amsterdam that you can work from.
• You'll be part of a close-knit, young team, participating in numerous company events and enjoying ample opportunities to bond while collectively shaping the future of Etpa.

Join our dynamic team and contribute to the advancement of our data engineering capabilities. Please apply via email by sending your CV and cover letter to e-mail: recruitment@etpa.nl.

Etpa is committed to equal employment opportunity regardless of race, colour, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, or gender identity.