Junior Data Engineer

Your connection with Aubay starts in the following lines:

Aubay Portugal is a multinational French company, in Portugal since 2007. We have offices in Lisbon and Oporto and we are a specialized consultant in Management, Implementation, Development and Maintenance of Information Systems. We have more than 150 active partners and we operate in sectors such as banking, insurance, telecommunications, services, energy and transports.

What are you going to do?

• Develop data pipelines with Spark and Scala to ingest and transform data from different sources (Kafka topics, APIs, files) into HDFS, IBM Cloud Storage, Parquet, or SQL/NoSQL databases;
• Manage big data storage solutions in the platform (HDFS, IBM Cloud Storage, Parquet);
• Implementation of schemas and queries in SQL/NoSQL databases like PostgreSQL, MongoDB, or Oracle;
• Develop and configure scheduling of data pipelines with a combination of shell scripting and Autosys, Oozie, or AirFlow;
• Configure Dremio Data Virtualization to interface with Parquet or as a way to expose the data in the different data products;
• Configure Dataiku pipelines as a data preparation tool for loading and transformation of data from our big data platform into specific data marts for data or business products;
• Provide N3-level support to end users.

What are we looking for?

• Spark and Scala as legacy data pipeline development language;
• SparkaaS and Scala as data pipeline development platform;
• Experience in the design and development of streaming processes using Spark Streaming, Spart Structure Streaming, and Apache Kafka;
• Management of legacy big data storage solutions (HDFS and Parquet);
• Management of big data storage solutions (IBM Cloud Object Storage and Parquet):
• Implementation of SQL/NoSQL database schemas and queries (MongoDB, Oracle, PostgreSQL);
• Ksh, Autosys, and Oozie as legacy data pipeline scheduling solution;
• Ksh and Airflow as data pipeline scheduling solution;
• Dremio as data virtualization tool;
• Dataiku as data preparation tool;
• Migration from SAS to Spark as a bonus;

What do we have to offer you?

• Health Insurance;
• Feedback&Coffee – the system that includes Aubay’s Performance Management Cycle, with regular moments of feedback and the Career Progression Model;
• Working model: Hybrid;
• A Training Academy designed to think of you and your needs. You can count on technical and behavioural training sessions (e-Learning and classroom), Meet&Techs, Webinars and Mentoring;
• Events and Teambuildings.

What makes us different?

• Our organizational culture is based on values such as transparency, team spirit, regular feedback and continuous learning;
• Reference prizes and birthday present;
• Lounge;
• We don’t have a dress code;
• We know your name and all address each other informally;
• Coffee and varied snacks at your disposal;
• You will grow in good company. Isn’t this one of the ways in which to develop professionally?

If we got your attention, we now hope to spark your enthusiasm. Ready for the first phase?

We are waiting for you at maria.a.anazario@aubay.pt.

We will use personal information that you have sent us to consider your application for the relevant function. By giving us your information/documents by email, you consent to Aubay Portugal treating your personal information. Your privacy is important to us. Please check our Privacy Policy, which explains how we will use and store your personal information. It also explains the rights you have regarding your information and how you can contact us for clarification of questions or requests.