Data Engineer (Python, Hadoop, Spark)

Omschrijving:

Data Engineer to work on domain-aligned applications in Leads & Insights.

The ICT Data Engineer will join an ambitious team and will contribute to transform, ingest and execute functional/technical mapping of all data needed for sending out leads and insights to our cliënts.
Our client is looking for an experienced and technically skilled data engineer profile with a good functional background, someone who can tackle and improve their dataflows and technical mapping issues and most important to implement data pipelines that will generate their leads and insights.

In the near future they are moving their on prem application to the cloud AWS - also there they are expecting your contribution to make this transition a success.

Bringing their models through Modelfactoring to the next level is also a very big challenge ahead.

They are looking for a cross-over profile doing also some functional work.

• Gathering requirements in cooperation with business
• Join in with the feasibility to determine what data we need, define the work to be done and where
• Write qualitative data selections ~ queries based upon these requirements in Jupyter Notebooks
• Support and guide business with testing
• Guard the quality of the data and verify the integrity, think ahead to define possible issues
• Good knowledge of GDPR and how to deal with it in a data driven context is a plus
• Testing functional via notebooks, remote and local via InitelliJ
• Defining new data sources to be ingested in the data lake, close cooperation and guiding the team who is responsible for it with attention to detail of GDPR and integrity
• Focus on quality and ownership, meaning maintenance of your own builds ~DevOps
  

The most important technologies you will work with are:

• Python, Pyspark, Pybuilder
• Hadoop, Spark, Jupyter Notebook
• Basic Linux
• Scheduling tool UAC
• Deployments will be done with Jenkins - Bladelogic
• indirect context (not deep knowledge needed, but notions is a plus): Cloud native event based architecture

Day to day responsibilities:
Good and performant coding
Development within the above frameworks
Lots of proactive communication together with your team
Agile ~ SAFe context

Needed Qualifications
They are looking for a self-driven, dynamic professional who:

• Is good in analysis of data processes
• Is excellent in analytical and problem-solving capabilities
• Is able to solve, monitor and create technical solutions in the dataflow
• Has some experience of working in a data warehouse environment is a plus
• Is dynamic, proactive and driven
• Is a good communicator, both written and oral – mainly in English
• Is able to deal with a wide range of people in an international environment
• Is able to work independently in a fast-changing environment under high time-pressure
• Is flexible in workplace Brussels / Mechelen / Leuven

Talent must be pampered. That is why we not only offer you a permanent contract of indefinite duration in exchange for your knowledge and commitment. We also act as your career planner and look for the best opportunities, training and growth opportunities for you. In addition to your competitive salary, you will receive a lot of fringe benefits (daily allowance, group and hospitalization insurance, company car, ...).

Are you a freelancer? Of course, you are also eligible and will receive a competitive rate.

Are you looking for an employer to guide your ICT skills in the right direction? Are you impatient to let your dynamism and ambition bear fruit in difficult projects? This is what we offer you at USG ICT, because we make the most of your knowledge and skills. Your mission: to carry out various missions and thus strengthen the results of our customers.

Freelancers are also welcome !!!