Data Engineer (240000E2)

We are looking for a Data Engineer to join our Data Science team at CWT. The Data Engineer will evolve the design and development of data and machine learning products to support business strategies, as well as ensuring that our data and machine learning services are scalable, reliable and robust. The Data Engineer will work closely with other data and machine learning engineers, as well as product owners and other teams within CWT to deliver solutions and insights for the business. Daily Responsibilities

• Build pipelines for data cleansing, transformation and aggregation to deliver business- and client-facing insights and reports.
• Develop AWS serverless solutions for data services within CWT.
• Work closely with Data Scientists/Machine Learning Engineers to build feature engineering, machine learning training and deployment pipelines.
• Develop and manage new data source ingestion pipelines when they are occasionally required.
• Rotational responsibility to monitoring production pipelines and AWS EMR clusters owned by the team.
  
  Degree in computer science, information technology, engineering or equivalent
• At least 2 years of experience in Scala/Python programming
• Comfortable working with Linux and bash scripting
• Familiarity with distributed systems
• Familiarity with data structures and algorithms
Desired Skillset
• Experience with AWS
• Familiarity with Apache Spark and/or Apache Kafka
• Experience in machine learning or data engineering
• Familiarity with SQL
• Exposure to Terraform