Go to job search

Lead Data Engineer

Curve Technology

Lead Data Engineer

£90000

Curve Technology, City of Westminster

  • Full time
  • Permanent
  • Onsite working

Posted 2 weeks ago, 1 Sep | Get your application in now before you miss out!

Closing date: Closing date not specified

job Ref: d8c5599557a447c191d9fe637270212b

Full Job Description

  • London, UK

  • Technology

  • CI/CD

  • Databricks

  • ETL

  • PySpark

  • Snowflake

  • SQL

  • Azure, You will join our Technology Practice, leading the design, build and deployment of innovative data solutions, in the cloud, from scratch. You'll report directly to the Head of Technology, and manage and collaborate with members of our fantastic team of 25+ Data Scientists & Engineers, Software Engineers and Architects to deliver and manage innovative projects which push the boundaries of what we can do with data. You'll represent Curve Technology; interfacing with technical stakeholders from our clients and members of our Insights & Analytics team. This will require a combination of strong technical understanding, and an aptitude and an excitement to lead, mentor and guide others.


    • Beyond client work, you will play a key part in shaping the future of our fast-growing and vibrant start-up, and to support the development of our great team of Data Engineers. We'll let you challenge yourself, from your core of data engineering and architecture to support our data science and DevOps work, to grow your solution & cloud architecture knowledge, and to understand the business and strategic impact of your great engineering work - to whatever extent suits you.

    You should be passionate about developing industry first data, AI and analytics capabilities and have an innovative and creative mindset.
    WHAT YOU'LL BE DOING
  • Lead technical delivery of strategic programmes, taking ownership for designing and building innovative data solutions, including managing teams of Data Scientists & Engineers

  • Work with a mix of cloud services (majoring in Databricks and Azure, alongside AWS and Snowflake)

  • Support in setting the direction and vision of the Data Engineering part of our business, putting in place frameworks and guidance to support colleagues in reaching this vision, instilling a culture of quality, reliability and innovation in the team

  • Develop and deploy automated ETL/ELT pipelines using Python, PySpark and SQL, to bring together best-in-class technologies to meet our clients' needs

  • Design Data and Solution Architectures, supporting in assuring they're robust, secure, scalable and ultimately meet the needs of our great clients

  • Help to deploy machine learning models and AI capabilities

  • Act as a technical counterpart to a project lead from our Consulting practice, and alongside Product Managers, data scientists and analysts - and interface with client stakeholders

  • Interrogate rich data sources such as social, search, surveys, reviews, clickstream, sales, connected devices and beyond

  • Identify and explore opportunities to acquire new data sources that deliver innovative perspectives to our clients

  • Work with increasing autonomy to shape the data engineering and technology work we do now and in the future

    Bachelor's degree or higher in an applicable field such as Computer Science, Statistics, Maths or similar Science or Engineering discipline

  • A track record of developing leading data solutions (likely 4+ years professional experience) in cloud environments such as Azure, AWS or GCP - including Databricks

  • An expert in designing efficient physical data models/schemas and developing ETL/ELT scripts in large organisations or data platforms

  • Strong experience leading teams and managing individuals

  • Strong Python and other programming skills (Spark/Scala desirable)

  • Experience both using and building APIs

  • Strong SQL background

  • Exposure to big data technologies (Spark, Hadoop, Presto, etc.)

  • Works well collaboratively, and independently, with a proven ability to form and manage strong relationships within the organisation and clients.

  • Ability to support others and clients in understanding and applying technical best practice

  • You proactively identify issues and opportunities for our code, methods, practices and team to be better and resolve them


    • NICE TO HAVES OR EXCITED TO LEARN
  • Experience of Solution, Integration and Network Architecture design

  • Experience utilising consumer datasets such as eCommerce, CRM, Social Media

  • Experience of Object Oriented Programming, CI/CD pipelines and broader DevOps technologies and practices

  • Experience setting technology and data strategies, shaping technical teams and ways of working

    Curve is a next-gen insights, analytics and technology consultancy that leverages digital consumer data and advanced technology to help businesses unlock consumer opportunities. Digital consumer data is powerful; it's big, it's real, and it's always updating. We use a combination of in-house technology and bespoke solutions, powered by AI, to transform data from sources such as Social, Reviews, Search, and broader marketing and sales data. These reveal fresh insights for our clients; helping them to build better products and brands, to deliver effective marketing to consumers.


    • Our software, machine learning and AI are key to how we deliver impact, centred on:
  • Natural Language Processing, GPT & other LLMs: unearthing trends, themes and other patterns from large text-based data sets, and deploying state-of-the-art AI to automate and empower consumer facing businesses and their insights & analytics functions

  • Marketing Data Science & Personalisation: using first party consumer data to understand each client's consumer base, building personalisation and other machine learning models to better engage with and excite consumers

  • Data Engineering & Data Architecture: data engineering across a variety of tools to integrate these leading technologies into optimised and efficient data models and ecosystems, feeding into best-in-class analytics dashboards, marketing activation and front-end platforms

  • Software Engineering: full stack expertise to build, maintain and support internal and externally facing Software & Data as a Service solutions, in AWS, that accelerate delivery and unlock deeper insights for our clients