Lead Data Engineer

Please note: This is to assist candidates in assessing their suitability for the role. The range does not represent a longer term salary commitment for the role., Our data team is responsible for helping us realise the value of data. That could mean anything from up-skilling and training our people, to increasing our revenue through experimentation and personalisation. The list is truly endless. The Innovation strand consists of Data Scientists, Operational Researchers and Behavioural Scientists. As a mix of number crunchers and behavioural experts, they put their curiosity to work in influencing the business. As we enter the fourth industrial revolution, anyone who joins the team will have the opportunity to work on genuinely ground-breaking innovations so that we can continue to blend science into a creative business. As a Lead Data Engineer, both directly and via your team, you will be responsible for developing, maintaining, and optimising our data pipeline infrastructure using our proprietary central data platform (Symphony), which is based on Databricks. You will collaborate with cross-functional teams to design and implement scalable data solutions, ensuring efficient data ingestion, transformation, storage, and analysis. This is a hands-on leadership role where you will be responsible for a small team of data engineers. You will work in partnership with your agile delivery manager and product manager to size up and breakdown work, and distribute that across your team, and be proactive in bringing that activity to closure. A key aspect of this role is to inspire and motivate your team and encourage them to give their best effort and contribute enthusiastically to the shared vision.,

• Provide technical leadership by overseeing productivity, setting objectives and assessing the performance of a few data engineers.
• Design, develop and maintain scalable data pipelines and products using Databricks, including efficient data ingestion, processing and storage as well as prototyping new approaches
• Develop and optimise ELT processes for smooth data flow into the platform and perform data cleansing, validation and enrichments for accurate analysis
• Ensure adherence to data architecture principles and best practices by collaborating with data scientists to define requirements, design data models and optimise storage and partitioning for efficient retrieval and analysis
• Monitor and optimise data pipelines for high performance and reliability, resolving bottlenecks, data quality issues and interaction challenges
• Define data quality characteristics for your products and collaborate with the QA team to implement automated processes for measuring and reporting data correctness through our observability infrastructure
  
  Experience designing and building Databricks data products
• Strong programming skills in languages such as Python (PySpark preferred), Scala, or SQL
• Experience in owning, designing and implementing data pipelines ingesting enterprise levels of data volume with strong knowledge of data engineering concepts, data integration, and ETL processes
• Ability to write production-grade code including automated testing and able to deploy code via CI/CD platforms (e.g. Github Actions, Jenkins)
• Experience in Proficiency in working with distributed computing frameworks, such as Apache Spark and data modelling, database systems, and SQL optimisation techniques
• Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and associated services (e.g., S3, Redshift, BigQuery)
Other things we're looking for (key criteria) · Knowledge of the UK broadcast industry & broadcast/OTT advertising market and digital marketing and advertising industry · Familiar with: Data design techniques for classification and regression as well as knowledge of A/B testing, experimental design, and general statistical modelling · Architecture disciplines such as Data Mesh Architecture, Data Architecture, BI Architecture and Enterprise Architecture · Tools across other data management capabilities - data engineering, data governance, data quality, metadata management and master data management · Designing and building out data visualisation solutions based on technology such as: Tableau, AWS QuickSight, Looker, ThoughtSpot

Shaping culture is in the DNA of ITV. So, it's not surprising that you'll find us in every home in the UK, our productions are famous all over the world and we're at the forefront of the digital streaming revolution. When you join us, you enter a fun working environment. With opportunities to learn, to grow and make a real difference. Small enough that your impact's felt in the business, but big enough that your impact reaches millions of people. Come develop your skills, change TV and the course of your career. Don't just watch it. Be part of it. Join ITV.