Data Engineer Data Engineer

Join us in building the new Crown Estate Azure data platform! As a Data Engineer, you'll deliver technical solutions and create end-to-end analytical systems that are highly available, scalable, stable, secure, and cost-effective. Your main goal will be to build, deploy and automate data pipelines using the Azure cloud platform, including IAC (Bicep), Data Factory, Python/PySpark and Databricks. You'll support the Senior Data Engineer in making data usable, monitoring its quality, and delivering data services that drive The Crown Estate's strategy.,

• Design, implement, and operate large-scale, high-performance data pipelines for analytics and data science.
• Apply best practices in development, test design/automation, CI/CD, and documentation.
• Develop, monitor, and maintain data pipelines, ensuring consistent availability and quality.
• Analyse data structures, identify performance changes, and suggest improvements.
• Understand and meet stakeholder requirements.
• Ensure information security at all times., We value that you get more time with your family and friends. Therefore, we offer everybody 28 days paid holiday, 2 volunteering days and the option to purchase further holiday.
• Flexible working
We value work-life balance highly, therefore we have flexible & hybrid working policies.
• Market leading family friendly policies
We want the support we offer to be inclusive of everyone, reflective of the wide range of situations people face in life.Workplace, Culture & Diversity It's our ambition to be an inclusive organisation that reflects our society, one where all feel welcome, able to be themselves and belong. A workplace where you're empowered to be yourself, diverse thinking is encouraged, and a place where you can realise your potential. Digital · The Crown Estate, 1 St James Market · Hybrid Remote


• Degree in Computer Science, Engineering, Mathematics, or related field, or equivalent industry experience.
• Proven experience in developing data ingestion, processing, and analytical pipelines for big data, relational databases, NoSQL, and data warehouse solutions.
• Extensive hands-on experience implementing data pipelines & services using Azure: Data Factory, Functions, Synapse, Azure SQL DB, Event Hubs/Grids, Databricks, Unity Data Catalog, Cosmos Db, etc.
• Hands-on experience in Big Data technologies such as PySpark, Python, SQL, Databricks, ADLS/Storage options, Lakehouse design, File formats (JSON/Delta/Parquet), Powershell, and streaming technologies.
• Well versed in DevOps, IaC (Bicep/Terraform) and CI/CD deployments, experience working with Developer tools such as Azure DevOps, VS Code, GitHub, etc.
• Experience in building scalable and performant API data services.
• Awareness with Azure AI technologies and tools: openAI, ML, Cognitive Services, AI Studio
• Excellent stakeholder management skills at all levels.
Nice-to-Have Certifications:
• DP-200 Implementing an Azure Data Solution / DP-201 Designing an Azure Data Solution is desirable but not essential
Key Business Capabilities:
• Continuous Improvement: Lead in adopting industry best practices and staying updated with technical and compliance requirements.
• Communication: Strong interpersonal skills, especially under pressure. Ability to explain technical topics in simple terms.
• Technology and Data: Expertise in developing and implementing high-quality technology solutions and processes in a matrixed organisation.
• Customer Excellence: Commitment to continuous improvement and challenging the status quo to enhance business growth and customer experience.
• Collaboration: Team player with the ability to influence and align stakeholders and partners, build consensus, and support major initiatives.