Data Engineer

The Data Engineer is responsible for the design, development, and maintenance of data engineer solutions for Irwin Mitchell. They understand requirements, create technical designs, configure applications, write code, and conduct testing. They collaborate with various teams, participate in meetings, and provide technical advice to address concerns. They troubleshoot and resolve technical issues, debug code, investigate failures, support end-users and the maintenance of the solutions. They document the data engineering development process, including technical specifications. Collaboration with other engineers, testers, and project managers is crucial for successful delivery. They ensure quality and reliability by conducting thorough testing, performing code reviews, and implementing best practices in data engineering development. This role is vital to the on-going transition from on-prem to Azure. Initially this role will support the on-prem requests and requirements and will transition to Azure Cloud Data Engineering. What You'll Be Doing

• Consolidating workloads on the on-premises data warehouse (SQL Server, SSIS) to become leaner.
• Optimizing end-to-end SSIS packages to run efficiently.
• Documenting activities adequately and contributing to the knowledge base.
• Monitoring and troubleshooting data pipelines, ensuring data integrity, availability, and security.
• Collaborating with cross-functional teams to understand complex data requirements and translating them into efficient data models and ETL processes.
• Collaborating closely with data scientists and analysts to understand their requirements and providing data engineering support for their projects.
• Staying up-to-date with the latest advancements in data engineering and Azure technologies, evaluating and recommending new tools, frameworks, and architectures to enhance our data capabilities.
  
  Proven experience as a Data Engineer, with a strong focus on SQL Server, SSIS, Visual Studio
• Expertise in SQL for developing ETL and investigating
• Experience in Python or C# programming for data engineering tasks, including data extraction, transformation, and loading
• In-depth understanding of data modelling, ETL development, and data integration techniques.
• Experience with Azure data services, such as Azure Data Factory, Azure Databricks, Azure SQL Database, Azure Data Lake Storage, Azure Functions, and Azure Synapse Analytics
• Strong knowledge of cloud-based data architectures, distributed computing concepts, and big data technologies (e.g., PySpark, Spark)
• Experience with cloud platform engineering
• Experience with Terraform / Bicep for deploying infrastructure as code and application code via Azure DevOps
  
  This is a very exciting time to join Irwin Mitchell as we are creating a greenfield modern cloud data platform in Azure.
The role will be able to make a big impact on the way we work and the platform we create. You would be part of a wonderful and supportive team.


• 25 days holidays as standard plus bank holidays - You can buy' up to 35hrs of extra holiday too.
• Generous and flexible pension schemes.
• Volunteering days Two days of volunteering every year for a cause of your choice (fully paid)
• Westfield Health membership, offering refunds on medical services alongside our Aviva Digital GP services.
We also offer a wide range of well-being initiatives to encourage positive mental health both in and out of the workplace and to make sure you're fully supported. This includes our Flexible by Choice programme which gives our colleagues more choice over a hybrid way of working subject to role, team and client requirements. We have been ranked in the Best Workplaces for Wellbeing for Large Organisations for 2024! Our responsible business programmes are fundamental to who we are and our purpose. We're committed to being a diverse and inclusive workplace where our colleagues can flourish, and we have established a number of inclusion network groups across our business to support this aim. Our commitment to Social Responsibility, community investment activity and tackling climate change is a fundamental part of who we are. It's made up of four strands : Our People, Our Community, Our Environment and Our Pro Bono.