3,698 Lead Data Engineer jobs in the United Kingdom

We are searching for an experienced Lead Data Engineer for an exciting data-driven business with multiple office locations across the UK.

The role is offered as a remote position, but you MUST be based in the UK to be considered for the opportunity.

In this role you will be involved in determining the future direction and impact of data engineering within the business.

This is an extremely exciting Lead Data Engineering role, a fantastic opportunity for an experienced, innovative and hands-on data professional to help shape our client's products and solutions.

Reporting directly to the Head of Data Engineering, you will play a crucial role in driving the team's vision and objectives to completion. You will be expected to provide technical leadership, own the solution, ensure the reliability of data products, and collaborate closely with your team and other teams to optimise data solutions.

This is a fantastic opportunity for highly skilled and motivated Lead Data Engineer with strong expertise in data architecture, ETL pipelines, cloud technologies and big data solutions - knowledge of Hadoop and HDFS is 100% required!

You MUST have also run a team to be considered for this role - it could be a small or medium sized team but you do need team management experience - this is a MUST HAVE requirement for our client.

In this role, you will have the following: -

• Technical Leadership
• Team Management
• Leading by Example (100% hands-on)
• Data Strategy & Solutions
• Cloud (Azure or AWS)
• Championing Technical Excellence
• Leverage Big Data Technologies
• Collaboration
• Developing Large, Complex Data Sets
• Building Advanced Data Solutions
• Ensuring Cost Efficiency - Cost Models and Budgets.

About You

You will be a passionate leader with extensive, in-depth and hands-on experience within Data Engineering - you will have the ability to inspire, mentor and motivate a talented team of Data Engineers.

The ideal candidate will be/have: -

• Seasoned hands-on experience in data engineering, with extensive experience in a leadership or mentoring role. Demonstrated track record of leading complex data engineering initiatives at scale. Extensive experience in designing, implementing, and optimising data solutions, supported by a history of successfully managing technical teams and projects.
• Exceptional coding skills.
• Degree in Computer Science, Software Engineering, or similar (applied to Data/Data Specialisation).
• Extensive experience in Data Engineering, in both Cloud & On-Prem, Big Data and Data Lake environments.
• Expert knowledge in data technologies, data transformation tools, data governance techniques.
• Strong analytical and problem-solving abilities.
• Good understanding of Quality and Information Security principles.
• Effective communication, ability to explain technical concepts to a range of audiences.
• Able to provide coaching and training to less experienced members of the team.

Essential Skills:

• Programming Languages such as Spark, Java, Python, PySpark, Scala or similar (minimum of 2).
• Extensive Big Data hands-on experience across coding/configuration/automation/monitoring/security is necessary.
• Significant AWS or Azure hands-on experience.
• ETL Tools such as Azure Data Fabric (ADF) and Databricks or similar ones.
• Data Lakes: Azure Data, Delta Lake, Data Lake or Databricks Lakehouse.
• Certifications: AWS, Azure, or Cloudera certifications are a plus.
• To be considered for this role you MUST have in-depth experience of Hadoop and HDFS.

The role comes with an extensive benefits package including a good pension, good holiday allowance, healthcare, bonus and more!

Please note, to be considered for this opportunity you MUST have the Right to Work in the UK long-term as our client is NOT offering sponsorship for this role.

KEYWORDS

Lead Data Engineer, Senior Data Engineer, Spark, Java, Python, PySpark, Scala, Big Data, AWS, Azure, Cloud, On-Prem, ETL, Azure Data Fabric, ADF, Hadoop , HDFS , Azure Data, Delta Lake, Data Lake

Please note that due to a high level of applications, we can only respond to applicants whose skills and qualifications are suitable for this position.

No terminology in this advert is intended to discriminate against any of the protected characteristics that fall under the Equality Act 2010.

Bowerford Associates Ltd is acting as an Employment Agency in relation to this vacancy.

We are looking for a highly skilled and motivated Lead Data Engineer to join our growing data team. In this role, you will lead a small team of Data Engineers and Analysts, driving the design, development, and maintenance of scalable, secure, and high-performance data solutions on the Microsoft Azure platform. You will provide hands-on technical leadership while shaping the strategic direction of the data infrastructure to support analytics, reporting, and business intelligence across the organisation.

Location: Stockton on Tees
Department: Data & Analytics
Employment Type: Full-Time / Permanent / Hybrid

Key Responsibilities

• Lead and mentor a team of Data Engineers/Analysts, fostering a culture of innovation, collaboration, and continuous improvement.
• Design, build, and optimise data pipelines, data models, and data warehouses using Azure Synapse Analytics, Azure Data Factory, and Azure SQL.
• Ensure the reliability and performance of data pipelines and infrastructure.
• Collaborate with stakeholders and analysts to deliver impactful dashboards and reports using Power BI.
• Provide technical guidance and support, acting as a subject matter expert on Azure data tools and best practices.
• Develop and maintain ETL processes using SQL, SSIS, and Azure Data Factory.
• Integrate data from diverse sources including internal databases, APIs, and third-party providers.
• Monitor and tune performance of data components, ensuring data quality, consistency, and timeliness.
• Automate deployment and data processing tasks where appropriate.
• Document data flows, lineage, and transformation logic to support transparency and maintainability.

• Bachelor's degree in Computer Science, Information Systems, or a related technical field.
• 3+ years of experience in data engineering and ETL development.
• Advanced proficiency in T-SQL, SSIS, and Azure Data Factory.
• Strong experience with data modelling, performance tuning, and troubleshooting.
• Proven ability to lead and manage a small technical team.
• Excellent problem-solving skills and a passion for automation and operational excellence.
• Strong organisational skills and attention to detail.

• Microsoft Azure Data Engineer certification.
• Experience with Azure Synapse Analytics and Power BI.
• Familiarity with data governance, data quality frameworks, and metadata management.

• Personal Drive - Resilient and committed to achieving goals through persistent effort.
• Innovation - Generates new ideas and solutions through original thinking.
• Results Focus - Aligns team efforts with business objectives to deliver measurable outcomes.
• Teamwork - Builds strong team dynamics and contributes to collective success.
• Decision Making - Makes sound judgments and decisions at the right pace.
• Relationship Building - Develops and maintains strong internal and external networks.
• Performance Management - Sets and reviews performance goals aligned with business strategy.
• Integrity - Demonstrates ethical behaviour and lives the organisation's values.

Lead Data Engineer

Salary/Rate : £100,000 - £110,000 per annum + Bonus

Location : North London

Company : Retelligence

About Retelligence

Retelligence is partnering with a high-growth, forward-thinking organization that specializes in digital innovation and marketing across international markets. The company is on an exciting journey, rapidly scaling its capabilities and leveraging advanced technology to deliver cutting-edge solutions. Join a dynamic team within a business that values innovation, supports professional development, and offers exceptional career progression.

The Role

Retelligence is seeking a Lead Data Engineer to take a hands-on role in designing and delivering robust, real-time data pipelines and infrastructure in a Google Cloud Platform (GCP) environment. The company is particularly interested in candidates with strong expertise in SQL. As the Lead Data Engineer, you’ll play a critical role in shaping their data architecture and driving transformation. You’ll partner closely with engineering, product, and analytics teams to ensure efficient, high-performance data systems that enable the business to thrive in a fast-paced environment.

Key Responsibilities:

• p>Design, develop, and maintain scalable, data pipelines and infrastructure in a GCP environment.

• Integrate multiple data sources to ensure seamless data flow across the organization.

• Build and optimize data models for querying and analytics use cases.

• Develop fault-tolerant, highly available data ingestion and processing pipelines.

• Continuously monitor and improve pipeline performance for low-latency and high-throughput operations.

• Ensure data quality, integrity, and security across all systems.

• Implement effective monitoring, logging, and alerting mechanisms.

• Collaborate with product, engineering, and analytics teams to deliver tailored solutions that meet business needs.

About You

• Strong hands-on experience in data engineering with expertise in Python.

• Proven track record of building and managing data pipelines.

• In-depth experience with Google Cloud Platform (GCP) and its associated tools for data ingestion and processing.

• Familiarity with distributed streaming platforms such as Kafka or similar technologies.

• Advanced knowledge of SQL.

• Experience with data orchestration tools.

• Ability to optimize and refactor data pipelines for improved performance and scalability.

• Strong problem-solving skills and the ability to thrive in a collaborative, fast-paced environment.

WHJS1_UKTJ

We are searching for an experienced Lead Data Engineer for an exciting data-driven business with multiple office locations across the UK.

The role is offered as a remote position, but you MUST be based in the UK to be considered for the opportunity.

In this role you will be involved in determining the future direction and impact of data engineering within the business.

This is an extremely exciting .

WHJS1_UKTJ

About Zego

At Zego, we understand that traditional motor insurance holds good drivers back. It's too complicated, too expensive, and it doesn't reflect how well you actually drive. Since 2016, we have been on a mission to change that by offering the lowest priced insurance for good drivers.

From van drivers and gig workers to everyday car drivers, our customers are the driving force behind everything we do. We've sold tens of millions of policies and raised over $200 million in funding. And we’re only just getting started.

About the Role

We’re looking for a Lead Data Engineer to be a technical authority within our data engineering function, helping to shape and evolve our data platform to meet Zego’s ambitious growth.

This is a hands-on technical role- you’ll design and build scalable, resilient, and secure data systems, while acting as a mentor and technical guide for other engineers. You’ll work closely with peers across Engineering, Data Science, Analytics, and Product to ensure our data infrastructure is efficient, reliable, and future-proof.

While this role doesn’t involve direct line management, you’ll be expected to foster technical excellence within the team by sharing knowledge, reviewing code, and coaching others.

What you will be doing

Technical Leadership & Mentorship

• Act as a senior technical reference point for the Data Engineering team.
• Mentor and coach other engineers, helping them grow their technical skills and deliver high-quality work.
• Promote best practices in data engineering, including testing, CI/CD, observability, and infrastructure as code.
  
  

Platform & Architecture

• Design, build, and maintain scalable and secure data pipelines, warehouses, and streaming systems.
• Ensure data is modelled and structured to meet the needs of analytics, data science, and operational use cases.
• Contribute to the evolution of our data architecture, ensuring it can support both current and future business needs.
  
  

Collaboration & Delivery

• Partner with teams across the business to understand requirements and translate them into robust technical solutions.
• Identify opportunities for optimisation, re-architecture, or tool improvements.
• Contribute to the definition and execution of the technical roadmap for data engineering.
  

About You

You’re a hands-on engineer who thrives on solving complex problems and helping others do their best work. You bring technical depth, a pragmatic mindset, and a collaborative approach to everything you do.

What You Bring:

• Experience: 5+ years as a Data Engineer working on scalable data platforms, ideally in product-led or high-growth environments.
  

• Technical Skills:
• Proven expertise in designing, building, and operating ETL/ELT pipelines and large-scale data architectures.
• Deep hands-on experience with modern data stacks — our tech includes Python, SQL, Snowflake, Apache Iceberg, AWS S3, PostgresDB, Airflow, dbt, and Apache Spark, deployed via AWS, Docker, and Terraform (experience with similar technologies is essential).
  

• Collaboration: Ability to work effectively with cross-functional stakeholders, translating technical concepts into business value.
  

• Mentorship: Demonstrated experience in supporting the growth of other engineers through code reviews, pairing, and knowledge sharing.

• Problem-Solving: Pragmatic approach to balancing technical excellence with delivery needs.
  
  

Nice to Have:

• Experience building Data Mesh or Lakehouse architectures.
• Familiarity with Kubernetes, Docker, and real-time streaming technologies (e.g. Kafka, Kinesis).
• Exposure to ML engineering pipelines or MLOps frameworks.
  
  

What’s it like to work at Zego?

Joining Zego is a career-defining move. People go further here, reaching their full potential to achieve extraordinary things.

We're spread throughout the UK and Europe, and united by our drive to get things done. We're proud of our company and our culture - a friendly and inclusive space where we can lift each other up and celebrate our wins every day.

Together, we're setting the bar higher, delivering exceptional work that makes a difference.

Our people are the most important part of our story, and everyone here plays a role. There's loads of room to learn and grow, and you'll get the freedom to steer your career wherever you want.

You'll work alongside a talented group who embrace each other's differences and aren't afraid of a challenge. We recognise our achievements, learn from our mistakes, and help each other to be the best we can be. Together, we're making insurance matter

How we work

kWe believe that teams work better when they have time to collaborate and space to get things done. We call it Zego Hybrid. While some of our team choose to come into our central London office once a week, we're flexible - some people prefer being in once a month or even quarterly. It's all about finding the right balance between collaborative face time and focused home-working, so we can achieve great results while maintaining a healthy work-life balance

Our approach to AI

We believe in the power of AI to meaningfully improve how we work - helping us move faster, think differently, and focus on what matters most. At Zego, we encourage people to stay curious and intentional about how AI is leveraged in their work and teams to drive practical impact every day. This is your chance to do the most meaningful work of your career - and we'll provide you with the tools, support, and freedom to do it well

Benefits

We reward our people well. Join us and you'll get a market-competitive salary, private medical insurance, company share options, generous holiday allowance, and a whole lot of wellbeing benefits. We also offer an annual flexible hybrid working contribution, which you can use to support with your travel to the office or towards your own personal development. And that's just for starters!

There's more to Zego than just a job - Check out our blog for insights, stories, and more.

We're an equal opportunity employer and we value diversity at our company. We do not discriminate on the basis of race, religion, national origin, gender, sexual orientation, age, marital status, or disability status.

#LI-Hybrid

#LI-AD