119 Data Engineer jobs in London

Data Engineer
12 months
Remote
470 per day inside IR35 - Umbrella only
Active SC clearance and eligible candidates will be considered

Project description
Our client playing a crucial role in the ESN project in 2025, which aims to modernize and enhance communication for frontline emergency services in the UK. This initiative will greatly enhance the capabilities of emergency services, allowing them to share vital data and information quickly and securely, improving response times and overall public safety.
Required Skills

• Data Stage, RedShift, QuickSight, S3
• data migration/ ETL both batch and real time
• data warehouse development
• DevSecOps
• Java
• SQL
• relational databases
• GitHub/lab experience

Nice to have:

• Data quality
• xml
• AWS Data Specialty certification

All profiles will be reviewed against the required skills and experience. Due to the high number of applications we will only be able to respond to successful applicants in the first instance. We thank you for your interest and the time taken to apply!

Data Engineer

I am working with a leading, AI driven organisation that supports its clients' marketing campaigns, expanding its data engineering capabilities and looking for a Data Engineer to join their team. This is a fantastic opportunity to join an award-winning tech business where you will have the opportunity to work at the forefront of data architecture, building distributed systems and leveraging Gen AI tools to accelerate development and improve quality.

You will be encouraged to explore new ways to utilise modern technologies and will be positioned to help mentor other members of the technical team whilst leading innovative technical projects.

As part of this role, you will be responsible for some of the following areas:

• Design and build distributed data pipelines using languages such as Spark, Scala, and Java
• Collaborate with cross-functional teams to deliver user-centric solutions
• Lead on the design and development of relational and non-relational databases
• Apply Gen AI tools to boost development speed and code quality
• Maintain and optimise large-scale data collection processes
• Support the deployment of machine learning models into production

To be successful in the role you will have:

• Creating scalable ETL jobs using Scala and Spark
• Strong understanding of data structures, algorithms, and distributed systems
• Experience working with orchestration tools such as Airflow
• Familiarity with cloud technologies (AWS or GCP)
• Hands-on experience with Gen AI tools for coding and debugging

This is a remote-first role with flexibility to work from anywhere in the UK. Some of the benefits included with the role are listed below:

• Salary of up to 140,000
• Unlimited holiday allowance
• Fully covered private medical care
• Enhanced parental leave policies
• Bespoke training and career development plans
• Regular socials, volunteer days, and a birthday day off

This is just a brief overview of the role. For the full information, simply apply with your CV and I'll be in touch to discuss further. Interviews are already underway, so don't miss out - APPLY now! To do so, please email me at (url removed) or call directly on (phone number removed).

A fantastic opportunity has arisen for a person to join a department to assist with the delivery of innovative generative AI data solutions.

** Please note that due to the nature of the client’s business, only candidates who currently hold full sole British Citizenship (without limitations) will be considered. **

This role performs the duties of a Data Engineer specialising in generative AI and reports to the senior lead within a developing international and transversal structure.

Key Responsibilities:

• Evaluate, design, deploy, improve and support data sets for internal customers, ensuring resilience, security and responsiveness.
li>Collaborate with internal stakeholders to optimise and secure their data in line with operational needs.
• Maintain and enhance data pipelines, ensuring compliance with data governance standards.
• Provide expertise in data management and quality assurance.
• Stay up to date with evolving technology, contribute to the technology roadmap, and deliver cutting-edge solutions.
   

The successful candidate will ideally have:

li>Knowledge in NLP (Natural Language Processing).
• Knowledge in OCR (Object Character Recognition).
• Knowledge in Generative AI.
• Experience in containerisation technologies (e.g. Docker).
• Knowledge in the industrial and/or defence sector.
   

The successful candidate must have:

li>SQL technologies skills (e.g. MS SQL, Oracle).
• noSQL technologies skills (e.g. MongoDB, InfluxDB, Neo4J).
• Data exchange and processing skills (e.g. ETL, ESB, API).
• Development skills (e.g. Python).
• Big data technologies knowledge (e.g. Hadoop stack).
   

This position offers a lucrative benefits package, which includes but is not inclusive of:

li>Salary: Circa £45,000 - £5,000 (depending on experience) < i>Bonus scheme: Up to ,500 (based on company performance) li>Annual pay reviews and promotion reviews (based on personal performance)
• Overtime paid at an enhanced rate
• Flexi-Leave (of up to 15 days)
• Pension scheme (total contribution of up to 14%)
• Flexible working arrangements (2-3 days per week on-site)
• Enhanced parental leave (including maternity, adoption, shared parental, paternity, neonatal, and fertility treatment enhancements)
• Subsidised site facilities and restaurants
• Free parking
• Excellent career progression and training/career development opportunities
   

If this role looks like your next challenge, please contact Keelan ASAP or apply via this advert!
 

We endeavour to reply to every candidate, every time but if you haven’t heard back within 10 days, please understand that you have unfortunately been unsuccessful for this position, or the position has been filled. Please call the office or send an email to discuss other potential positions.

WHJS1_UKTJ

Data Engineer (one day a week in the central London office)

Sports Analytics

This role is a unique opportunity for aData Engineer to combine technical challenges with creativity in a collaborative, high-standard work environment.

By joining this team, youll not only be part of a creative and open work culture focused on innovation and excellence but also have the chance to work with and collaborate wit.

About Us: 

Solirius Reply, part of the Reply Group, delivers technical consultancy and application delivery to our clients in order to solve real world problems and allow our clients to respond to an ever-changing technical landscape. We partner closely with our clients, embedding our consultants into their businesses in order to provide a bespoke service, allowing us to truly understand our clients’ needs.

It is this close collaboration with our clients that has enabled us to grow rapidly in recent years and will drive our ambitious future growth plans. We currently have over 300 consultants working with a variety of key clients from both the public and private sectors such as the Ministry of Justice, Department for Education, FCDOS, UEFA, International Olympic Committee and Mercedes Benz; with plans to increase our client base further in the near future. 

We operate as a flat organisation and believe in trusting and supporting our team to operate independently. We pride ourselves on being specialists at what we do, making the most of our consultants’ expertise in their fields in order to provide a best-in-class service to our clients. All our consultants have the opportunity to work on a range of different projects, providing a broad range of knowledge on which to develop their careers and progress in the direction they choose.

About You:

You are a motivated and adaptable professional with a strong analytical mindset and a passion for using technology to solve real-world problems. You enjoy working in collaborative, agile teams and take pride in delivering high-quality solutions that make a tangible impact. With strong communication skills and a consultative approach, you’re comfortable engaging with clients, understanding their needs, and translating them into effective outcomes. You understand and align with Solirius Reply Values.

The Role:
We are looking for an experienced Senior Data Engineer to join our team here at Solirius. You will be working as part of a team, developing and delivering exciting projects with a fantastic team of technology experts.

You will be responsible for designing, developing, and maintaining data pipelines and systems that enable data analysis and machine learning. You will also collaborate with data scientists, analysts, and other stakeholders to ensure data quality and reliability.

Requirements

Key Responsibilities:

• Develop Data Engineering solutions for our clients/projects
• Design and build data models, schemas to support business requirements
• Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc.
• Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based.
• Working collaboratively with the client and cross-functional teams to identify and address data-related issues and opportunities.
• Stay updated with the latest trends and developments in the data engineering field, such as modern data stack, big data technologies, cloud computing, etc
• Assist with defining the processes needed to achieve operational excellence
• Ensure data quality across all projects/clients
• Define and manage SLA’s for data sets and processes throughout production
• Lead on the design, build and launch of new data models and pipelines

Key Skills/Experience:

• Experience in operating as part of data engineering teams and independently.
• Line/Team management experience, leadership experience
• Experience of working with cloud infrastructure (Azure or AWS, GCP is beneficial)
• SQL and relational databases (e.g. MS SQL/Azure SQL, PostgreSQL)
• You have framework experience within either Flask, Tornado or Django, Docker
• Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo
• Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc.
• Data acquisition and development of data sets and improving data quality
• Preparing data for predictive and prescriptive modelling
• Hands on coding experience, such as Python
• Reporting tools (e.g. Tableau, PowerBI, Qlik)
• GDPR and Government Service Standard (desirable)
• Passionate, motivated and enthusiastic about developing technology solutions.
• Experience working in an Agile development environment
• Data architecture experience.

Benefits

Package and Benefits:

• Competitive Salary
• Bonus Scheme
• Private Healthcare Insurance
• 25 Days Annual Leave + Bank Holidays
• Up to 10 days allocated for development training per year
• Enhanced Parental Leave
• Paid Fertility Leave (5 Days)
• Statutory & Contributory Pension
• EAP with
• Gym Membership Benefits
• Flexible Working
• Annual Away Days/Company Socials

Equality & Diversity:
Solirius Consulting is an equal opportunities employer. We are committed to creating a work environment that supports, celebrates, encourages, and respects all individuals and in which all processes are based on merit, competence and business needs. We do not discriminate on the basis of race, religion, gender, sexuality, age, disability, ethnicity, marital status or any other protected characteristics.
Should you require further assistance or require any reasonable adjustments be put in place to better support your application process, please do not hesitate to raise this with us.

What We’re All About at Kroo

Kroo has a big vision. To be the first bank that is both trusted and loved by its customers. We’re helping people take control of their financial future and achieve their goals, whilst making a positive impact on the planet. Here at Kroo, doing what is right is in our DNA. We act with integrity, transparency and honesty. We think big, dream big, and relentlessly pursue our goals. We like to be bold, break new ground, and we never stop learning. But most importantly, we are on this journey together.

How You’ll Contribute

You’ll report into the Tech Lead and will take ownership of the existing data analytics solution. You will drive the evolution of this critical piece of our decision making capability so that it remains efficient, fit for purpose and scalable.

As we build the best bank for our customers, the ability to make informed and timely decisions is critical to our success. Kroo Bank’s engineering and products teams are clustered around our main banking products - Accounts, Lending, Overdrafts and so on. Data Analytics works across all these products and other business areas like Finance and Operations to deliver business intelligence.

You will work on:

• Design and build a company wide data model.
• Build and maintain an effective data stack, empowering all parts of the business to make timely effective decisions.
• Collaborate with stakeholders and design analytic solutions to be applied across the business.
• Oversee the collection, storage, management, quality, and protection of data.

Requirements

• Extensive experience wrangling raw datasets into published data for analytic use cases: MI, BI or data science.
• Skilled in SQL and proficient in building data pipelines
• Experience in collaborating with stakeholders and establishing consensus over prioritisation and implementation
• Able to define and communicate technical architectures and data models to both business users and engineers
• Excellent understanding of data architecture principles and how these can be practically applied.
• Experience with Python or other scripting languages
• Good working knowledge of the AWS data management stack (RDS, S3, Redshift, Athena, Glue, QuickSight) or Google data management stack (Cloud Storage, Airflow, Big Query, DataPlex, Looker)

About Our Process

We can be flexible with the structure of our interview process if someone's circumstances or timescales require it but our general structure is:

• 20–30 minute conversation with a member of our Talent Team
• 45-minute Hangout Interview with our Data Engineering Lead
• 1-hour System Design interview with members of our Engineering Team and a 30 minute conversation with our CTO

Please let us know if you have any specific requirements or need assistance so we can be as accommodating as possible for you.

Benefits

What we offer:

At our cutting-edge fintech company, we know that attracting and retaining the best talent means offering top-notch benefits that help our employees thrive both in and outside of work. Check out what we currently offer:

• Generous holiday time: 25 days annual leave, 8 bank holidays, 1 Kroo bank holiday (June 24th), and 1 day off during the week of your birthday.
• Personal days: We know that life can be unpredictable, so we offer 3 personal days to use as needed.
• Employer-sponsored volunteer program: We're passionate about giving back to our community, and we support our employees in doing the same with up to 4 hours per month of employer-sponsored volunteer time.
• Mental health support: We care about the mental health of our team members and offer access to Spill, our mental health support partner.
• Workplace pension: We want you to feel secure about your future, so we offer a workplace pension with a 5% employee contribution and a 3% employer top-up.
• Top-notch equipment: We provide top-of-the-line equipment necessary for smooth hybrid work, including a MacBook laptop. Additionally, we also offer support in establishing your home office by contributing towards your setup if required.
• Modern office: When you're in the office, you'll enjoy access to our modern, bustling workspace in Central London (Farringdon).
• Cycle to Work scheme: We encourage sustainable transportation with our Cycle to Work scheme.
• Electric Car scheme: We're committed to reducing our carbon footprint, and our Electric Car scheme makes it easy for our employees to do the same.
• Enhanced parental leave: We know that family comes first, and we offer an enhanced parental leave policy to support our employees in starting and growing their families.
• Room for growth: As a fast-paced, high-growth start-up, we're dedicated to providing our employees with room to grow and excel.
• You get full healthcare for you and your nuclear family via Vitality.

Hybrid Working:

At Kroo Bank, we have a hybrid policy that gives both individuals and teams a lot of freedom when it comes to using the office space to boost productivity. Our London office is a great resource when used effectively. So, employees who can occasionally come to the office are a good fit for how we work right now. Keep in mind that this job involves working from Monday to Friday, with a mix of remote and office work, so you won't need to be on-site all the time.

Diversity and Inclusion:

We wholeheartedly uphold our commitment to fostering a diverse and inclusive workplace. Every employee is highly regarded, respected, and supported without any form of judgement or prejudice. We consider Diversity, Equality, and Inclusion as fundamental pillars guiding our path in all aspects of our bank. We also ensure that reasonable adjustments are made available to all candidates throughout the recruitment process.

To all Recruitment Agencies:

At Kroo Bank, agency resumes are strictly prohibited. Do not submit agency resumes or forward them to our job advertisements or Kroo Bank employees. Be aware that Kroo Bank will not assume any responsibility for fees incurred due to unsolicited resumes.

To ensure a fair and efficient application process, all candidates are kindly requested to submit their applications directly through the advertised platform. We kindly ask that you refrain from reaching out to the company or its employees via email, LinkedIn, or any other communication channels for inquiries or updates. Please note that any attempts to contact us through these channels will not receive a response. Thank you for your understanding and cooperation.

We are seeking a talented Data Engineer to join us here at UBDS Group. As a Data Engineer you will be crucial in supporting the growth of our Data and AI services, This role will work with clients, support and adhere data engineering best practices and take responsibility for the low-level design and development of client data platforms and ETL/ELT pipelines.

Responsibilities  

• Development of data platform solutions, including data storage and ETL pipelines to serve data for analytics or ML. Develop and maintain data models, data integration, and pipeline monitoring and support clients to implement best practices and appropriate architectures against their needs.
• Includes greenfield projects as well as migration and modernisation projects.
• Identifying opportunities for re-usable components and leading their development, maintenance and sharing.
• Collaborate with stakeholders, including client teams, your wider project team such as Cloud Architect, Scrum Master and BA, and Partner technical representatives (Databricks, Microsoft, AWS) to ensure that data solutions meet business requirements and are scalable, secure, and compliant with organizational policies and standards.
• Support technical pre-sales to provide guidance on client requirements for proposals, providing guidance on the technical approach and timeframes.
• Advance our technology vendor partnerships in the area of data engineering, including maintaining and taking relevant certifications and building relationships e.g. with Databricks product team and solution architects.
• Contribute to marketing events and thought leadership articles, as well as attending and possibility to present at conferences and webinars.

Requirements

• Continuously seeks to understand and apply industry best practice and technologies as relevant to the client. Understands your limits and when to ask for help or collaborate with other SMEs.
• Confident leading the design, development and maintenance of data platforms and ETL solutions in a highly-regulated industry.
• Knowledge of all stages of the data lifecycle, and applying DataOps principles to manage pipeline testing, deployment, integration and monitoring.
• Data modelling expertise to develop low-level designs and implement models against business requirements, using design patterns such as Inmon, Kimball and Data Vault.
• Excellent Databricks, Python, PySpark and Spark SQL knowledge, including writing, testing and quality assuring code and knowledge of Unity Catalog best practice to govern data assets as well as Azure and/or AWS experience.
• Experienced in working with sensitive data (e.g. health or financial), understands and applies security, ethics and privacy best practices.
• Consulting experience.

Benefits

Why people choose to grow their careers at UBDS Group

Professionals choose to grow their careers at UBDS Group for its reputation as a dynamic and forward-thinking organisation that is deeply committed to both innovation and employee development. At UBDS Group, employees are given unique opportunities to work on cutting-edge projects across a diverse range of industries, exposing them to new challenges and learning opportunities that are pivotal for professional growth. The Group’s culture emphasises continuous improvement, offering ample training programs, mentorship, and the chance to gain certifications that enhance their skills and marketability.

UBDS Group fosters a collaborative environment where creativity and innovation are encouraged, allowing employees to contribute ideas and solutions that have a tangible impact on the company and its clients. This combination of professional development, a culture of innovation, and the opportunity to make meaningful contributions makes UBDS Group an attractive place for those looking to advance their careers and be at the forefront of technological and operational excellence.

Employee Benefits

• Training – All team members are offered a number of options in terms of personal development, whether it is technical led, business acumen or methodologies. We want you to grow with us and to help us achieve more
• Private medical cover for you and your spouse/partner, offered via Vitality
• Discretionary bonus based on a blend of personal and company performance
• Holiday – You will receive 25 Days holiday, plus 1 day for Birthday and 1 day for your work anniversary in addition to UK bank holidays
• Electric Vehicle leasing with salary sacrifice
• Contributed Pension Scheme
• Death in service cover
• Hybrid Working – UBDS Group offers a flexible working environment to help enable you to operate at your maximum regardless of your location. With offices in London & Manchester we offer a culture that is focussed on outcomes and giving you a work life balance but at that same time creating and driving a culture of inclusivity and togetherness

About UBDS Group

At UBDS Group our mission is to support entrepreneurs who are setting new standards with technology solutions across cloud services, cybersecurity, data and AI, ensuring that every investment advances our commitment to innovation, making a difference, and creating impactful solutions for organisations and society.

With a portfolio including UBDS Digital and Rayo, UBDS Group Companies proudly offer comprehensive, end-to-end digital solutions tailored for both the public and private sectors. By harnessing the strengths of leading technology partners, we deliver innovative strategies, services and solutions that address complex challenges and drive significant value. Our services cover digital consulting, cloud platforms, data and ai, cybersecurity, managed services and delivery management.

Equal Opportunities

We are an equal opportunities employer and do not discriminate on the grounds of gender, sexual orientation, marital or civil partner status, pregnancy or maternity, gender reassignment, race, colour, nationality, ethnic or national origin, religion or belief, disability or age.