224 Data Engineer jobs in London

Big Data Engineer

London, London Northrop Grumman

Posted 5 days ago

Job Viewed

Tap Again To Close

Job Description

UK CITIZENSHIP REQUIRED FOR THIS POSITION: Yes
RELOCATION ASSISTANCE: Relocation assistance may be available
CLEARANCE TYPE: UK-Highest Level of Government Clearance
TRAVEL: Yes, 10% of the Time
**Salary: £77,400 - £116,000**
**Define Possible at Northrop Grumman UK**
At Northrop Grumman UK, our mission is to solve the most complex challenges by shaping the technology and solutions of tomorrow. We call it Defining Possible.
This mind-set goes beyond our customer solutions; it's the foundation for your career development and the impact we have within the community. So, what's your possible?
**Opportunity:**
This is more than just a job; it's a mission.
As a Big Data Systems Engineer, you will be responsible for designing, implementing, and maintaining scalable big data systems that support our data science, machine learning, and AI workloads. You will work closely with cross-functional teams to ensure seamless data integration, security, and compliance with GDPR and privacy regulations. Your expertise in scripting, troubleshooting, and integration testing will be essential in optimizing our data pipelines and orchestration processes.
Our UK Cyber & Intelligence business combines modern software development approaches with a rich heritage and experience in the Defence and security sectors. Our customers have complex and sensitive data and information requirements requiring a mission partner who quickly understands the context, delivering and sustaining a portfolio of challenging technology projects at scale and pace, supporting them through an ambitious digital transformation programme.
If you are looking for a career with meaning where you can make a difference to the nation's security and support critical missions, then look no further.
_"My purpose; to lead a team of engineers with the brightest minds, to push the boundaries and define possible together."_
**Role responsibilities:**
+ Scripting and Automation: Develop and maintain scripts for automating data processes and workflows.
+ Troubleshooting: Diagnose and resolve technical issues related to big data systems, data pipelines, and integrations.
+ Integration Testing: Conduct thorough testing to ensure seamless integration of new tools and technologies into existing systems.
+ Linux Internals: Utilize in-depth knowledge of Linux internals to optimize performance and reliability of big data infrastructure.
+ Network Protocols: Apply understanding of TCP/IP and OSI models in the design and troubleshooting of networked systems.
+ Data Pipeline Support: Manage and optimize data pipelines and orchestration tools such as NiFi and Airflow.
+ Scalable System Design: Design scalable big data systems with a focus on security, GDPR compliance, and privacy
+ Hybrid Cloud Management: Leverage hybrid cloud experience to manage on-premise and AWS cloud resources effectively
+ Data Science Support: Support data science, machine learning, and AI workloads using tools like Jupyter, Spacy, Transformers, and NLTK.
+ Big Data Platforms: Utilize big data NoSQL engines and platforms such as Hive, Impala, and Elasticsearch for data storage and processing
+ BI and Visualization: Implement and support business intelligence and visualization tools like Tableau, Kibana, and PowerBI to provide actionable insights.
**We are looking for:**
+ Strong troubleshooting skills and the ability to diagnose and resolve complex technical issues.
+ Experience with integration testing and ensuring seamless tool integration.
+ In-depth knowledge of Linux internals and system administration.
+ Understanding of TCP/IP and OSI models.
+ Hands-on experience with data pipeline tools like NiFi and Airflow.
+ Proven ability to design scalable big data systems with a focus on security and GDPR compliance.
+ Hybrid cloud experience, specifically with on-premise and AWS cloud environments.
+ Familiarity with data science, machine learning, and AI tools such as Jupyter, Spacy, Transformers, and NLTK.
+ Experience with big data NoSQL engines/platforms such as Hive, Impala, and Elasticsearch.
+ Proficiency with business intelligence and visualization tools like Tableau, Kibana, and PowerBI.
+ Excellent communication and collaboration skills.
Preferred Qualifications:
+ Certification in AWS or other cloud platforms.
+ Experience with additional data orchestration tools.
+ Familiarity with other big data tools and technologies.
+ Previous experience in a similar role within a dynamic and fast-paced environment.
Experience in Cloudera, Hadoop, Cloudera Data Science Workbench (CDSW), Cloudera Machine Learning (CML) would be highly desirable.
**Work Environment:**
+ Full time on-site presence required
**If you don't meet every single requirement, we still encourage you to apply.**
Sometimes, people hesitate to apply because they can't tick every box. We encourage you to apply if you believe the role will suit you well, even if you don't meet all the criteria. You might be exactly who we are looking for, either for this position or for our opportunities at Northrop Grumman UK. We are on an exciting growth trajectory and growing our teams across the UK.
**Security clearance:**
You must hold the highest level of UK Government security clearance. Our requirement team is on hand to answer any questions and we will guide you through the process: .
**Benefits:**
We can offer you a range of flexible working options to suit you, including optional compressed working schedule with every other Friday off. Our benefits including private health care, cash health plan, holiday buy and sell, career development opportunities and performance bonuses. For a comprehensive list of benefits, speak to our recruitment team.
**Why join us?**
+ **A mission to believe in** **-** Every day we contribute to building a more secure and connected world, expanding our reach from land, sea, and air to space and cyberspace. From engineering data and intelligence solutions, to developing maritime navigation and control systems and innovating command and control systems for the UK and NATO, what we do together matters.
+ **A place to belong and thrive** **-** Every voice matters at our table meaning you can bring your authentic self to work. From our Employee Resource Groups backed by thousands of employees to our partnerships with the Association For Black and Minority Ethnic Engineers, Forces Transition Group, Mind, and Women in Defence - we are passionate about growing and supporting our inclusive community where everyone can belong.
+ **Your career, your way** - Shape your career journey with diverse roles, mentorship, and development opportunities that fuel your curiosity, channel your expertise, and nurture your passion. Looking for flexibility? Balance your professional career with your personal life through our health and wellbeing benefits, discount schemes, and investment in your future development. Speak to our team to find the balance that's right for you.
**Ready to apply?**
**Yes** - Submit your application online. Your application will be reviewed by our team and we will be in touch.
**Possibly, I'd like to find out more** **about this role** - Reach out to our team for more information and support: .
**No, I don't think this role is right for me** - Our extensive UK growth means we have exciting, new opportunities opening all the time. Speak to our team to discuss your career goals.
Northrop Grumman is committed to hiring and retaining a diverse workforce, and encourages individuals from all backgrounds and all abilities to apply and consider becoming a part of our diverse and inclusive workforce.
This advertiser has chosen not to accept applicants from your region.

Lead Data Engineer - Big Data Platforms

SW1A 0AA London, London £80000 Annually WhatJobs

Posted 15 days ago

Job Viewed

Tap Again To Close

Job Description

full-time
Our client is seeking a highly experienced and visionary Lead Data Engineer to spearhead the development and optimization of their big data infrastructure. This senior role is critical for enabling data-driven decision-making across the organization by building and maintaining robust, scalable, and efficient data pipelines. You will be responsible for designing, implementing, and managing our data lakes, data warehouses, and ETL/ELT processes, ensuring data quality, reliability, and accessibility for analytics and machine learning teams.

The ideal candidate will possess a Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field, with a minimum of 7 years of professional experience in data engineering. Extensive hands-on experience with distributed big data technologies such as Apache Spark, Hadoop, Kafka, and cloud-based data services (e.g., AWS EMR, Azure Databricks, Google Cloud Dataflow) is essential. Proficiency in SQL and NoSQL databases, as well as strong programming skills in Python or Scala, are required. You should have a deep understanding of data warehousing concepts, data modeling, and ETL/ELT best practices. Experience in designing and implementing robust data governance and data quality frameworks is highly desirable. Strong leadership qualities, with the ability to mentor junior engineers, guide technical strategy, and collaborate effectively with cross-functional teams (data scientists, analysts, business stakeholders), are paramount.

This role offers a unique opportunity to shape the future of data architecture within a dynamic and growing company. You will be instrumental in architecting solutions that handle vast amounts of data, unlock new insights, and drive business value. The position involves a hybrid working model, allowing for flexibility while fostering team collaboration. You will be based in our state-of-the-art offices in London, England, UK , working with cutting-edge technologies and solving complex data challenges. Join a team committed to excellence and innovation in the heart of the tech industry.

Key Responsibilities:
  • Design, build, and maintain scalable big data pipelines and infrastructure.
  • Develop and optimize ETL/ELT processes for data ingestion and transformation.
  • Manage and enhance data lakes and data warehouses.
  • Ensure data quality, integrity, and reliability.
  • Implement data governance and security best practices.
  • Lead and mentor a team of data engineers.
  • Collaborate with data scientists and analysts to meet their data needs.
  • Evaluate and adopt new data technologies and tools.
This advertiser has chosen not to accept applicants from your region.

Data Engineer

London, London TechYard

Posted today

Job Viewed

Tap Again To Close

Job Description

We are partnering with a rapidly scaling Palantir implementation partner, expanding across all seniority levels, having secured a strong pipeline of projects across various industries. This is an exciting opportunity to specialise in one of the fastest-growing technology markets and collaborate with some of the most experienced leaders in the field. Responsibilities: Develop and maintain data pipelines and ETL processes Analyse client challenges and design tailored solutions Collaborate on implementing AI and machine learning models Serve as a point of contact, providing consultative guidance and support Requirements: Proven programming skills in Python and JavaScript Familiarity with Palantir is a plus Excellent communication and consultative abilities Must hold SC or DV Clearance Please apply if this is of interest.
This advertiser has chosen not to accept applicants from your region.

Data Engineer

London, London i3

Posted today

Job Viewed

Tap Again To Close

Job Description

We're Hiring: Data Engineer – Insurance Data & Power BI Location: London, UK (Hybrid) Industry: Insurance / London Market Type: Full-time Are you a data expert with strong SQL and Power BI skills, and experience in the London Market insurance space? My client is looking for a Data Engineer to join their London-based team to help build and support the growing data infrastructure. You’ll play a key role in the development of the data warehouse, data marts, system integrations , and Power BI dashboards , supporting business-critical insights across Underwriting, Claims, and Operations . This role offers exposure to international projects while remaining rooted in the dynamic London Market. What You'll Be Doing Collaborate with the Lead Data Engineer and Business Analysts to define and deliver data solutions Design and support a star schema-based data warehouse Perform application data modelling and develop internal/external data integrations Build insightful and actionable Power BI dashboards and reports Provide proactive data support services to business users Support the integration of business systems and data Develop robust reconciliation and unit testing processes Document solutions, processes, and technical decisions clearly What We’re Looking For Technical Skills Strong SQL Server and T-SQL expertise Deep understanding of data warehouse (DWH) modelling and star schema design Hands-on experience with Azure Data Factory, Pipelines, and Fabric Proficient with Power BI dashboard/report development Familiar with DevOps , Git , and modern CI/CD practices Solid understanding of insurance data – including both syndicate and company markets Professional Skills Strong communication and interpersonal abilities Excellent documentation and organisation skills Self-motivated with the ability to independently manage tasks and deadlines Detail-oriented with a commitment to high-quality deliverables Proven problem-solving and solution-design capabilities Comfortable analysing data to inform requirements and decision-making Domain Knowledge London Market / insurance experience (required) Understanding of application databases , testing , and reconciliation procedures Experience working with international teams/projects is a plus
This advertiser has chosen not to accept applicants from your region.

Data Engineer

London, London HCLTech

Posted today

Job Viewed

Tap Again To Close

Job Description

HCLTech is a global technology company, home to more than 220,000 people across 60 countries, delivering industry-leading capabilities centered around digital, engineering, cloud and AI, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Experience : Minimum 10 Years Strong Knowledge in Hadoop, Kafka, SQL/NoSQL Specialization in designing and implementing large-scale data pipelines, ETL processes, and distributed systems Should be able to work independenty with minimal help/guidance Good Understanding of Airflow,Data Fusion and Data Flow Strong Background and experience in Data Ingestions,Transformation,Modeling and Performance tuning. One migration Experience from Cornerstone to GCP will be added advantage Suppport the design and development of BigData echosystem Experience in building complex SQL Queries Strong Communication Skills
This advertiser has chosen not to accept applicants from your region.

Data Engineer

London, London Singular Recruitment

Posted today

Job Viewed

Tap Again To Close

Job Description

Are you an experienced Data Engineer looking for your next big challenge? Do you have a passion for football? Why not bring the two together? This is a unique opportunity to work at the cutting edge of AI, data technology, and elite football analytics. As a Data Engineer, you’ll join a collaborative, high-performing team with a culture rooted in creativity, innovation, and excellence. In this role, you’ll design and develop scalable data pipelines and infrastructure that power decision-making across every area of the club — from strategy, tactics, recruitment, and performance, to pre- and post-match analysis. You'll also play a key role in supporting the club’s commercial operations, including e-commerce and fan engagement. If you're passionate about using advanced data engineering to drive real-world impact in sport, this is the role for you. Key Responsibilities Architect, design and build scalable and high-performance data pipelines Ensure all systems adhere to best practices for data quality, integrity, and security Integrate modern data engineering tools and technologies into cloud-native infrastructure Develop tools and solutions to support sports data modeling, analytics, and predictive insights Collaborate with data scientists, analysts, and performance experts to enable end-to-end data workflows Own the architecture and maintenance of a GCP-based data lake/lakehouse environment Your Background 4 years of industry experience in Data Engineering roles Advanced-level Python for data applications and high proficiency in SQL (query tuning, complex joins) Hands-on experience designing and deploying ETL/ELT pipelines Proficiency in data architecture, data modeling, and scalable storage design Solid engineering practices: Git and CI/CD for data systems Highly Desirable Skills GCP Stack: Hands-on expertise with BigQuery, Cloud Storage, Pub/Sub, and orchestrating workflows with Composer or Vertex Pipelines. Domain Knowledge: Understanding of sports-specific data types ( event, tracking, scouting, video ) API Development: Experience building data-centric APIs using FastAPI , especially in serverless environments (e.g., Google App Engine) Streaming Data: Familiarity with real-time data pipelines and data ingestion at scale DevOps/MLOps: Exposure to Terraform , Docker , Kubernetes , and MLOps workflows What They Offer A chance to work on real-world data that impacts elite football performance Access to high-value datasets , sports science teams, and cross-disciplinary experts A flexible hybrid working model (1 day per week in the London office) The opportunity to grow within a digital-first team at a world-renowned football club The satisfaction of applying your engineering skills in an environment where your work directly influences results on the pitch
This advertiser has chosen not to accept applicants from your region.

Data Engineer

London, London Norton Blake

Posted today

Job Viewed

Tap Again To Close

Job Description

Data Engineer (Hybrid – London, UK) Overview An opportunity for a skilled and motivated Data Engineer to join a growing data engineering team within a fast-paced and innovative environment. This position is ideal for someone who enjoys designing robust, scalable data infrastructure using modern open-source technologies. The role offers exposure to diverse datasets, complex data challenges, and opportunities to influence technical direction and best practices. Key Responsibilities Design, develop, and maintain data ingestion pipelines using open-source frameworks and tools. Build and optimise ETL/ELT processes for small to large-scale data processing requirements. Develop data models and schemas to support analytics, business intelligence, and product needs. Monitor, troubleshoot, and optimise data pipeline performance and reliability . Collaborate with analysts, engineers, and product teams to gather and translate data requirements. Implement data validation and quality checks to ensure data integrity. Participate in architecture discussions and contribute to technical roadmap planning . Skills and Attributes Strong analytical mindset with exceptional attention to detail. Excellent problem-solving and debugging skills. Ability to work independently and manage multiple priorities effectively. Clear and confident communication skills , with the ability to explain technical concepts to non-technical stakeholders. Experience working in agile development environments . Passion for continuous learning and staying current with emerging data technologies. Technical Requirements Proficiency in SQL , including complex query design and optimisation. Strong Python programming skills, particularly with libraries such as pandas , NumPy , and Apache Spark . Experience building and maintaining data ingestion pipelines and optimising performance. Hands-on experience with open-source data frameworks such as Apache Spark , Apache Kafka , or Apache Airflow . Knowledge of distributed computing and big data concepts. Experience using version control systems (Git) and CI/CD practices. Familiarity with relational databases (PostgreSQL, MySQL, or similar). Experience with containerisation technologies ( Docker , Kubernetes ). Understanding of data orchestration tools (e.g., Airflow or Dagster). Knowledge of data warehousing principles and dimensional modelling . Understanding of cloud platforms and infrastructure-as-code (IaC) practices. Awareness of real-time streaming and data quality monitoring solutions. Preferred Qualifications Bachelor’s degree in Computer Science , Engineering , Mathematics , or a related field. 2–5 years of experience in data engineering or similar technical roles. Familiarity with data processes and challenges in the energy or commodities sectors (beneficial but not required).
This advertiser has chosen not to accept applicants from your region.
Be The First To Know

About the latest Data engineer Jobs in London !

Data Engineer

London, London Peaple Talent

Posted today

Job Viewed

Tap Again To Close

Job Description

Databricks Data Engineer | London | £50,000 - £0,000 Peaple Talent have partnered with a specialist data consultancy delivering services across data engineering, data strategy, data migration, BI & analytics. As a trusted Microsoft Partner, my client are leaders in the industry, with a diverse portfolio of clients and projects. Due to exciting growth plans we are now looking for a Data Engineer, specialising in Databricks. We are looking for: Demonstratable data engineering/BI skills, with a focus on having delivered in Microsoft Azure Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication skills, effectively participating with Senior Stakeholders Nice to haves: Azure Data Engineering certifications Databricks certifications What's in it for you: Location: London (Hybrid) Remote working: Occasional office visits each month Salary: 0,000- 0,000 DOE Collaborative culture and great team support Vast L&D opportunities both internally and externally
This advertiser has chosen not to accept applicants from your region.

Data Engineer

London, London JSS Search

Posted today

Job Viewed

Tap Again To Close

Job Description

Data Engineer London (Hybrid) Up to £60k DOE About the Company We’re partnered with a rapidly scaling AI-driven SaaS business that helps global enterprises unlock the potential of their data. They are entering a really exciting next phase of expansion - scaling their platform, deepening automation and developing their AI solutions. About the Role This business is looking to hire a very talented Mid Level Data Engineer to take a leading role in building and scaling the data foundations of the platform. You’ll combine strong SQL and Python skills with a DevOps mindset, architecting pipelines, automating workflows, and ensuring AI outputs are production-ready. Requirements 1.5 years’ experience in Data Engineering or Data Platform development Experience working with First-Party Data Strong SQL and Python fluency (data transformations, orchestration, automation) Familiarity with cloud ecosystems - Azure, AWS, or GCP Experience with CI/CD tools (Azure DevOps, GitHub Actions, or similar) Excellent communication skills and a structured, detail-oriented approach BSc in a related STEM degree from a top university Please note that visa sponsorship is not available
This advertiser has chosen not to accept applicants from your region.

Data Engineer

London, London Oscar

Posted today

Job Viewed

Tap Again To Close

Job Description

contract
Data Engineer | 6 Month Contract | Outside IR35 | London We’re looking for an experienced Data Engineer to join a leading technology-driven business on an initial 6 month contract. This is a hands-on, onsite position where you’ll be responsible for designing, building, and maintaining scalable data models, pipelines, and cloud infrastructure that power business-critical insights. Location: London (on-site) Contract: 6 Months (very likely extension) Rate: £450-550 p/day Start date: ASAP Required Skills: Strong expertise with MSSQL Server and OLAP/Tabular models (SSAS Tabular, Azure Analysis Services, or Power BI Semantic Models). Advanced SQL and dimensional data modelling skills (fact/dimension design, hierarchies, SCDs). Proven experience building ETL/ELT pipelines using tools such as SSIS , dbt , or Airflow . Solid understanding of database administration , tuning, and performance optimisation across MSSQL and PostgreSQL . Key Responsibilities: Design and maintain data models that meet business requirements, ensuring scalability, consistency, and accuracy. Build and manage ETL/ELT pipelines to integrate data from S3 and various sources into the analytical layer. Administer and optimise MSSQL Server and PostgreSQL databases for performance and reliability. Collaborate with internal teams to improve data availability, reliability, and performance across systems. Design and maintain secure, scalable AWS cloud infrastructure supporting analytical workloads. Create and maintain dimensional and semantic data models for Power BI and Excel integration. Evaluate new data modelling tools and techniques to continuously improve efficiency and scalability. If you're interested, apply now for immediate consideration.
This advertiser has chosen not to accept applicants from your region.
 

Nearby Locations

Other Jobs Near Me

Industry

  1. request_quote Accounting
  2. work Administrative
  3. eco Agriculture Forestry
  4. smart_toy AI & Emerging Technologies
  5. school Apprenticeships & Trainee
  6. apartment Architecture
  7. palette Arts & Entertainment
  8. directions_car Automotive
  9. flight_takeoff Aviation
  10. account_balance Banking & Finance
  11. local_florist Beauty & Wellness
  12. restaurant Catering
  13. volunteer_activism Charity & Voluntary
  14. science Chemical Engineering
  15. child_friendly Childcare
  16. foundation Civil Engineering
  17. clean_hands Cleaning & Sanitation
  18. diversity_3 Community & Social Care
  19. construction Construction
  20. brush Creative & Digital
  21. currency_bitcoin Crypto & Blockchain
  22. support_agent Customer Service & Helpdesk
  23. medical_services Dental
  24. medical_services Driving & Transport
  25. medical_services E Commerce & Social Media
  26. school Education & Teaching
  27. electrical_services Electrical Engineering
  28. bolt Energy
  29. local_mall Fmcg
  30. gavel Government & Non Profit
  31. emoji_events Graduate
  32. health_and_safety Healthcare
  33. beach_access Hospitality & Tourism
  34. groups Human Resources
  35. precision_manufacturing Industrial Engineering
  36. security Information Security
  37. handyman Installation & Maintenance
  38. policy Insurance
  39. code IT & Software
  40. gavel Legal
  41. sports_soccer Leisure & Sports
  42. inventory_2 Logistics & Warehousing
  43. supervisor_account Management
  44. supervisor_account Management Consultancy
  45. supervisor_account Manufacturing & Production
  46. campaign Marketing
  47. build Mechanical Engineering
  48. perm_media Media & PR
  49. local_hospital Medical
  50. local_hospital Military & Public Safety
  51. local_hospital Mining
  52. medical_services Nursing
  53. local_gas_station Oil & Gas
  54. biotech Pharmaceutical
  55. checklist_rtl Project Management
  56. shopping_bag Purchasing
  57. home_work Real Estate
  58. person_search Recruitment Consultancy
  59. store Retail
  60. point_of_sale Sales
  61. science Scientific Research & Development
  62. wifi Telecoms
  63. psychology Therapy
  64. pets Veterinary
View All Data Engineer Jobs View All Jobs in London