890 Big Data Hadoop jobs in the United Kingdom
Big Data Engineer
Posted 16 days ago
Job Viewed
Job Description
RELOCATION ASSISTANCE: Relocation assistance may be available
CLEARANCE TYPE: UK-Highest Level of Government Clearance
TRAVEL: Yes, 10% of the Time
**Salary: £77,400 - £116,000**
**Define Possible at Northrop Grumman UK**
At Northrop Grumman UK, our mission is to solve the most complex challenges by shaping the technology and solutions of tomorrow. We call it Defining Possible.
This mind-set goes beyond our customer solutions; it's the foundation for your career development and the impact we have within the community. So, what's your possible?
**Opportunity:**
This is more than just a job; it's a mission.
As a Big Data Systems Engineer, you will be responsible for designing, implementing, and maintaining scalable big data systems that support our data science, machine learning, and AI workloads. You will work closely with cross-functional teams to ensure seamless data integration, security, and compliance with GDPR and privacy regulations. Your expertise in scripting, troubleshooting, and integration testing will be essential in optimizing our data pipelines and orchestration processes.
Our UK Cyber & Intelligence business combines modern software development approaches with a rich heritage and experience in the Defence and security sectors. Our customers have complex and sensitive data and information requirements requiring a mission partner who quickly understands the context, delivering and sustaining a portfolio of challenging technology projects at scale and pace, supporting them through an ambitious digital transformation programme.
If you are looking for a career with meaning where you can make a difference to the nation's security and support critical missions, then look no further.
_"My purpose; to lead a team of engineers with the brightest minds, to push the boundaries and define possible together."_
**Role responsibilities:**
+ Scripting and Automation: Develop and maintain scripts for automating data processes and workflows.
+ Troubleshooting: Diagnose and resolve technical issues related to big data systems, data pipelines, and integrations.
+ Integration Testing: Conduct thorough testing to ensure seamless integration of new tools and technologies into existing systems.
+ Linux Internals: Utilize in-depth knowledge of Linux internals to optimize performance and reliability of big data infrastructure.
+ Network Protocols: Apply understanding of TCP/IP and OSI models in the design and troubleshooting of networked systems.
+ Data Pipeline Support: Manage and optimize data pipelines and orchestration tools such as NiFi and Airflow.
+ Scalable System Design: Design scalable big data systems with a focus on security, GDPR compliance, and privacy
+ Hybrid Cloud Management: Leverage hybrid cloud experience to manage on-premise and AWS cloud resources effectively
+ Data Science Support: Support data science, machine learning, and AI workloads using tools like Jupyter, Spacy, Transformers, and NLTK.
+ Big Data Platforms: Utilize big data NoSQL engines and platforms such as Hive, Impala, and Elasticsearch for data storage and processing
+ BI and Visualization: Implement and support business intelligence and visualization tools like Tableau, Kibana, and PowerBI to provide actionable insights.
**We are looking for:**
+ Strong troubleshooting skills and the ability to diagnose and resolve complex technical issues.
+ Experience with integration testing and ensuring seamless tool integration.
+ In-depth knowledge of Linux internals and system administration.
+ Understanding of TCP/IP and OSI models.
+ Hands-on experience with data pipeline tools like NiFi and Airflow.
+ Proven ability to design scalable big data systems with a focus on security and GDPR compliance.
+ Hybrid cloud experience, specifically with on-premise and AWS cloud environments.
+ Familiarity with data science, machine learning, and AI tools such as Jupyter, Spacy, Transformers, and NLTK.
+ Experience with big data NoSQL engines/platforms such as Hive, Impala, and Elasticsearch.
+ Proficiency with business intelligence and visualization tools like Tableau, Kibana, and PowerBI.
+ Excellent communication and collaboration skills.
Preferred Qualifications:
+ Certification in AWS or other cloud platforms.
+ Experience with additional data orchestration tools.
+ Familiarity with other big data tools and technologies.
+ Previous experience in a similar role within a dynamic and fast-paced environment.
Experience in Cloudera, Hadoop, Cloudera Data Science Workbench (CDSW), Cloudera Machine Learning (CML) would be highly desirable.
**Work Environment:**
+ Full time on-site presence required
**If you don't meet every single requirement, we still encourage you to apply.**
Sometimes, people hesitate to apply because they can't tick every box. We encourage you to apply if you believe the role will suit you well, even if you don't meet all the criteria. You might be exactly who we are looking for, either for this position or for our opportunities at Northrop Grumman UK. We are on an exciting growth trajectory and growing our teams across the UK.
**Security clearance:**
You must hold the highest level of UK Government security clearance. Our requirement team is on hand to answer any questions and we will guide you through the process: .
**Benefits:**
We can offer you a range of flexible working options to suit you, including optional compressed working schedule with every other Friday off. Our benefits including private health care, cash health plan, holiday buy and sell, career development opportunities and performance bonuses. For a comprehensive list of benefits, speak to our recruitment team.
**Why join us?**
+ **A mission to believe in** **-** Every day we contribute to building a more secure and connected world, expanding our reach from land, sea, and air to space and cyberspace. From engineering data and intelligence solutions, to developing maritime navigation and control systems and innovating command and control systems for the UK and NATO, what we do together matters.
+ **A place to belong and thrive** **-** Every voice matters at our table meaning you can bring your authentic self to work. From our Employee Resource Groups backed by thousands of employees to our partnerships with the Association For Black and Minority Ethnic Engineers, Forces Transition Group, Mind, and Women in Defence - we are passionate about growing and supporting our inclusive community where everyone can belong.
+ **Your career, your way** - Shape your career journey with diverse roles, mentorship, and development opportunities that fuel your curiosity, channel your expertise, and nurture your passion. Looking for flexibility? Balance your professional career with your personal life through our health and wellbeing benefits, discount schemes, and investment in your future development. Speak to our team to find the balance that's right for you.
**Ready to apply?**
**Yes** - Submit your application online. Your application will be reviewed by our team and we will be in touch.
**Possibly, I'd like to find out more** **about this role** - Reach out to our team for more information and support: .
**No, I don't think this role is right for me** - Our extensive UK growth means we have exciting, new opportunities opening all the time. Speak to our team to discuss your career goals.
Northrop Grumman is committed to hiring and retaining a diverse workforce, and encourages individuals from all backgrounds and all abilities to apply and consider becoming a part of our diverse and inclusive workforce.
Data Analysis Engineer - SOX Algorithm
Posted 1 day ago
Job Viewed
Job Description
At Tata Technologies we make product development dreams a reality by designing, engineering and validating the products of tomorrow for the worlds leading manufacturers. Due to our continued growth we are now recruiting for a Data Analysis Engineer SOX Algorithm to be based at our customers site in Gaydon .
Our Embedded Systems Solutions department (ESS) encompasses all activities outlined under th.
WHJS1_UKTJ
Data Engineer
Posted 6 days ago
Job Viewed
Job Description
Data Engineer - Portsmouth - 60,000
A rapidly growing software company in Portsmouth is looking for a Data Engineer to lead the build of their first-ever data platform. This is a greenfield opportunity to architect and implement scalable data infrastructure from scratch, with the potential to grow into a team lead role.
With over 100 employees and continued expansion, they're now investing in data to support smarter decision-making across their subscription-based business. This is a hands-on role with strategic impact. You'll have the autonomy to choose the right tools and shape the data architecture. If successful, you'll have the opportunity to grow the data function and step into a leadership role.
The Role:
- Designing and building robust ETL pipelines using tools like dbt or Apache Airflow
- Integrating data from APIs, databases, and SaaS platforms into BigQuery
- Structuring clean, queryable data models to support analytics and reporting
- Collaborating with analysts to deliver insightful dashboards via Looker
- Establishing data governance and quality processes
Requirements:
- GCP (BigQuery), but open to other cloud backgrounds
- ETL: dbt, Apache Airflow, or similar
- BI: Looker (preferred), or other BI tools
- Languages: SQL, Python, Java
- Experienced data engineer, with strong ETL and cloud data warehouse experience
- Proficiency in SQL and data modelling best practices
- Experience with BI tools and dashboard creation
- Self-starter attitude with minimal supervision required
Package:
- Competitive salary up to 70,000 + discretionary bonus (performance reviewed twice a year)
- Discounts, perks, and 20 days holiday
Please Note: This is a permanent role for UK residents only. This role does not offer Sponsorship. You must have the right to work in the UK with no restrictions. Some of our roles may be subject to successful background checks including a DBS and Credit Check.
Contact me: (url removed)
Data Engineer
Posted 10 days ago
Job Viewed
Job Description
Data Engineer - Portsmouth - 60,000
A rapidly growing software company in Portsmouth is looking for a Data Engineer to lead the build of their first-ever data platform. This is a greenfield opportunity to architect and implement scalable data infrastructure from scratch, with the potential to grow into a team lead role.
With over 100 employees and continued expansion, they're now investing in data to support smarter decision-making across their subscription-based business. This is a hands-on role with strategic impact. You'll have the autonomy to choose the right tools and shape the data architecture. If successful, you'll have the opportunity to grow the data function and step into a leadership role.
The Role:
- Designing and building robust ETL pipelines using tools like dbt or Apache Airflow
- Integrating data from APIs, databases, and SaaS platforms into BigQuery
- Structuring clean, queryable data models to support analytics and reporting
- Collaborating with analysts to deliver insightful dashboards via Looker
- Establishing data governance and quality processes
Requirements:
- GCP (BigQuery), but open to other cloud backgrounds
- ETL: dbt, Apache Airflow, or similar
- BI: Looker (preferred), or other BI tools
- Languages: SQL, Python, Java
- Experienced data engineer, with strong ETL and cloud data warehouse experience
- Proficiency in SQL and data modelling best practices
- Experience with BI tools and dashboard creation
- Self-starter attitude with minimal supervision required
Package:
- Competitive salary up to 70,000 + discretionary bonus (performance reviewed twice a year)
- Discounts, perks, and 20 days holiday
Please Note: This is a permanent role for UK residents only. This role does not offer Sponsorship. You must have the right to work in the UK with no restrictions. Some of our roles may be subject to successful background checks including a DBS and Credit Check.
Contact me: (url removed)
Data Engineer
Posted 10 days ago
Job Viewed
Job Description
Are you a Data Engineer, seeking a new challenge? Do you have passion for future vehicle development?
If so, Expleo have an opportunity for you! Our client, a prestigious Automotive Manufacturer, is currently recruiting for a Data Engineer to join their Customer Relationship Centre team. As a Data Engineer you will play a key part in enabling data-driven decision-making by building and maintaining robust data pipelines, integrating customer service systems, and ensuring high-quality data is available for analysis and insights. Based in Warwickshire, you will be supporting, a dedicated and enthusiastic team, on a contract basis.
You will work closely with analysts, developers, and operational teams to support customer experience initiatives.
.
Responsibilities of the Data Engineer will include:
* Design, build, and maintain scalable data pipelines to support customer service operations
* Migrate and modernise legacy data systems to cloud-based solutions
* Integrate data from CRM systems and customer touchpoints into cloud platforms
* Ensure data quality, consistency, and availability for reporting and analytics
* Collaborate with Data Analysts to deliver actionable insights
* Develop and maintain documentation for data architecture, workflows, and processes
* Troubleshoot and resolve data-related issues, ensuring minimal disruption to service operations
* Support automation of reporting and data delivery
* Design and implement API integrations to integrate data
Qualifications and skills required for the Data Engineer position:
* Ideally degree educated in Computer Science, Data Engineering, or a related field
* Experience as a Data Engineer, ideally within a customer service or contact centre environment
* Knowledge of SQL, Python, and data pipeline development
* Skilled in Google Cloud Platform (GCP) and cloud data tools
* Background in CRM systems and customer data structures
* Understanding of data warehousing concepts and cloud architecture
* Experience with ETL tools and frameworks Airflow, Git, CI/CD pipeline
* Data Insights reporting experience
* Competent with real-time data processing and streaming technologies
* Proficiency in Tableau or other data visualisation tools is desirable
** PLEASE NOTE ** To meet with current legislation, right to work checks will be carried out to ensure candidates can work in the UK. Regretfully, we are unable to support applications that require sponsorship.
If you are interested in applying for the role of Data Engineer or require further information, please contact:
Jacquie Linton
(phone number removed)
(url removed)
Data Engineer
Posted 10 days ago
Job Viewed
Job Description
Data Engineer
Reference: (phone number removed)
Umbrella Rate: £30.42/hr (Inside IR35)
Are you ready to elevate your career as a Data Engineer? This is your chance to join a forward-thinking company that’s revolutionising customer experience through cutting-edge technology. This role offers you the opportunity to work on innovative projects, modernising data infrastructure and enabling data-driven decision-making. With hybrid working arrangements and exposure to advanced cloud-based tools, this position is perfect for someone looking to make a meaningful impact while enjoying flexibility and professional growth.
What You Will Do:
• Design, build, and maintain scalable data pipelines and ETL processes to enhance customer service operations.
• Migrate and modernise legacy data systems to cloud-based solutions for improved performance and accessibility.
• Integrate data from CRM systems and customer touchpoints into cloud platforms for unified analysis.
• Ensure data quality, consistency, and availability to support reporting and analytics.
• Collaborate with Data Analysts to deliver actionable insights and support proactive customer engagement strategies.
• Implement data solutions using Google Cloud Platform services such as BigQuery, Cloud Functions, and Dataflow.
What You Will Bring:
• Proven experience as a Data Engineer, ideally within a customer service or contact centre environment.
• Proficiency in SQL, Python, and data pipeline development.
• Hands-on experience with Google Cloud Platform and cloud data tools.
• Familiarity with CRM systems and customer data structures.
• Strong problem-solving skills, attention to detail, and the ability to work collaboratively with cross-functional teams.
This company values innovation, collaboration, and excellence in customer service. By joining this team, you’ll play a pivotal role in modernising data systems and enabling data-driven strategies that enhance customer experiences. Your contributions will directly support initiatives that align with the company’s commitment to delivering exceptional service and cutting-edge solutions.
Location:
This hybrid role is based in Whitley, Coventry, with hybrid working opportunities after an initial training period.
Interested?
If you’re ready to take your career as a Data Engineer to the next level, don’t miss this opportunity to work on exciting projects and shape the future of customer experience. Apply today and let your expertise shine!
Your CV will be forwarded to Jonathan Lee Recruitment, a leading engineering and manufacturing recruitment consultancy established in 1978. The services advertised by Jonathan Lee Recruitment are those of an Employment Agency.
In order for your CV to be processed effectively, please ensure your name, email address, phone number and location (post code OR town OR county, as a minimum) are included.
Data Engineer
Posted 12 days ago
Job Viewed
Job Description
Data Engineer - Portsmouth - 70,000
A rapidly growing software company in Portsmouth is looking for a Data Engineer to lead the build of their first-ever data platform. This is a greenfield opportunity to architect and implement scalable data infrastructure from scratch, with the potential to grow into a team lead role.
With over 100 employees and continued expansion, they're now investing in data to support smarter decision-making across their subscription-based business. This is a hands-on role with strategic impact. You'll have the autonomy to choose the right tools and shape the data architecture. If successful, you'll have the opportunity to grow the data function and step into a leadership role.
The Role:
- Designing and building robust ETL pipelines using tools like dbt or Apache Airflow
- Integrating data from APIs, databases, and SaaS platforms into BigQuery
- Structuring clean, queryable data models to support analytics and reporting
- Collaborating with analysts to deliver insightful dashboards via Looker
- Establishing data governance and quality processes
Requirements:
- GCP (BigQuery), but open to other cloud backgrounds
- ETL: dbt, Apache Airflow, or similar
- BI: Looker (preferred), or other BI tools
- Languages: SQL, Python, Java
- Experienced data engineer, with strong ETL and cloud data warehouse experience
- Proficiency in SQL and data modelling best practices
- Experience with BI tools and dashboard creation
- Self-starter attitude with minimal supervision required
Package:
- Competitive salary up to 70,000 + discretionary bonus (performance reviewed twice a year)
- Discounts, perks, and 20 days holiday
Please Note: This is a permanent role for UK residents only. This role does not offer Sponsorship. You must have the right to work in the UK with no restrictions. Some of our roles may be subject to successful background checks including a DBS and Credit Check.
Contact me: (url removed)
Be The First To Know
About the latest Big data hadoop Jobs in United Kingdom !
Data Engineer
Posted 12 days ago
Job Viewed
Job Description
Join a dynamic team committed to driving data excellence. Our client is seeking a Data Engineer to become an integral part of their Azure - Data team in Fareham. This permanent, hybrid role will require you to work three days a week in the office, starting on November 3, 2025. You will play a key role in enhancing their Microsoft Business Intelligence capabilities.
**Key Responsibilities:**
- Develop and maintain data pipelines and architectures using Azure tools.
- Collaborate with cross-functional teams to support data-driven decision-making.
- Implement best practices in data management and security.
- Optimize data flow and collection for improved analytics performance.
- Design effective data models to facilitate reporting and analysis.
The ideal candidate will have a strong understanding of Microsoft Business Intelligence and demonstrate a commitment to collaboration and innovation. Our client values inclusivity and welcomes candidates from diverse backgrounds.
If you are excited about the prospect of shaping impactful data solutions and working in a supportive environment, we encourage you to apply for this opportunity. Your contributions will be vital in supporting the team's growth and success.
Data Engineer
Posted 13 days ago
Job Viewed
Job Description
Job Description
We are seeking a skilled Data Engineer to join an existing Market Access project.
You will play a key part in building scalable data solutions that empower market readiness.
3 months initial contract
Remote working ( UK based)
Inside IR35
Responsibilities
- Design, develop, and maintain data pipelines using Palantir Foundry, PySpark, and TypeScript.
- Collaborate with cross-functional teams to integrate data sources and ensure data quality and consistency.
- Implement robust integration and unit testing strategies to validate data workflows.
- Engage with business stakeholders to understand requirements and translate them into technical solutions.
- Contribute to product-oriented thinking and help shape the data engineering roadmap.
Essential Skills
- Strong experience with Palantir Foundry.
- Proficiency in TypeScript and PySpark.
- Understanding of data integration and unit testing practices.
- Excellent communication skills and ability to work with non-technical stakeholders.
Additional Skills & Qualifications
- Experience with OpenTelemetry is desired and beneficial but not needed
Bracknell, UK
Trading as TEKsystems. Allegis Group Limited, Maxis 2, Western Road, Bracknell, RG12 1RT, United Kingdom. No. (phone number removed). Allegis Group Limited operates as an Employment Business and Employment Agency as set out in the Conduct of Employment Agencies and Employment Businesses Regulations 2003. TEKsystems is a company within the Allegis Group network of companies (collectively referred to as "Allegis Group"). Aerotek, Aston Carter, EASi, Talentis Solutions, TEKsystems, Stamford Consultants and The Stamford Group are Allegis Group brands. If you apply, your personal data will be processed as described in the Allegis Group Online Privacy Notice available at (url removed)>
To access our Online Privacy Notice, which explains what information we may collect, use, share, and store about you, and describes your rights and choices about this, please go to (url removed)>
We are part of a global network of companies and as a result, the personal data you provide will be shared within Allegis Group and transferred and processed outside the UK, Switzerland and European Economic Area subject to the protections described in the Allegis Group Online Privacy Notice. We store personal data in the UK, EEA, Switzerland and the USA. If you would like to exercise your privacy rights, please visit the "Contacting Us" section of our Online Privacy Notice at (url removed)/en-gb/privacy-notices for details on how to contact us. To protect your privacy and security, we may take steps to verify your identity, such as a password and user ID if there is an account associated with your request, or identifying information such as your address or date of birth, before proceeding with your request. If you are resident in the UK, EEA or Switzerland, we will process any access request you make in accordance with our commitments under the UK Data Protection Act, EU-U.S. Privacy Shield or the Swiss-U.S. Privacy Shield.