434 Spark jobs in the United Kingdom
Big Data Scala Spark Developer
Posted today
Job Viewed
Job Description
HCLTech is a global technology company, home to 219,000+ people across 54 countries, delivering industry-leading capabilities centered on digital, engineering and cloud, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of $13+ billion.
Spark - Must have
Scala - Must Have
Hive & SQL - Must Have
Recent hands-on experience with Scala coding is required.
Banking /Capital Markets Domain - Good to have
Interview includes coding test.
Job Description:
===
Scala/Spark
• Good Big Data resource with the below Skillset:
§ Spark
§ Scala
§ Hive/HDFS/HQL
• Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.)
• Experience in Big data technologies, real time data processing platform (Spark Streaming) experience would be an advantage.
• Consistently demonstrates clear and concise written and verbal communication
• A history of delivering against agreed objectives
• Ability to multi-task and work under pressure
• Demonstrated problem solving and decision-making skills
• Excellent analytical and process-based skills, i.e. process flow diagrams, business modelling,
Big Data Scala Spark Developer
Posted today
Job Viewed
Job Description
HCLTech is a global technology company, home to 219,000+ people across 54 countries, delivering industry-leading capabilities centered on digital, engineering and cloud, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of $13+ billion.
Spark - Must have
Scala - Must Have
Hive & SQL - Must Have
Recent hands-on experience with Scala coding is required.
Banking /Capital Markets Domain - Good to have
Interview includes coding test.
Job Description:
===
Scala/Spark
• Good Big Data resource with the below Skillset:
§ Spark
§ Scala
§ Hive/HDFS/HQL
• Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.)
• Experience in Big data technologies, real time data processing platform (Spark Streaming) experience would be an advantage.
• Consistently demonstrates clear and concise written and verbal communication
• A history of delivering against agreed objectives
• Ability to multi-task and work under pressure
• Demonstrated problem solving and decision-making skills
• Excellent analytical and process-based skills, i.e. process flow diagrams, business modelling,
Software Engineer - SPARK - Bath
Posted 3 days ago
Job Viewed
Job Description
At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world’s most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and engineering services across all industries. Join us for a career full of opportunities. Where you can make a difference. Where no two days are the same.
Your role
Capgemini Engineering is looking for a software engineers to join our High Integrity Software team. You will join a diverse team of excellent and experienced engineers who are committed to ensuring our projects meet our customers’ high expectations.
- We develop award-winning software for safety-related and mission-critical applications across Aerospace, Defence, Rail, Automotive, and Energy sectors, often supporting critical national infrastructure.
- Our projects deliver transformative improvements in operational efficiency and safety, earning high respect and trust from our customers.
Your Profile
Essential
- Background in Software Testing and/or Software development.
- Static code analysis experience or experience with Formal Methods( SPARK, ADA, Z language)
- Experience in working safety critical environment, particularly in the nuclear industry.
- Experience with Spark/Software Language ADA.
- Excellent interpersonal skills and the ability to quickly build rapport with others
- Self-motivated and able to use initiative
- Organised, good time management and prioritisation
- Methodical approach to work, with good attention to details and strong logic and reasoning skills
- Willingness to learn new languages, skills and techniques.
Desirable
- Degree qualified in software engineering or related/relevant subject.
- Experience in at least one of the following markets: aerospace, transport, defence, rail, automotive.
- Knowledge and experience in static analysis techniques
If you're excited about this role but don’t meet every requirement, we still encourage you to apply, your unique experience could be just what we need.
What you’ll love about working here
- Well-being hub and different wellbeing initiatives
- Hybrid working up to 70%
- Possibility to work up to 45 days per year from abroad
- Open access to digital learning platforms
- Active employee networks promoting diversity, equity and inclusion like OutFront, CapAbility or
Need to know
- This is Hybrid role. 2-3 days/week presence is required in office.
- All roles will require a level of security clearance; BPSS OR Security Clearance OR Developed Vetting.
- You can bring your whole self to work. At Capgemini building an inclusive future is part of everyday life and will be part of your working reality. We have built a representative and welcoming environment, for everyone
Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry expertise and partner ecosystem.
Electrician / Spark / Improver / mate
Posted 1 day ago
Job Viewed
Job Description
Electrician Spark/Mate/Improver required to fit and fix new installs for shop fittings & restaurants in Windsor - very advantageousyou will live in Windsor.
Ideally, you will come as a pair, if not I will need to match you up as a pair.
Only looking for reliable people, no days off and you need to work the full 3 months. My client has loads of work coming up so if you are good, then you will have lots of work for the foreseeable future.
Working hours are 8am-4pm - there may be some projects that are nights but mainly days so you must be flexible.
- You MUST be reliable, turn up every shift and graft hard.
- You must have your own transportand tools. You will be working with a qualified Spark so you must be able to work well together and get on.
- You must also have your ECS Card, own UTR number under sole trader CIS Scheme.
- Gold Card very desired but not essential.
Please send in your CV if you are interested and can start from next Wednesday 13th August 2025.
Electrician Spark/Mate/Improver
Posted 3 days ago
Job Viewed
Job Description
Electrician Spark/Mate/Improver required to fit and fix new installs for shop fittings & restaurants in Windsor - very advantageousyou will live in Windsor.
Ideally, you will come as a pair, if not I will need to match you up as a pair.
Only looking for reliable people, no days off and you need to work the full 3 months. My client has loads of work coming up so if you are good, then you will have lo.
WHJS1_UKTJ
Big Data Engineer
Posted 26 days ago
Job Viewed
Job Description
RELOCATION ASSISTANCE: Relocation assistance may be available
CLEARANCE TYPE: UK-Highest Level of Government Clearance
TRAVEL: Yes, 10% of the Time
**Salary: £77,400 - £116,000**
**Define Possible at Northrop Grumman UK**
At Northrop Grumman UK, our mission is to solve the most complex challenges by shaping the technology and solutions of tomorrow. We call it Defining Possible.
This mind-set goes beyond our customer solutions; it's the foundation for your career development and the impact we have within the community. So, what's your possible?
**Opportunity:**
This is more than just a job; it's a mission.
As a Big Data Systems Engineer, you will be responsible for designing, implementing, and maintaining scalable big data systems that support our data science, machine learning, and AI workloads. You will work closely with cross-functional teams to ensure seamless data integration, security, and compliance with GDPR and privacy regulations. Your expertise in scripting, troubleshooting, and integration testing will be essential in optimizing our data pipelines and orchestration processes.
Our UK Cyber & Intelligence business combines modern software development approaches with a rich heritage and experience in the Defence and security sectors. Our customers have complex and sensitive data and information requirements requiring a mission partner who quickly understands the context, delivering and sustaining a portfolio of challenging technology projects at scale and pace, supporting them through an ambitious digital transformation programme.
If you are looking for a career with meaning where you can make a difference to the nation's security and support critical missions, then look no further.
_"My purpose; to lead a team of engineers with the brightest minds, to push the boundaries and define possible together."_
**Role responsibilities:**
+ Scripting and Automation: Develop and maintain scripts for automating data processes and workflows.
+ Troubleshooting: Diagnose and resolve technical issues related to big data systems, data pipelines, and integrations.
+ Integration Testing: Conduct thorough testing to ensure seamless integration of new tools and technologies into existing systems.
+ Linux Internals: Utilize in-depth knowledge of Linux internals to optimize performance and reliability of big data infrastructure.
+ Network Protocols: Apply understanding of TCP/IP and OSI models in the design and troubleshooting of networked systems.
+ Data Pipeline Support: Manage and optimize data pipelines and orchestration tools such as NiFi and Airflow.
+ Scalable System Design: Design scalable big data systems with a focus on security, GDPR compliance, and privacy
+ Hybrid Cloud Management: Leverage hybrid cloud experience to manage on-premise and AWS cloud resources effectively
+ Data Science Support: Support data science, machine learning, and AI workloads using tools like Jupyter, Spacy, Transformers, and NLTK.
+ Big Data Platforms: Utilize big data NoSQL engines and platforms such as Hive, Impala, and Elasticsearch for data storage and processing
+ BI and Visualization: Implement and support business intelligence and visualization tools like Tableau, Kibana, and PowerBI to provide actionable insights.
**We are looking for:**
+ Strong troubleshooting skills and the ability to diagnose and resolve complex technical issues.
+ Experience with integration testing and ensuring seamless tool integration.
+ In-depth knowledge of Linux internals and system administration.
+ Understanding of TCP/IP and OSI models.
+ Hands-on experience with data pipeline tools like NiFi and Airflow.
+ Proven ability to design scalable big data systems with a focus on security and GDPR compliance.
+ Hybrid cloud experience, specifically with on-premise and AWS cloud environments.
+ Familiarity with data science, machine learning, and AI tools such as Jupyter, Spacy, Transformers, and NLTK.
+ Experience with big data NoSQL engines/platforms such as Hive, Impala, and Elasticsearch.
+ Proficiency with business intelligence and visualization tools like Tableau, Kibana, and PowerBI.
+ Excellent communication and collaboration skills.
Preferred Qualifications:
+ Certification in AWS or other cloud platforms.
+ Experience with additional data orchestration tools.
+ Familiarity with other big data tools and technologies.
+ Previous experience in a similar role within a dynamic and fast-paced environment.
Experience in Cloudera, Hadoop, Cloudera Data Science Workbench (CDSW), Cloudera Machine Learning (CML) would be highly desirable.
**Work Environment:**
+ Full time on-site presence required
**If you don't meet every single requirement, we still encourage you to apply.**
Sometimes, people hesitate to apply because they can't tick every box. We encourage you to apply if you believe the role will suit you well, even if you don't meet all the criteria. You might be exactly who we are looking for, either for this position or for our opportunities at Northrop Grumman UK. We are on an exciting growth trajectory and growing our teams across the UK.
**Security clearance:**
You must hold the highest level of UK Government security clearance. Our requirement team is on hand to answer any questions and we will guide you through the process: .
**Benefits:**
We can offer you a range of flexible working options to suit you, including optional compressed working schedule with every other Friday off. Our benefits including private health care, cash health plan, holiday buy and sell, career development opportunities and performance bonuses. For a comprehensive list of benefits, speak to our recruitment team.
**Why join us?**
+ **A mission to believe in** **-** Every day we contribute to building a more secure and connected world, expanding our reach from land, sea, and air to space and cyberspace. From engineering data and intelligence solutions, to developing maritime navigation and control systems and innovating command and control systems for the UK and NATO, what we do together matters.
+ **A place to belong and thrive** **-** Every voice matters at our table meaning you can bring your authentic self to work. From our Employee Resource Groups backed by thousands of employees to our partnerships with the Association For Black and Minority Ethnic Engineers, Forces Transition Group, Mind, and Women in Defence - we are passionate about growing and supporting our inclusive community where everyone can belong.
+ **Your career, your way** - Shape your career journey with diverse roles, mentorship, and development opportunities that fuel your curiosity, channel your expertise, and nurture your passion. Looking for flexibility? Balance your professional career with your personal life through our health and wellbeing benefits, discount schemes, and investment in your future development. Speak to our team to find the balance that's right for you.
**Ready to apply?**
**Yes** - Submit your application online. Your application will be reviewed by our team and we will be in touch.
**Possibly, I'd like to find out more** **about this role** - Reach out to our team for more information and support: .
**No, I don't think this role is right for me** - Our extensive UK growth means we have exciting, new opportunities opening all the time. Speak to our team to discuss your career goals.
Northrop Grumman is committed to hiring and retaining a diverse workforce, and encourages individuals from all backgrounds and all abilities to apply and consider becoming a part of our diverse and inclusive workforce.
Big Data Java Developer
Posted 7 days ago
Job Viewed
Job Description
In this role, you'll work in one of our IBM Consulting Client Innovation Centres (Delivery Centres), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology.
A career in IBM CIC is rooted by long-term relationships and close collaboration with clients across the globe.
You'll work with visionaries across multiple industries to improve the hybrid cloud and AI journey for the most innovative and valuable companies in the world. Your ability to accelerate impact and make meaningful change for your clients is enabled by our strategic partner ecosystem and our robust technology platforms across the IBM portfolio; including Software and Red Hat.
Curiosity and a constant quest for knowledge serve as the foundation to success in IBM Consulting. In your role, you'll be encouraged to challenge the norm, investigate ideas outside of your role, and come up with creative solutions resulting in ground breaking impact for a wide network of clients. Our culture of evolution and empathy centers on long-term career growth and development opportunities in an environment that embraces your unique skills and experience.
**Your role and responsibilities**
IBM is currently seeking a seasoned Big Data Java Developer to join our Public Sector team for a significant project. The ideal candidate will possess a deep understanding of Big Data technologies, particularly within a Java context, along with a passion for creating innovative solutions tailored to the unique demands of public sector projects.
As a Developer at IBM you'll participate in many aspects of the software development lifecycle, such as design, code implementation, testing, and support. You will create software that enables your clients' hybrid-cloud and AI journeys. You'll have the opportunity to work with the latest technologies, ensuring the applications delivered are high performing, highly available, responsive, and maintainable. Your primary responsibilities include: Analytical Problem-Solving and Solution Enhancement: Analyze, validate and propose improvements to existing failures, with the support of the architect and technical leader. Comprehensive Engagement Across Process Phases: Involvement in every step of the process, from design, development, testing release changes and troubleshoot where necessary, providing a great customer service. Strategic Stakeholder Engagement and Innovative Coding Solutions: Drive key discussions with your stakeholders and analyze the current landscape for opportunities to operate and code creative solutions.
**Required technical and professional expertise**
* Design, develop, and maintain Java-based applications for processing and analyzing large datasets, utilizing frameworks such as Apache Hadoop, Spark, and Kafka.
* Collaborate with cross-functional teams to define, design, and ship data-intensive features and services.
* Optimize existing data processing pipelines for efficiency, scalability, and reliability.
* Develop and execute unit tests, ensuring the quality and robustness of the code.
* Implement best practices for software development, including version control, continuous integration, and automated testing.
* Provide technical guidance and mentorship to junior developers.
* Stay abreast of the latest trends, best practices, and emerging technologies in Big Data and Java development.
* Contribute to the design and implementation of highly scalable, secure, and resilient data processing systems.
**Preferred technical and professional experience**
* Bachelor's or Master's degree in Computer Science, Information Technology, or a related field, or equivalent experience.
* Experience in Big Data Java development.
* In-depth knowledge of Big Data frameworks, such as Hadoop, Spark, and Kafka, with a strong emphasis on Java development.
* Proficiency in data modeling, ETL processes, and data warehousing concepts.
* Experience with data processing languages like Scala, Python, or SQL.
* Familiarity with containerization technologies (Docker) and orchestration tools (Kubernetes).
* Strong knowledge of software development principles, including object-oriented design, design patterns, and clean code practices.
* Excellent problem-solving skills and attention to detail.
* Strong communication and team collaboration skills.
* Ability to obtain necessary security clearances as required by public sector projects.
Preferred Qualifications:
* Experience working in the public sector or understanding of public sector operations.
* Knowledge of IBM Big Data and AI solutions.
* Experience with cloud platforms, preferably IBM Cloud.
* Contributions to open-source projects or personal projects demonstrating Big Data and Java development skills.
* Relevant certifications such as Cloudera Certified Associate (CCA) or Hortonworks Certified Developer (HCD) are considered a plus.
By joining IBM's Public Sector team as a Big Data Java Developer, you'll have the opportunity to make a significant impact on public services while growing your career with one of the world's leading technology companies. We offer a competitive benefits package, ongoing learning opportunities, and a dynamic, inclusive work environment where your skills and passions can reach their full potential.
As an equal opportunities' employer, we welcome applications from individuals of all backgrounds. However, for you to be eligible for this role, you must have the valid right to work in the UK. Unfortunately, we do not offer visa sponsorship and have no future plans to do so. You must be a resident in the UK and have been living continuously in the UK for the last 10 years. You must be able to hold or gain a UK government security clearance.
IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.
Be The First To Know
About the latest Spark Jobs in United Kingdom !
Data Solutions Architect (Modern Big Data)
Posted 5 days ago
Job Viewed
Job Description
We are seeking a highly experienced and visionary Data Solutions Architect (Modern Big Data) to join our Data & AI practice. The successful candidate will bring extensive expertise in architecting and delivering modern big data platforms that are scalable, reliable, and business-aligned. This role is pivotal in enabling clients to harness the power of streaming data, data lakes, lakehouses, and advanced analytics platforms, while guiding them on their data modernisation journeys.
As a trusted advisor, you will collaborate with executives, stakeholders, and technical teams to define modern big data strategies, design cloud-native architectures, and implement industry-leading best practices. You will thrive in a fast-paced, evolving technology environment, continuously expanding your knowledge to ensure NTT DATA and our clients remain leaders in data-driven innovation.
**What you'll be doing:**
**Primary Responsibilities:**
+ Client Engagement & Delivery
+ Solution Design & Implementation
+ Modernisation & Transformation
+ Thought Leadership & Knowledge Sharing
+ Collaboration & Leadership
**Business Relationships:**
+ Client Partners
+ Practice Leaders and Members
+ Peer-level relationships within client organisations up to Head of Data Engineering, Chief Data Architect, CIO, and CDO level
**What experience you'll bring:**
**Key Competencies:**
+ Demonstrated expertise in designing and implementing modern big data architectures for large enterprises.
+ Strong consulting values with the ability to engage effectively with senior client stakeholders.
+ Hands-on experience across the data lifecycle: ingestion, storage, transformation, governance, analytics, and consumption.
+ Deep understanding of data product strategies, Data Mesh, and Data Fabric concepts.
+ Ability to align data strategies with business objectives, driving measurable value.
+ Excellent analytical, problem-solving, and communication skills.
+ Proven experience leading data modernisation initiatives across multiple sectors.
**Technical Expertise:**
+ Proficiency with big data technologies: Apache Spark, Kafka, Confluent, Databricks, Unity Catalog.
+ Strong knowledge of cloud ecosystems (AWS, Azure, GCP) and infrastructure-as-code (Terraform, CloudFormation).
+ Hands-on expertise with data lake/lakehouse architectures and modern streaming solutions.
+ Experience with SQL, NoSQL, ETL/ELT pipelines, and programming languages (Python, R, Java).
+ Familiarity with BI, data visualisation, and DevOps principles.
+ Strong understanding of data governance, security, and compliance frameworks (GDPR, CCPA, HIPAA).
+ Exposure to machine learning and AI integration within modern data platforms.
**Experience, Qualifications:**
+ Experience: Minimum 8-12 years in data architecture, engineering, or consulting, with at least 4+ years in modern big data solution architecture.
+ Education: University degree required.
+ Preferred: BSc/MSc in Computer Science, Data Engineering, or related field.
+ Relevant certifications in Databricks, Kafka, or cloud platforms highly desirable.
**Measures of Success:**
+ Delivery of modern big data solutions aligned to client business objectives.
+ High client satisfaction and repeat engagements.
+ Leadership in client transformation journeys to cloud-native data platforms.
+ Contribution to practice growth through assets, accelerators, and thought leadership.
**Who we are:**
We're a business with a global reach that empowers local teams, and we undertake hugely exciting work that is genuinely changing the world. Our advanced portfolio of consulting, applications, business process, cloud, and infrastructure services will allow you to achieve great things by working with brilliant colleagues, and clients, on exciting projects.
Our inclusive work environment prioritises mutual respect, accountability, and continuous learning for all our people. This approach fosters collaboration, well-being, growth, and agility, leading to a more diverse, innovative, and competitive organisation. We are also proud to share that we have a range of Inclusion Networks such as: the Women's Business Network, Cultural and Ethnicity Network, LGBTQ+ & Allies Network, Neurodiversity Network and the Parent Network.
For more information on Diversity, Equity and Inclusion please click here: Creating Inclusion Together at NTT DATA UK | NTT DATA ( we'll offer you:**
We offer a range of tailored benefits that support your physical, emotional, and financial wellbeing. Our Learning and Development team ensure that there are continuous growth and development opportunities for our people. We also offer the opportunity to have flexible work options.
You can find more information about NTT DATA UK & Ireland here: are an equal opportunities employer. We believe in the fair treatment of all our employees and commit to promoting equity and diversity in our employment practices. We are also a proud Disability Confident Committed Employer - we are committed to creating a diverse and inclusive workforce. We actively collaborate with individuals who have disabilities and long-term health conditions which have an effect on their ability to do normal daily activities, ensuring that barriers are eliminated when it comes to employment opportunities. In line with our commitment, we guarantee an interview to applicants who declare to us, during the application process, that they have a disability and meet the minimum requirements for the role. If you require any reasonable adjustments during the recruitment process, please let us know. Join us in building a truly diverse and empowered team.
Back to search Email to a friend Apply now
Data Engineer
Posted 1 day ago
Job Viewed
Job Description
Data Engineer
Inside IR35
Day Rate: £555.56 per day
Location: Homebased
Start Date: 03/09/2025
End Date: 31/12/2025
Hours per week: 37.5 hours
Max Number of days: 5 days
Role Overview
My client is seeking an experienced Data Engineer to design, build, and optimize data solutions using the Microsoft technology stack. This role focuses on developing scalable ETL pipelines, data models, and cloud-based solutions leveraging Azure Data Factory, Azure Synapse, Azure SQL, and Azure Functions. The ideal candidate will have a strong background in data engineering, cloud architecture, and enterprise data systems.
Key Responsibilities
* Design and implement robust ETL pipelines using Azure Data Factory.
* Develop and maintain data models and data warehouses using Azure Synapse and Azure SQL.
* Build scalable data processing solutions using Azure Functions.
* Create interactive dashboards and reports using Power BI.
* Collaborate with business and technical teams to gather requirements and translate them into data solutions.
* Ensure data quality, security, and compliance across all data engineering processes.
* Optimize performance of data pipelines and storage solutions.
* Support and enhance existing data infrastructure and troubleshoot issues.
Required Skills & Experience
* Minimum of 5 years of experience in data engineering roles.
* Strong hands-on expertise in Azure Data Factory, Azure Synapse, Azure SQL, and Azure Functions.
* Proficiency in SQL, data modeling, and ETL processes.
* Knowledge of cloud-based data architecture and best practices.
* Familiarity with DevOps practices and CI/CD pipelines.
* Excellent problem-solving and communication skills.
Preferred Qualifications
* Microsoft certifications such as Azure Data Engineer Associate or Azure Solutions Architect Expert.
* Experience with data governance, security, and compliance standards (e.g., GDPR, ISO).
* Knowledge of Python or other scripting languages for data processing.