454 Spark jobs in the United Kingdom

Get the future you want!

Choosing Capgemini means choosing a company where you will be empowered to shape your career in the way you’d like, where you’ll be supported and inspired by a collaborative community of colleagues around the world, and where you’ll be able to reimagine what’s possible. Join us and help the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world.

Your Role

We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making.

Key Responsibilities:

• Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala.
• Design scalable and robust data processing solutions for batch and real-time data.
• Collaborate with cross-functional teams to gather requirements and translate them into technical specifications.
• Perform data ingestion, transformation, and cleansing from various structured and unstructured sources.
• Monitor and troubleshoot Spark jobs, ensuring high performance and reliability.
• Write clean, maintainable, and well-documented code.
• Participate in code reviews, design discussions, and agile ceremonies.
• Implement data quality and governance best practices.
• Stay updated with emerging technologies and propose improvements

About Capgemini

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, cloud and data, combined with its deep industry expertise and partner ecosystem. Get the future you want |

At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world’s most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and engineering services across all industries. Join us for a career full of opportunities. Where you can make a difference. Where no two days are the same.

Your role 

Capgemini Engineering is looking for a software engineers to join our High Integrity Software team. You will join a diverse team of excellent and experienced engineers who are committed to ensuring our projects meet our customers’ high expectations. 

• We develop award-winning software for safety-related and mission-critical applications across Aerospace, Defence, Rail, Automotive, and Energy sectors, often supporting critical national infrastructure.
• Our projects deliver transformative improvements in operational efficiency and safety, earning high respect and trust from our customers.

Your Profile

Essential

• Background in Software Testing and/or Software development.
• Static code analysis experience or experience with Formal Methods( SPARK, ADA, Z language)
• Experience in working safety critical environment, particularly in the nuclear industry.
• Experience with Spark/Software Language ADA.
• Excellent interpersonal skills and the ability to quickly build rapport with others
• Self-motivated and able to use initiative
• Organised, good time management and prioritisation
• Methodical approach to work, with good attention to details and strong logic and reasoning skills
• Willingness to learn new languages, skills and techniques.

Desirable

• Degree qualified in software engineering or related/relevant subject.
• Experience in at least one of the following markets: aerospace, transport, defence, rail, automotive.
• Knowledge and experience in static analysis techniques

If you're excited about this role but don’t meet every requirement, we still encourage you to apply, your unique experience could be just what we need.

What you’ll love about working here

• Well-being hub and different wellbeing initiatives
• Hybrid working up to 70%
• Possibility to work up to 45 days per year from abroad
• Open access to digital learning platforms
• Active employee networks promoting diversity, equity and inclusion like OutFront, CapAbility or

Need to know

• This is Hybrid role. 2-3 days/week presence is required in office.
• All roles will require a level of security clearance; BPSS OR Security Clearance OR Developed Vetting.
• You can bring your whole self to work. At Capgemini building an inclusive future is part of everyday life and will be part of your working reality. We have built a representative and welcoming environment, for everyone

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry expertise and partner ecosystem.

Spotlight Sports Group is a global media and technology company specialising in content and data within sports betting, horse racing and fantasy sports. With over 400 employees, the group operates multiple award-winning brands, including Racing Post, the world’s largest horse racing affiliate, Pickswise, myracing and Free Super Tips. We partner with leading operators across the betting industry to produce and build multilingual, best-in-class digital products and content to engage and educate customers. ICS-digital, an international marketing agency including ICS-translate, also operates under the group.

This role will be working from our London office 3 days a week and 2 days working from home.

Job Purpose:

To provide a professional service inputting into and maintaining the horse racing database ensuring a high degree of accuracy. The data held in the database is used to provide content to Racing Post’s publications, the website and is also syndicated to other publishers and betting operators.

Key Accountabilities/Responsibilities:

• Collates and inputs horse racing data into the Company databases (Sybase and/or GRP) to a high degree of accuracy.
• Processes horse racing data and maintains the database to the standards required.
• Manages and creates Silks images for international horse racing. 
• Ensure all B2B products are released to clients in a timely fashion. Where issues occur, lead the resolution process and keep the B2B clients informed. 
• Train new or inexperienced staff.
• Writing for various Racing Post papers/publications as and when required.
• Proof read work produced, ensuring the delivery to error-free pages (web and print). 
• Occasionally, take on other production and writing duties as required. 
• Quality assurance.

Key Relationships:

• Content, including other Data Operations teams

• B2B

• External clients. 

Skills and Attributes:

Essential

• Good knowledge of horse racing, greyhounds and sport.
• Ability to produce work to the highest standard.
• Outstanding attention to detail.
• Excellent team working skills.
• Excellent organisation skills with the ability to understand schedules and meet deadlines.
• Ability to work efficiently under pressure.
• Good communication skills.
• IT literate, including good operational knowledge of Microsoft or Google packages.

Desirable

• Previous experience of working in a publishing environment.
• Previous experience with InDesign, Adobe Photoshop, Google Drive

Benefits

We offer a range of well-being initiatives, including private medical insurance, excellent parental leave, a working globally policy, mental health support, assistance programs, and social gatherings. We also provide a pension scheme and various other benefit schemes. Plus, we all get our birthdays off work and enjoy 25 days of holiday per year, as well as the opportunity to buy 5 additional days per year and you can be flexible about when you use your public holidays.

We’ve also got you covered with life assurance and exclusive perks like the Star card and our Step Further Awards (our employee recognition program) to recognise your dedication. For those working via the hybrid model (in the office and at home) we’ve made commuting easier with our Season Ticket Loan and Cycle to Work Scheme.

You can also take advantage of complimentary access to our Racing Post Members Club, complete with an Ultimate Membership. We believe in making a positive impact beyond the workplace, and you'll have the chance to volunteer two days per year with our charity partner, Autism in Racing .

For this role, you will be responsible for providing the framework that appropriately replicates the Big Data needs of a company utilizing data.

Essential requirements:

• More than 3 years of presales experience in the design of Big Data and Data analytics solutions according to customer requirements
• Previous experience with the preparation of high-quality engaging customer presentations, excellent communication skills, experience in conversations at CxO level, ability to adapt the message to the customer feedback, etc.
• Experience in preparation answering RFPs: organize the offer solution team, solution definition, effort and cost estimation,
• Past experience in dealing with partners, tools vendors, etc.
• Business Domain Knowledge
• More than 5 years of experience in Big Data implementation projects
• Experience in the definition of Big Data architecture with different tools and environments: Cloud (AWS, Azure and GCP), Cloudera, No-sql databases (Cassandra, Mongo DB), ELK, Kafka, Snowflake, etc.
• Past experience in Data Engineering and data quality tools (Informatica, Talend, etc.)
• Previous involvement in working in a Multilanguage and multicultural environment
• Proactive, tech passionate and highly motivated

Desirable requirements:

• Experience in Data analysis and visualization solutions: Microstrategy, Qlik, PowerBI, Tableau, Looker,…
• Background in Data Governance and Data Catalog solutions: Axon, Informatica EDC, Colibra, Purview, etc.
• Previous experience in Artificial Intelligence techniques: ML/Deep Learning, Computer Vision, NLP, etc

General information:

• Start Date: ASAP
• Length of Contract: 1 year (minimum)
• Work Location: Madrid
• Remote working. (It may be necessary at some point on-site presence in the customer office in Madrid).

We look forward to receiving your application!

• Design, build, and maintain scalable and efficient data pipelines and ETL/ELT processes using big data technologies (e.g., Spark, Hadoop ecosystem).
• Develop and optimize data models and structures within data warehouses and data lakes.
• Ensure data quality, integrity, and security across all data systems.
• Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver data solutions.
• Implement and manage cloud-based data platforms (e.g., AWS Redshift, S3; Azure Data Lake, SQL Data Warehouse; GCP BigQuery).
• Monitor data pipeline performance, troubleshoot issues, and implement performance enhancements.
• Develop and maintain data dictionaries, metadata management, and documentation.
• Stay abreast of emerging trends and technologies in big data and data engineering.
• Mentor junior data engineers and contribute to best practices within the data team.
• Automate data processes and infrastructure where possible.

• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
• Minimum of 5-7 years of experience in data engineering or a related role, with a strong focus on big data technologies.
• Proficiency in programming languages such as Python, Scala, or Java.
• Hands-on experience with distributed data processing frameworks like Apache Spark.
• Strong SQL skills and experience with various database systems (relational and NoSQL).
• Experience with cloud data platforms (AWS, Azure, GCP) is essential.
• Solid understanding of data warehousing concepts, data modeling, and ETL/ELT principles.
• Excellent problem-solving, analytical, and critical thinking skills.
• Strong communication and collaboration skills, with the ability to work effectively in a remote team.
• Experience with containerization technologies (e.g., Docker, Kubernetes) is a plus.
• Familiarity with data visualization tools is beneficial.

• Design, build, and maintain scalable big data pipelines and infrastructure.
• Develop and optimize ETL/ELT processes for data ingestion and transformation.
• Manage and enhance data lakes and data warehouses.
• Ensure data quality, integrity, and reliability.
• Implement data governance and security best practices.
• Lead and mentor a team of data engineers.
• Collaborate with data scientists and analysts to meet their data needs.
• Evaluate and adopt new data technologies and tools.

• Design, construct, install, and maintain scalable data pipelines and data warehousing solutions.
• Develop and implement ETL/ELT processes for ingesting, transforming, and loading data from various sources.
• Optimize data storage and processing for performance, reliability, and cost-efficiency on cloud platforms.
• Build and maintain data infrastructure using big data technologies such as Spark, Hadoop, and Kafka.
• Collaborate with data scientists, analysts, and other engineers to understand data needs and deliver solutions.
• Implement data governance, quality, and security best practices.
• Monitor data systems, troubleshoot issues, and implement solutions to ensure high availability.
• Develop and maintain robust data models for analytical purposes.
• Stay current with emerging technologies and trends in data engineering and big data.
• Mentor junior data engineers and contribute to the team's technical growth.

• Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field.
• 5+ years of experience in data engineering, with a focus on big data and cloud environments.
• Proficiency in at least one major cloud platform (AWS, Azure, or GCP) and its data services.
• Strong experience with programming languages like Python, Scala, or Java.
• Expertise in SQL and experience with distributed data processing frameworks (e.g., Apache Spark, Hadoop ecosystem).
• Solid understanding of data warehousing concepts, data modeling, and ETL/ELT design patterns.
• Experience with real-time data streaming technologies (e.g., Kafka, Kinesis) is a plus.
• Excellent problem-solving, analytical, and communication skills.
• Ability to work independently and collaboratively in a remote, fast-paced environment.
• This is a fully remote role, ideal for a skilled Data Engineer located near **Manchester, Greater Manchester, UK**, or elsewhere.