Didn't find the right job?

Get expert career advice to help you find the ideal role and improve your job search strategy.

99 Big Data jobs in the United Kingdom

Data Architect

EC1 London, London Randstad Technologies Recruitment

Posted 3 days ago

Tap Again To Close

Job Description

full time

Data Architect

Location: Hybrid - London (2-3 days onsite)
Type: Permanent
Industry: Data Consultancy
Salary: Competitive + Benefits

The Opportunity

Our client, a growing data consultancy with deep expertise in the Microsoft Azure ecosystem, is looking for a Data Architect to join their high-performing team. This role is both strategic and hands-on, focused on designing modern, scalable, and secure data platforms for enterprise clients.

You'll play a key role in shaping data architecture across the full Azure stack-including Azure Databricks and Azure Data Factory (ADF) -and will guide engineering teams in delivering robust, future-proof solutions using lakehouse and medallion architecture principles .

Key Responsibilities

Design end-to-end data architectures using Azure services, including Azure Databricks, ADF, Synapse Analytics , and Data Lake Storage
Define scalable data models and implement architectural patterns such as lakehouse and medallion
Lead technical solution design during client engagements, from discovery to delivery
Establish and enforce data governance, modelling, and lifecycle standards
Support engineering and DevOps teams with guidance on best practices, CI/CD, and infrastructure-as-code

Requirements

7+ years in data architecture or senior engineering roles
Strong hands-on experience with Azure Databricks and Azure Data Factory
Proficient in SQL, Python , and Spark
Expertise in data modelling and architectural patterns for analytics (e.g., lakehouse, medallion, dimensional modelling)
Solid understanding of cloud security, private networking, GDPR, and PII compliance
Excellent communication skills with a strong consulting mindset

Desirable

Experience with Microsoft Purview, Power BI, or Microsoft Fabric
Familiarity with CI/CD pipelines (Azure DevOps), Terraform, and Docker/Kubernetes environments

What's on Offer

Hybrid working with a central London office (2-3 days onsite)
Opportunity to work on large-scale, cloud-native data projects across sectors
Supportive, collaborative team with strong technical leadership
Clear path for career development and ongoing learning

Randstad Technologies Ltd is a leading specialist recruitment business for the IT & Engineering industries. Please note that due to a high level of applications, we can only respond to applicants whose skills & qualifications are suitable for this position. No terminology in this advert is intended to discriminate against any of the protected characteristics that fall under the Equality Act 2010. For the purposes of the Conduct Regulations 2003, when advertising permanent vacancies we are acting as an Employment Agency, and when advertising temporary/contract vacancies we are acting as an Employment Business.

This advertiser has chosen not to accept applicants from your region.

Big Data Java Developer

Cheltenham, South West IBM

Posted 9 days ago

Tap Again To Close

Job Description

**Introduction**
At IBM CIC, we deliver deep technical and industry expertise to a wide range of public and private sector clients in the UK.
A career in IBM CIC means you'll have the opportunity to work with visionaries across multiple industries to improve the hybrid cloud and AI journey for the most innovative and valuable companies in the world. Your ability to accelerate impact and make meaningful change for your clients is enabled by our strategic partner ecosystem and our robust technology platforms across the IBM portfolio.
Curiosity and a constant quest for knowledge serve as the foundation to success in IBM CIC. In your role, you'll be encouraged to challenge the norm, investigate ideas outside of your role, and come up with creative solutions which impact a wide network of clients, whom may be at their site or one of our CIC or IBM locations. Our culture of evolution centres on long-term career growth and development opportunities in an environment that embraces your unique skills and experience.
We offer:
* A multitude of training opportunities from classroom to e-learning, mentoring and coaching programs as well as the chance to gain industry recognized certifications
* Regular and frequent promotion and progression opportunities to ensure you can drive and develop your career with us
* Feedback and checkpoints throughout the year
* Diversity & Inclusion as an essential and authentic component of our culture through our policies and process as well as our Employee Champion teams and support networks
* A culture where your ideas for growth and innovation are always welcome
* Internal recognition programs for peer-to-peer appreciation as well as from manager to employees
* Tools and policies to support your work-life balance from flexible working approaches, sabbatical programs, paid paternity leave, maternity leave and an innovative maternity returners scheme
* More traditional benefits, such as 25 days holiday (in addition to public holidays), online shopping discounts, an Employee Assistance Program, a group personal pension plan of an additional 5% of your base salary paid by us monthly to save for your future.
**Your role and responsibilities**
As a Big Data Engineer you'll participate in many aspects of the software development lifecycle, such as design, code implementation, testing, and support. You will create software that enables your clients' hybrid-cloud and AI journeys.
* You'll have the opportunity to work with the latest technologies, ensuring the applications delivered are high performing, highly available, responsive, and maintainable.
Your primary responsibilities include:
* Analyze, validate and propose improvements to existing failures, with the support of the architect and technical leader.
* Involvement in every step of the process, from design, development, testing release changes and troubleshoot where necessary, providing a great customer service.
* Drive key discussions with your stakeholders and analyze the current landscape for opportunities to operate and code creative solutions.
**Required technical and professional expertise**
* Experienced Java developer with experience of designing and building new applications.
* Excellent grasp of object oriented design and the experience to know when to apply common design patterns.
* Knowledge of and ability to apply SOLID principles.
* Experience of test driven development and related techniques such as mocking and dependency injection.
* Developing in a secure environment.
* Maven Git DevOps tooling experience
* Performance optimisation of Java application
* AWS (lambda S3 DynamoDB etc)
* Cloudformation
* javascript
As an equal opportunities' employer, we welcome applications from individuals of all backgrounds. However, for you to be eligible for this role, you must have the valid right to work in the UK. Unfortunately, we do not offer visa sponsorship and have no future plans to do so. You must be a resident in the UK and have been living continuously in the UK for the last 10 years. You must be able to hold or gain a UK government security clearance.
IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.

This advertiser has chosen not to accept applicants from your region.

Big Data Java Developer

Cheltenham, South West IBM

Posted 9 days ago

Tap Again To Close

Job Description

**Introduction**
In this role, you'll work in one of our IBM Consulting Client Innovation Centres (Delivery Centres), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world? Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology.
A career in IBM CIC is rooted by long-term relationships and close collaboration with clients across the globe.
You'll work with visionaries across multiple industries to improve the hybrid cloud and AI journey for the most innovative and valuable companies in the world. Your ability to accelerate impact and make meaningful change for your clients is enabled by our strategic partner ecosystem and our robust technology platforms across the IBM portfolio; including Software and Red Hat.
Curiosity and a constant quest for knowledge serve as the foundation to success in IBM Consulting. In your role, you'll be encouraged to challenge the norm, investigate ideas outside of your role, and come up with creative solutions resulting in ground breaking impact for a wide network of clients. Our culture of evolution and empathy centers on long-term career growth and development opportunities in an environment that embraces your unique skills and experience.
**Your role and responsibilities**
IBM is currently seeking a seasoned Big Data Java Developer to join our Public Sector team for a significant project. The ideal candidate will possess a deep understanding of Big Data technologies, particularly within a Java context, along with a passion for creating innovative solutions tailored to the unique demands of public sector projects.
As a Developer at IBM you'll participate in many aspects of the software development lifecycle, such as design, code implementation, testing, and support. You will create software that enables your clients' hybrid-cloud and AI journeys. You'll have the opportunity to work with the latest technologies, ensuring the applications delivered are high performing, highly available, responsive, and maintainable. Your primary responsibilities include: Analytical Problem-Solving and Solution Enhancement: Analyze, validate and propose improvements to existing failures, with the support of the architect and technical leader. Comprehensive Engagement Across Process Phases: Involvement in every step of the process, from design, development, testing release changes and troubleshoot where necessary, providing a great customer service. Strategic Stakeholder Engagement and Innovative Coding Solutions: Drive key discussions with your stakeholders and analyze the current landscape for opportunities to operate and code creative solutions.
**Required technical and professional expertise**
* Design, develop, and maintain Java-based applications for processing and analyzing large datasets, utilizing frameworks such as Apache Hadoop, Spark, and Kafka.
* Collaborate with cross-functional teams to define, design, and ship data-intensive features and services.
* Optimize existing data processing pipelines for efficiency, scalability, and reliability.
* Develop and execute unit tests, ensuring the quality and robustness of the code.
* Implement best practices for software development, including version control, continuous integration, and automated testing.
* Provide technical guidance and mentorship to junior developers.
* Stay abreast of the latest trends, best practices, and emerging technologies in Big Data and Java development.
* Contribute to the design and implementation of highly scalable, secure, and resilient data processing systems.
**Preferred technical and professional experience**
* Bachelor's or Master's degree in Computer Science, Information Technology, or a related field, or equivalent experience.
* Experience in Big Data Java development.
* In-depth knowledge of Big Data frameworks, such as Hadoop, Spark, and Kafka, with a strong emphasis on Java development.
* Proficiency in data modeling, ETL processes, and data warehousing concepts.
* Experience with data processing languages like Scala, Python, or SQL.
* Familiarity with containerization technologies (Docker) and orchestration tools (Kubernetes).
* Strong knowledge of software development principles, including object-oriented design, design patterns, and clean code practices.
* Excellent problem-solving skills and attention to detail.
* Strong communication and team collaboration skills.
* Ability to obtain necessary security clearances as required by public sector projects.
Preferred Qualifications:
* Experience working in the public sector or understanding of public sector operations.
* Knowledge of IBM Big Data and AI solutions.
* Experience with cloud platforms, preferably IBM Cloud.
* Contributions to open-source projects or personal projects demonstrating Big Data and Java development skills.
* Relevant certifications such as Cloudera Certified Associate (CCA) or Hortonworks Certified Developer (HCD) are considered a plus.
By joining IBM's Public Sector team as a Big Data Java Developer, you'll have the opportunity to make a significant impact on public services while growing your career with one of the world's leading technology companies. We offer a competitive benefits package, ongoing learning opportunities, and a dynamic, inclusive work environment where your skills and passions can reach their full potential.
As an equal opportunities' employer, we welcome applications from individuals of all backgrounds. However, for you to be eligible for this role, you must have the valid right to work in the UK. Unfortunately, we do not offer visa sponsorship and have no future plans to do so. You must be a resident in the UK and have been living continuously in the UK for the last 10 years. You must be able to hold or gain a UK government security clearance.
IBM is committed to creating a diverse environment and is proud to be an equal-opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, caste, genetics, pregnancy, disability, neurodivergence, age, veteran status, or other characteristics. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.

This advertiser has chosen not to accept applicants from your region.

Big Data Scala Spark Developer

London, London HCLTech

Posted today

Tap Again To Close

Job Description

HCLTech is a global technology company, home to 219,000+ people across 54 countries, delivering industry-leading capabilities centered on digital, engineering and cloud, powered by a broad portfolio of technology services and products. We work with clients across all major verticals, providing industry solutions for Financial Services, Manufacturing, Life Sciences and Healthcare, Technology and Services, Telecom and Media, Retail and CPG, and Public Services. Consolidated revenues as of $13+ billion.

Spark - Must have

Scala - Must Have

Hive & SQL - Must Have

Recent hands-on experience with Scala coding is required.

Banking /Capital Markets Domain - Good to have

Interview includes coding test.

Job Description:

===

Scala/Spark

• Good Big Data resource with the below Skillset:

§ Spark

§ Scala

§ Hive/HDFS/HQL

• Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.)

• Experience in Big data technologies, real time data processing platform (Spark Streaming) experience would be an advantage.

• Consistently demonstrates clear and concise written and verbal communication

• A history of delivering against agreed objectives

• Ability to multi-task and work under pressure

• Demonstrated problem solving and decision-making skills

• Excellent analytical and process-based skills, i.e. process flow diagrams, business modelling,

This advertiser has chosen not to accept applicants from your region.

Big Data Scala Spark Developer

HCLTech

Posted today

Tap Again To Close

Job Description

Spark - Must have

Scala - Must Have

Hive & SQL - Must Have

Recent hands-on experience with Scala coding is required.

Banking /Capital Markets Domain - Good to have

Interview includes coding test.

Job Description:

===

Scala/Spark

• Good Big Data resource with the below Skillset:

§ Spark

§ Scala

§ Hive/HDFS/HQL

• Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.)

• Experience in Big data technologies, real time data processing platform (Spark Streaming) experience would be an advantage.

• Consistently demonstrates clear and concise written and verbal communication

• A history of delivering against agreed objectives

• Ability to multi-task and work under pressure

• Demonstrated problem solving and decision-making skills

• Excellent analytical and process-based skills, i.e. process flow diagrams, business modelling,

This advertiser has chosen not to accept applicants from your region.

Big Data Architect (Open source contributor)

London, London HCLTech

Posted 7 days ago

Tap Again To Close

Job Description

Location - London

Skill - Apache Hadoop

We are looking for open-source contributors to Apache projects, who have an in-depth understanding of the code behind the Apache ecosystem, should have experience in Cloudera or similar distribution and should possess in depth knowledge of bigdata tech stack.

Requirement:

Experience of platform engineering along with application engineering (hands-on)

Experience in design of an open source platform based on Apache framework for Hadoop.

Experience in integrating Infra-as-a-Code in their platform (Bespoke implementation from scratch)

Experience of design & architect work for the open source Apache platform in hybrid cloud environment

Ability to do debug & fix code in the open source Apache code and should be an individual contributor to open source projects.

Job description: The Apache Hadoop project requires up to 3 individuals with experience in designing and building platforms, and supporting applications both in cloud environments and on-premises.

These resources are expected to be open source contributors to Apache projects, have an in-depth understanding of the code behind the Apache ecosystem, and be capable of identifying and fixing complex issues during delivery.

3 individuals required to support all developers in migrating and debugging various RiskFinder critical applications.

They need to be "Developers" who are expert in designing and building Big Data platforms using Apache Hadoop and support Apache Hadoop implementations both in cloud environments and on-premises.