Big Data Engineer
ASOS is the UK's number one fashion and beauty destination, expanding globally at a rapid pace and our award-winning Tech teams sit at the heart of our business. We deliver technical innovations and pioneer incredible solutions to keep our 20-something market engaged, the cloud based architecture to support our global reach and the agile engineering methods to deliver value fast. We're extremely ambitious and thrive on the individuality of our amazing employees.
We are building ASOS's AI platform - a unique big data infrastructure that will ingest and serve massive amount of data to power our predictive data products, leveraging the latest in machine learning and AI technologies.
We are looking for outstanding cloud and big data folks to join us early on and help build the new platform and grow our team of polyglot engineers and data scientists. If you are passionate about technology and architecture, love hands-on development and want to build very scalable, highly available systems- you should work with us! You'll have the unique opportunity to work alongside data scientists to build innovative data products that will impact the business performance at all levels.
What you'll be doing
- Evaluate, select and deploy massive data processing infrastructure
- Design and build cloud-scale Services and API's
- Handle all aspects of development - design, development, build, deployment, monitoring and operations.
- Research and experiment with emerging technologies and industry trends with a view to bringing business value through early adoption
- Work in an agile, cross-functional team composed of engineers and machine learning scientists, taking responsibility for delivering predictive data products
Our current stack consists of applications written in Python and Scala and deployed to Azure using Azure DevOps and Ansible. Our microservices are packaged in containers and deployed to Kubernetes. Our big data pipelines run on Databricks' Spark and PySpark clusters and are orchestrated using ADF. We use Git and VSTS for source control and Jira for backlog management.
Key Skills and Experience
Profound understanding of big data batch and streaming concepts and technologies:
- Experience in delivering big data solutions using Apache Spark, ideally Databricks
- Knowledge of spark frameworks like RDD, DataFrame, Datasets, Streaming
- Understanding of other big data technologies (Hadoop, MapReduce, YARN, Hive, Pig, HDFS, Flume, Sqoop,...)
- Working knowledge with cloud platforms
Core programming knowledge:
- OOP principles
- Proficient in one or more of the following: Java, Scala, Python
- Shell scripting
Previous significant experience in a Data Engineering role:
- Knowledge of RDBMS, ETL and Data warehouse technologies
Broad knowledge of software delivery lifecycle and CI/CD toolset:
- Git & Continuous integration tools such as Jenkins, TeamCity, Ansible, Terraform
- Testing frameworks like Junit, Scalatest, ScalaCheck, Mock testing
Nice to Have
- Exposure to Microsoft Azure stack
- Cassandra, MongoDB or equivalent NoSQL databases
- Container technologies like Docker
- Knowledge of advanced analytics and insights techniques (e.g. predictive analytics, machine learning, segmentation)
- Knowledge of deep learning frameworks like TensorFlow, Keras
- Knowledge of machine learning libraries like MLLIB, sklearn
- Experience in retail and/or e-commerce
When applying for a job, do not provide bank account details or any other financial information. Never make any form of payment. WhatJobs is not responsible for any external website content. Report this job