- Category: Engineering
- Location: England London
- Date: 1 month ago
As a Data Engineer you will have a major influence in building the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Google ‘big data’ technologies.
You will help us create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.
You’ll be given the opportunity to work on AI that uses machine learning to identify messages that could be considered as abusive behaviours, and many more other projects such as improving matchmaking on our competitive gaming platform.
ResponsibilitiesCreate and maintain optimal data pipeline architecture Create and maintain AI and ML infrastructures on GCP that our data scientist can leverage to train and host ML models Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and Google ‘big data’ technologies Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Work with data and analytics experts to strive for greater functionality in our data systems.
Requirements:Advanced working SQL knowledge and experience working with relational and non-relational databases Experience building and optimizing ‘big data’ data pipelines, architectures and data sets. Strong analytic skills related to working with unstructured datasets
Exposure to the following software/tools:Big data tools: Hadoop, Spark, Kafka, etc. SQL and NoSQL databases, including Postgres and MongoDB.
Graph technologies, including Neo4jData pipeline and workflow management tools: Airflow, Luigi, etc. Google Cloud services: Bigquery, Dataflow, Cloud Ai, Cloudrun, etc Stream-processing systems: Apache Beam, Spark-Streaming, etc. Object-oriented/object function scripting languages: Python, Java, etc.
Benefits:Lunch provided daily; Kitchen stocked with refreshments, cereals, bread, spreads (coffee machines everywhere – our founding team is Italian after all!);
Flexible working environmentMonthly drinks and dinner in the office; Monthly massages; Quarterly team outings; 5 a side twice monthly; Your professional growth is important to us. We provide ongoing training opportunities;
When applying for a job, do not provide bank account details or any other financial information. Never make any form of payment. WhatJobs is not responsible for any external website content. Report this job