As a data engineer you will be part of a team responsible for the development and overall delivery of big data platform solutions. You will be designing, maintaining, and optimizing data infrastructure for data collection, management, transformation, and access. You will create pipelines that extract data from API endpoints, loads data into staging areas and converts data into usable formats for data analysts, scientists and other data consumers to utilize. You should have a keen eye for details and an analytical mind.
Required Experience & Skills
4-5 Years relevant experience as a Data Engineer
Masters, or equivalent experience in a STEM (science, technology, engineering,mathematics) discipline
Strong experience with Python
Experience of SQL
Demonstrable experience in gathering and collecting data from different API sources –Previous experience connecting GMP data is preferred (GA360, DV360, CM, SA360, AdsData Hub)
Deep understanding of data manipulation/wrangling techniques.
Experience in providing secure public-facing access to solutions via REST APIs
Demonstrable knowledge and expertise in Cloud CI/CD and IAC (Jenkins, GIT, Terraform,Cloud Build, Docker/Kubernetes)
Experience with development best practices such as source control, continuous integration, unit testing, design patterns, and agile development methodologies
Working knowledge of machine learning principles and programming tools
Working knowledge of data analysis, visualization techniques, and frameworks
Documented experience of designing efficient and robust ETL/ELT workflows
Experience in connecting data platforms with visualisation tools such as Looker Studio.
2-4 years of experience in digital marketing/media is preferred – agency or publisherside
Demonstrated ability to work with people at all levels, develop solutions from conceptionto execution, and collaborate with a team of technically minded peers
Experience within GCP is essential (BigQuery, Cloud Functions, Dataflow, Data Fusion and Dataplex). Other cloud-based technologies such as Windows Azure, AWS are desirable.
Experience with Spark or similar technologies