Lead Data Engineer
Technology Recruiting Services
Oakland, california
***Must be located in the SF Bay Area or your application will not be considered***
Our client is a venture-backed SaaS startup that recently secured Series A funding. They're solving a multi-billion dollar problem of data quality for the entire marketing industry. Their platform enables major brands and publishers to optimize consumer data quality, improving marketing ROI. In a fast-paced and collaborative environment, they are committed to excellence and innovation.
Key Responsibilities:
- Design and maintain scalable data pipelines using Spark and AWS EMR for processing terabytes of consumer data
- Write, test, and optimize custom Scala code for ETL workflows
- Deploy and manage ETL processes in AWS cloud environments using Airflow
- Automate delivery of structured data to enterprise clients via Snowflake and Databricks
- Collaborate with the Data Science team to optimize...