Data Engineer

Job description

Apple's Media, Graphics, and Compute Technologies Group (MGC) is looking for a talented and dedicated big data engineer to join our Data Engineering team. The Data Engineering team within the MGC organization plays a critical role in supporting data-driven analytics by providing data collection, warehousing, and analytics at big data scale. Our team provides the infrastructure to power numerous trend and operational dashboards as well as other ad-hoc use cases in support of services like Apple TV, Apple Music, and FaceTime. We are leveraging Generative AI and Machine Learning technologies to provide best-in-class data analytics and monitoring. This role offers the opportunity to help design, enhance, and develop our very-high-volume processing pipeline. You'll work with talented engineers within our team as well as cross-functional teams in an agile and dynamic environment that values engineering excellence, creativity, and innovation, and you will be a key contributor to our next generation of processing pipeline and data analytics platform. Description Our team leverages modern Data Engineering, Generative AI and Machine Learning technologies to deliver actionable insights. You will be: • Collaborating with data scientists across functional teams to define and enhance performance metrics that provide valuable insights for stakeholders • Building and maintaining: - Ingestion pipelines for real-time data processing - Real-time applications driving operational monitoring - Batch ETL/ELT applications populating our data warehouse • Applying Generative AI and Retrieval Augmented Generation (RAG) techniques to enhance data analytics capabilities • Applying Machine Learning technologies for anomaly detection Responsibilities: Tuning and scaling Apache Kafka producer/consumer, Spark Structured Streaming and Flink applications running in cloud environment Managing and monitoring large scale data collection and analytics pipelines at the application level Performing capacity planning to scale infrastructure and applications running on Kubernetes Troubleshooting production issues and conducting performance analysis of distributed systems Collaborating with cross-functional teams to ensure high availability and reliability of data pipelines Keeping up with the latest data engineering trends and applying the corresponding technologies Preferred Qualifications Producti…