Principal Engineer, Data Analytics Engineering

You will be a key member of our team at Sandisk, dedicated to building robust data pipelines and handling large-scale data processing. Your passion for optimizing and maintaining efficient data workflows will drive you to thrive in our dynamic environment. With hands-on experience in Python, MariaDB, SQL, Linux, Docker, Airflow administration, and CI/CD pipeline creation and maintenance, you will play a crucial role in the application built using Python Dash. Your responsibilities will include application deployment, server administration, and ensuring the smooth operation and upgrading of the application. Your main responsibilities will involve: - Developing data pipelines using Spark with a minimum of 9+ years of experience. - Designing, developing, and optimizing Apache Spark applications for large-scale data processing. - Implementing efficient data transformation and manipulation logic using Spark RDDs and Data Frames. - Managing server administration tasks, including monitoring, troubleshooting, and optimizing performance. - Administering and managing databases (MariaDB) to ensure data integrity and availability. - Designing, implementing, and maintaining Apache Kafka pipelines for real-time data streaming and event-driven architectures. - Demonstrating deep technical skill in Python, PySpark, Scala, and SQL/Procedure. - Utilizing Unix/Linux operating system knowledge like awk, ssh, crontab, etc. - Writing transact SQL, developing and debugging stored procedures and user-defined functions in Python. - Working experience on Postgres and/or Redshift/Snowflake database. - Exposure to CI/CD tools like bit bucket, Jenkins, ansible, docker, Kubernetes, etc. - Integrating data pipelines with Splunk/Grafana for real-time monitoring, analysis, and Power BI visualization. - Creating and scheduling the Airflow Jobs. Qualifications: - Bachelor's degree in computer science or engineering. Master's degree preferred. - AWS developer certification will be preferred. - Any certification on SDLC methodology, integrated source control system, continuous development, and continuous integration will be preferred. At Sandisk, we value diversity and believe that embracing diverse perspectives leads to the best outcomes for our employees, customers, and communities. We are committed to creating an inclusive environment where every individual can thrive through a sense of belonging, respect, and contribution. If you require accommodation during the hiring process, please reach out to us at jobs.accommodations@sandisk.com with details of your request, including the specific accommodation needed and the job title and requisition number of the position you are applying for.,