Job
Description
Description: The Senior Data Engineer is responsible for leading the design, development, and optimization of data infrastructure and systems to meet complex business needs. They play a strategic role in developing and managing large-scale data pipelines, ensuring scalability, reliability, and performance. As a key technical expert, the Senior Data Engineer collaborates with cross-functional teams to deliver innovative data solutions and ensures adherence to best practices in data management and governance. Responsibilities Leads the design and development of advanced data pipelines, ensuring seamless ETL/ELT of data into enterprise storage systems. Oversees data integration from diverse sources, maintaining data consistency, integrity, and compliance with enterprise standards. Transforms raw data into usable formats through cleansing, aggregation, filtering, and enrichment techniques. Optimizes data pipelines and workflows for scalability, performance, and cost efficiency. Establishes robust data validation, quality checks, and error-handling mechanisms to ensure enterprise data integrity. Mentors junior data engineers, promoting technical growth and team development. Implements and enforces best practices in data governance, including metadata management and lineage tracking. Drives the adoption of real-time data streaming and processing workflows to enable advanced analytics and decision-making. Partners with stakeholders to identify and address complex data challenges, delivering scalable and innovative solutions. Stays updated with emerging data technologies, evaluating their potential to enhance enterprise capabilities. Develops and enforces data security protocols to protect sensitive information. Job Requirements Education A bachelor s degree in computer science, data science, software engineering, information systems, or related quantitative field; master s degree preferred Experience 8+ years of data engineering experience, including expertise in data integration, pipeline optimization, and enterprise-scale data solutions. Proven leadership in implementing big data solutions (e.g., Snowflake, Databricks) and distributed data systems (e.g., Apache Spark, Flink). Skills Advanced proficiency in Apache technologies (Kafka, Airflow, Spark) and programming languages (Python, Java, Scala). Expertise in data query tools (SQL, Hive) and database technologies (NoSQL, Hadoop, Teradata). Strong knowledge of cloud platforms (AWS, Azure, GCP) and modern data architectures. Experience with real-time streaming data workflows and AI/ML analytics initiatives. Exceptional analytical and problem-solving skills with debugging expertise in complex systems. Excellent communication skills to effectively collaborate with technical and non-technical stakeholders. Strong leadership abilities to mentor teams and influence cross-functional initiatives.
