Job
Description
Data Pipeline Developer (L1 Data Ops Analyst) Role Description: This role involves developing and maintaining data pipelines for ingesting, wrangling, transforming, and joining data from multiple sources. The candidate will work independently to deliver high-quality code and ensure seamless operation of data systems. This position requires proficiency in ETL tools such as Informatica, Glue, Databricks, and DataProc, along with strong coding skills in Python, PySpark, and SQL. The candidate should be capable of operating with minimal supervision while adhering to established processes and guidelines. Responsibilities: Independently develop, test, and maintain data processing pipelines ensuring performance and scalability. Monitor dashboards and databases to ensure smooth operation during a 9-hour shift. Identify, troubleshoot, and escalate system or data anomalies based on predefined thresholds. Follow Standard Operating Procedures (SOPs) and runbooks to resolve basic data issues. Collaborate with L2 and L3 support teams for issue escalation and resolution. Write and execute advanced SQL queries for data validation, extraction, and troubleshooting. Document all development processes, including code, testing procedures, and operational logs. Ensure adherence to engineering standards, timelines, and Service Level Agreements (SLAs). Conduct unit tests to maintain data integrity and quality. Estimate time, effort, and resource requirements for assigned tasks. Participate in knowledge-sharing activities and contribute to project documentation. Ensure compliance with release management processes for seamless deployments. Obtain and maintain relevant technical and domain certifications (e.g., Azure, AWS, GCP). Mandatory Skills: Proficiency in Python, PySpark, and SQL for data manipulation and transformation. Hands-on experience with ETL tools (e.g., Informatica, AWS Glue, Databricks, DataProc). Strong knowledge of cloud platforms (e.g., AWS, Azure, Google Cloud) and data-related services. Ability to conduct tests on data pipelines and evaluate results against data quality standards. Familiarity with advanced SQL functions, including windowing and analytical functions. Strong analytical and problem-solving skills for diagnosing data-related issues. Good-to-Have Skills: Experience with Apache Airflow, Talend, or other ETL orchestration tools. Knowledge of data schemas, models, and data governance practices. Understanding of low-level design (LLD) and ability to map it to user requirements. Experience with MS Excel for data analysis and reporting. Soft Skills: Strong written and verbal communication skills in English. Ability to work independently with minimal supervision. High attention to detail and ability to manage multiple monitoring tasks. Adaptability to work in a 24x7 shift schedule, including night shifts. Effective collaboration and communication with cross-functional teams and stakeholders. Experience Range: 2 to 5 years of experience in data pipeline development and data operations. Hiring Locations: Chennai Trivandrum Kochi Measures of Success: Adherence to engineering standards, schedules, and SLAs. Reduction of recurring defects and quick resolution of production bugs. Accuracy and completeness of project documentation. Successful completion of required technical and domain certifications. Effective communication and issue resolution within support teams. Consistent delivery of optimized and error-free code.
