Senior Data Engineer

Job Description

Senior Data Engineer Shift Time: 1 - 11 pm/3:30 pm - 1:30 am Start Date: Immediate Location: Anywhere in India (flexible to WFO Hybrid Mode) Salary : Upto 12 LPA Job Description: 1. 4+ years of experience working in data warehousing systems 2. 3+ strong hands-on programming expertise in the Databricks landscape, including SparkSQL , Workflows for data processing and pipeline development 3. 3+ strong hands-on data transformation/ ETL skills using Spark SQL, Pyspark, Unity Catalog working in Databricks Medallion architecture 4. 2+ yrs work experience in one of cloud platforms: Azure, AWS or GCP 5. Good exposure to Git version control, and CI/CD best practices 6. Experience in developing data ingestion pipelines from ERP systems (Oracle Fusion preferably) to a Databricks environment, using Fivetran or any alternative data connectors is a plus 7. Experience in a fast-paced, ever-changing and growing environment 8. Understanding of metadata management, data lineage, and data glossaries is a plus. Responsibilities: 1. Involve in design and development of enterprise data solutions in Databricks, from ideation to deployment, ensuring robustness and scalability. 2. Work with Sr. Data Engineer to build, and maintain robust and scalable data pipeline architectures on Databricks using PySpark and SQL 3. Assemble and process large, complex ERP datasets to meet diverse functional and non-functional requirements. 4. Involve in continuous optimization efforts, implementing testing and tooling techniques to enhance data solution quality 5. Focus on improving performance, reliability, and maintainability of data pipelines. 6. Implement and maintain PySpark and databricks SQL workflows for querying and analyzing large datasets Qualifications • Bachelors Degree in Computer Science, Engineering, Statistics, Finance or equivalent experience • Good communication skills Please share the following details along with the most updated resume to geeta.negi@compunnel.com if you are interested in the opportunity: Total Experience Relevant experience Current CTC Expected CTC Notice Period (Last working day if you are serving the notice period) Current Location SKILL 1 RATING OUT OF 5 SKILL 2 RATING OUT OF 5 SKILL 3 RATING OUT OF 5 (Mention the skill)