Job
Description
Job Summary: We are seeking a skilled Informatica IDMC and PySpark to lead the migration of ETL pipelines from Informatica IDMC to scalable, Spark-based solutions. The ideal candidate will possess hands-on expertise in cloud environments like AWS or Azure , a solid understanding of data warehousing, transformation, and orchestration, and familiarity with modern ETL migration tools such as BladeBridge Converter , EZConvertETL by Wavicle , Qlik Replicate for Azure Databricks , and others. Key Responsibilities: Analyze existing ETL workflows built in Informatica IDMC and design equivalent, scalable pipelines in PySpark . Migrate and re-engineer data integration pipelines from Informatica Cloud to PySpark on distributed data platforms. Utilize advanced ETL migration and modernization tools such as BladeBridge Converter , EZConvertETL (Wavicle Data Solutions) , Qlik Replicate for Azure Databricks , Travinto Technologies ETL solutions , LeapLogic by Impetus , Next Pathway’s Shift , Wavicle ETL Converter , and Propel ETL by Xoriant to accelerate and streamline migration efforts. Optimize PySpark code for performance and cost efficiency in cloud data platforms such as Databricks , EMR , or HDInsight . Collaborate with data architects and business stakeholders to validate transformed data and maintain high data quality. Ensure all ETL processes are reliable, maintainable, and well-documented. Leverage CI/CD pipelines for deployment and version control of PySpark workflows. Work with large-scale structured and unstructured datasets from multiple sources. Automate data quality checks and monitoring mechanisms. Follow and enforce data governance and compliance policies. Required Skills: 5+ years of hands-on experience in Informatica Cloud (IDMC) and enterprise-grade ETL development. Strong hands-on expertise in PySpark and Apache Spark for building high-performance data pipelines. Deep understanding of data lakes , data warehouses , and modern big data ecosystems . Experience in cloud environments such as AWS , Azure , or GCP . Proficiency with Databricks is a strong advantage. Excellent skills in SQL and query performance tuning. Familiarity with Git , CI/CD tools , and modern software development practices. Experience with job orchestration tools like Apache Airflow , Control-M , or Azure Data Factory . Exposure to industry-leading ETL conversion platforms such as BladeBridge Converter , EZConvertETL , Qlik Replicate , LeapLogic , Next Pathway’s Shift , and Propel ETL is highly desirable. Insurance domain experience will be preferable. Show more Show less
