PySpark Developer

As a skilled PySpark Developer with hands-on experience in Reltio MDM, your role will involve designing and implementing scalable data processing solutions using PySpark and integrating with Reltio's cloud-native MDM platform. Key Responsibilities: - Develop and maintain data pipelines using PySpark in distributed computing environments such as AWS EMR and Databricks. - Integrate and synchronize data between enterprise systems and the Reltio MDM platform. - Design and implement data transformation, cleansing, and enrichment processes. - Collaborate with data architects, business analysts, and Reltio solution architects to ensure high-quality data modeling. - Work on API-based integration between Reltio and upstream/downstream applications. - Optimize PySpark jobs for performance and cost-efficiency. - Ensure data quality, integrity, and governance throughout the pipeline. - Troubleshoot and resolve data and performance issues in existing workflows. Qualifications Required: - 6+ years of experience in PySpark development and distributed data processing. - Strong understanding of Apache Spark, DataFrames, and Spark SQL. - Experience with Reltio MDM, including entity modeling, survivorship rules, match & merge configuration. - Proficiency in working with REST APIs and JSON data formats. - Experience with cloud platforms like AWS and data services (e.g., S3, Lambda, step function). - Good knowledge of data warehousing concepts, ETL workflows, and data modeling. - Familiarity with CI/CD practices and version control tools like Git. - Strong problem-solving and communication skills.,