KloudPortal Technology Solutions PVT. LTD

Job Description Job Summary We are hiring a Senior PySpark Developer with 4-7 years of experience in building and optimising data pipelines using PySpark on Databricks , within AWS cloud environments . This role involves modernising legacy systems, integrating with Kafka, and collaborating across cross-functional teams. Key Responsibilities Develop and optimise scalable PySpark applications on Databricks. Work with AWS services (S3, EMR, Glue, Lambda) for data processing. Integrate streaming and batch data sources using Kafka. Tune Spark jobs for performance and cost efficiency. Collaborate with DevOps, product, and analytics teams. Ensure data governance, lineage, and quality compliance . Required Skills 3-7 years of strong PySpark development experience. Hands-on Databricks (Spark UI, performance tuning). Good understanding of AWS services (S3, EMR, Glue, Lambda). Experience with Kafka for streaming/batch processing. Spark optimisation (partitioning, caching, joins). Data formats: Parquet, Avro, ORC . Orchestration: Airflow / Databricks Workflows . Scala is a strong plus. Qualifications Bachelors or Master’s in Computer Science / IT.