Sr. Python Bigdata Engineer

Job Title:

Location:

Employment Type:

Experience Level:

About the Role:

We are looking for a Senior Python Big Data Developer with strong experience in Python, Spark, and Hadoop ecosystems for one of our leading MNC clients to join the data engineering team. The ideal candidate will build scalable data pipelines, optimize data processing workflows, and deliver high-quality, production-ready code. This role requires hands-on expertise with Big Data tools, ETL/ELT pipelines, and data manipulation frameworks such as PySpark and Pandas.

Key Responsibilities:

• Design, develop, and maintain end-to-end Big Data pipelines using Python, PySpark, and Hadoop technologies.
• Perform coding, testing, debugging, and performance tuning for large-scale data processing applications.
• Build and optimize data ingestion, transformation, and aggregation workflows using HDFS, Hive, Spark, and Kafka.
• Work extensively with Pandas, PySpark, and Polars for data manipulation, cleansing, and analysis.
• Develop ETL/ELT frameworks and ensure data accuracy, consistency, and integrity across systems.
• Collaborate with data architects, analysts, and business teams to translate business requirements into technical solutions.
• Integrate data pipelines with Enterprise Data Warehouse (EDW) and support reference/master data management processes.
• Support data reporting and visualization needs using tools like Tableau, Dataiku, or MicroStrategy (MSTR).
• Manage and automate workflows through Control-M or similar job orchestration tools.
• Utilize SQL and Bash scripting for data management, validation, and process automation.
• Conduct code reviews, ensure best coding practices, and participate in technical design discussions.

Required Skills & Qualification:

• Bachelors or Masters degree in Computer Science, Information Technology.
• 5–10 years of professional experience as a Python Developer or Big Data Engineer.
• Advanced proficiency in Python programming and object-oriented design.

Strong experience with:

• Apache Spark / PySpark for distributed data processing.
• Pandas / Polars for large-scale data manipulation.
• SQL (complex queries, optimization, and performance tuning).
• Hands-on experience with Hadoop ecosystem tools: HDFS, Hive, Kafka, Spark, Bash scripting.
• Proven experience building ETL/ELT data pipelines in on-premise Hadoop environments.
• Familiarity with Enterprise Data Warehouse (EDW) and Reference Data Management concepts.
• Knowledge of job orchestration/scheduling tools (e.g., Control-M, Autosys).
• Excellent analytical, problem-solving, and debugging skills.
• Strong verbal and written communication skills with the ability to work cross-functionally.

What We Offer:

• Opportunity to work on large-scale enterprise data platforms.
• Exposure to cutting-edge Big Data and analytics technologies.
• Collaborative and learning-focused work culture.
• Competitive compensation and benefits package.
• Long-term career growth in data engineering and analytics

career@jinraitech.com

Job Type: Full-time