Senior BigData Developer

As a Senior Big Data Developer with expertise in PySpark and the Apache Big Data ecosystem, your role will involve designing, developing, and optimizing scalable data pipelines. You will collaborate with cross-functional teams to deliver reliable, high-performance data solutions. Your responsibilities will include: - Building and optimizing large-scale data processing pipelines using PySpark - Working with Apache Hadoop, Hive, HDFS, Spark, and other tools for managing big data workloads - Handling ETL development, data ingestion, transformation, and integration from various sources - Collaborating with Data Scientists, Analysts, and business stakeholders to deliver data-driven solutions - Ensuring high availability, performance, and scalability of data jobs - Implementing data quality checks and lineage tracking for auditability - Developing reusable code in Python and Java - Contributing to architectural design, performance tuning, and tech discussions In order to excel in this role, you will need to have the following skills and qualifications: - 5-7 years of experience as a Big Data Developer - Expertise in PySpark and distributed data processing - Strong knowledge of the Apache ecosystem: Hadoop, Hive, HDFS, Spark - Experience with ETL pipelines, data modeling, and data warehousing - Proficiency in Python and working knowledge of Java - Familiarity with SQL/NoSQL databases - Hands-on experience with Git, CI/CD tools, and code versioning Nice-to-have skills for this role include knowledge of Apache NiFi, experience with real-time data streaming tools, exposure to cloud platforms such as AWS, Azure, or GCP, and familiarity with Docker, Kubernetes, or other orchestration tools. Additionally, soft skills such as excellent problem-solving and analytical abilities, strong collaboration and communication skills, and the ability to work independently and in a team will be beneficial. If you are passionate about big data and seeking impactful projects, please send your updated resume to nikhil.kumar@krtrimaiq.ai.,