Integration Engineer

As a Senior Data Engineer at Career Mantra, you will be responsible for designing, developing, and maintaining scalable data pipelines while optimizing data storage solutions. Your role will involve working with cloud platforms, big data technologies, and ETL processes to support business intelligence and analytics. Key Responsibilities: - Design and implement scalable data pipelines using Python, PySpark, and Big Data tools. - Optimize data processing performance in AWS. - Collaborate with business teams to deliver reliable data solutions. - Develop and maintain ETL processes using tools like Airflow, Azkaban, and AWS Glue. - Troubleshoot and ensure data integrity across data pipelines. - Mentor junior team members and promote best practices. Key Skills: - Strong experience with Python, PySpark, SQL, AWS (Redshift, Glue, S3). - Expertise in big data technologies (Hadoop, Hive, Spark). - Proficient in data modeling and ETL orchestration. - Experience with business intelligence tools (Tableau, Power BI). - Strong problem-solving and communication skills. Qualifications And Skills: - Extensive experience with data mapping for accurate data translation and transformation across systems. - Proficient in ETL processes for efficient extraction, transformation, and loading of data. - Strong knowledge of API Integration for smooth communication between different software solutions. - Expertise in System Integration to ensure harmonious operation of various IT systems and platforms. - Experience with Middleware Technologies to facilitate connectivity and collaboration between applications. - Competency in Scripting Languages for automating routine processes and enhancing system functions. - Proven Database Management skills to ensure data integrity and performance optimization. - Adept at Troubleshooting to identify, diagnose, and resolve technical issues promptly. Roles And Responsibilities: - Design and implement solutions for complex data integration projects to enhance operational efficiency. - Collaborate with cross-functional teams to ensure smooth integration and alignment of IT systems. - Develop and execute testing plans to verify the functionality and performance of integrated systems. - Monitor system performance and suggest necessary improvements for optimal operation and stability. - Serve as a technical expert, providing guidance and support to junior team members and stakeholders. - Maintain comprehensive documentation of integration processes, configurations, and best practices. - Stay current with technology trends and advancements to incorporate industry best practices. - Ensure compliance with company policies and standards in all integration activities.,