Data Mainframe Engineer

Job Summary:

We are seeking a skilled and experienced IT professional to join our team as a Mainframe and Teradata DataStage Associate. The successful candidate will be responsible for developing, maintaining, and optimizing ETL processes using IBM DataStage, as well as managing and supporting data operations on Mainframe and Teradata platforms.

We are looking for a highly skilled and experienced Senior Data Engineer to join our dynamic team. The ideal candidate will have a strong background in managing and optimizing data systems on Mainframe, Teradata, and IBM DataStage, as well as expertise in modern data platforms such as AWS, Databricks, and Apache Spark. You will play a key role in designing, developing, and maintaining robust data pipelines and ETL processes, ensuring efficient data integration and transformation across our enterprise systems.

Key Responsibilities:

• Design, develop, and optimize ETL processes using IBM DataStage, ensuring seamless data integration from diverse sources.
• Manage and maintain data operations on Mainframe and Teradata platforms, ensuring data accuracy and system performance.
• Develop and deploy scalable data solutions on AWS using Databricks, Spark, and Delta Lake technologies.
• Implement and manage streaming data solutions using Kafka and Spark Streaming for real-time data processing.
• Collaborate with cross-functional teams to gather data requirements and translate them into technical specifications and solutions.
• Develop and manage SQL queries and Delta Live Tables to support data analytics and reporting needs.
• Monitor and optimize the performance of ETL jobs, database queries, and data streams to ensure system reliability and efficiency.
• Troubleshoot and resolve issues across data platforms, ensuring minimal downtime and data integrity.
• Stay current with emerging technologies and industry best practices in data engineering and recommend improvements to existing systems.
• Document data processes, workflows, and system configurations to ensure knowledge sharing and operational efficiency.

Qualifications:

• Bachelors degree in Computer Science, Information Technology, or a related field.
• years of experience with Mainframe systems, including COBOL, JCL, and VSAM.
• years of experience with Teradata, including advanced SQL development and performance tuning.
• Strong expertise in IBM DataStage, with a proven track record of designing and implementing complex ETL processes.
• Proficiency in AWS cloud services and experience with Databricks for building data pipelines.
• Solid understanding of Apache Spark, including Spark Streaming, and experience with Kafka for real-time data processing.
• Experience with Delta Live Tables and Delta Tables for managing and querying large datasets.
• Strong problem-solving skills and the ability to work effectively in a fast-paced, collaborative environment.
• Excellent communication skills, with the ability to convey complex technical concepts to non-technical stakeholders. Preferred Qualifications:
• Experience with additional cloud platforms, such as Azure or Google Cloud.
• Familiarity with other ETL tools and data integration frameworks.
• Experience with message queuing systems such as IBM MQ.
• Knowledge of agile development methodologies and version control systems.