Job
Description
Key Responsibilities: Ab Initio Development & Optimization: Design, develop, test, and deploy high-performance, scalable ETL/ELT solutions using Ab Initio components (GDE, Co>Operating System, EME, Control Center). Translate complex business requirements and data transformation rules into efficient and maintainable Ab Initio graphs and plans. Optimize existing Ab Initio applications for improved performance, resource utilization, and reliability. Troubleshoot, debug, and resolve complex data quality and processing issues within Ab Initio graphs and systems. Data Modeling & Advanced SQL: Apply expertise in advanced SQL to write complex queries for data extraction, transformation, validation, and analysis across various relational databases (e.g., DB2, Oracle, SQL Server). Design and implement efficient relational data models (e.g., Star Schema, Snowflake Schema, 3NF) for data warehousing and analytics. Understand and apply big data modeling concepts (e.g., denormalization for performance, schema-on-read, partitioning strategies for distributed systems). Spark & Big Data Integration: Collaborate with data architects on data integration strategies in a hybrid environment, understanding how Ab Initio processes interact with or feed into big data platforms. Analyze and debug data flow issues that may span across traditional ETL and big data platforms (e.g., HDFS, Hive, Spark). Demonstrate strong foundational knowledge in Apache Spark, including understanding Spark SQL and DataFrame operations, to comprehend and potentially assist in debugging Spark-based pipelines. Collaboration & Documentation: Work effectively with business analysts, data architects, QA teams, and other developers to deliver high-quality data solutions. Create and maintain comprehensive technical documentation for Ab Initio graphs, data lineage, data models, and ETL processes. Participate in code reviews, design discussions, and contribute to best practices within the team. Required Skills & Qualifications: Bachelor's or Master's degree in Computer Science, Engineering, or a related technical field. 5+ years of hands-on, in-depth development experience with Ab Initio GDE, Co>Operating System, and EME. Expert-level proficiency in SQL for complex data manipulation, analysis, and optimization across various relational databases. Solid understanding of relational data modeling concepts and experience designing logical and physical data models. Demonstrated proficiency or strong foundational knowledge in Apache Spark (Spark SQL, DataFrames) and familiarity with the broader Hadoop ecosystem (HDFS, Hive). Experience with Unix/Linux shell scripting. Strong understanding of ETL processes, data warehousing concepts, and data integration patterns. Excellent problem-solving, analytical, and troubleshooting skills. Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams Show more Show less
