Etl Administrator

Technical Skills:

Strong knowledge of Talend/EMR/Airflow/DataStage/Hue/DataHub including its architecture, components (e.g., Designer, Director, Administrator), and administration.

Proficiency in ETL processes, data warehousing concepts, and data integration methodologies.

Platform Management: Install, configure, and manage IBM DataStage and its related components, such as Information Server.

Experience in any reporting tools like PowerBI / Tableau/ Cognos/ MicroStrategy/ Sigma/ QlikView

Experience in any analytics tools like Sagemaker, aiOla,SparkBeyond, Rstudio

Performance Optimization: Monitor and tune DataStage job performance, system resource utilization (CPU, memory, storage), and troubleshoot bottlenecks.

Environment Maintenance: Manage DataStage projects, environments, user access (including role-based access control), permissions, and project configurations.

Problem Resolution: Diagnose and resolve issues with DataStage jobs, server performance, and connectivity to data sources.

Software Updates & Patches: Apply DataStage software updates, patches, and service packs to ensure the environment is current and secure.

Collaboration: Work closely with DataStage developers, data engineers, database administrators, and other teams to ensure data integration processes meet technical and business requirements.

Security & Compliance: Enhance security practices, implement best practices for data management, and ensure compliance with enterprise technology standards and regulatory expectations.

Documentation: Maintain technical documentation, including runbooks, disaster recovery plans, and job run reports.

Monitoring & Reporting: Monitor job executions, generate reports on job runs, and create tickets for any IBM-related issues

Experience with scripting languages (e.g., PL/SQL, PowerShell, Python, Bash) is often preferred.

User and Group Management: Adding, deleting, and managing Hue users and groups, configuring group permissions, and importing users and groups from LDAP or other directories.

Permissions Management: Defining which Hue applications and features are visible and available to users based on their group memberships.

Hue Process Management: Monitoring and managing Hue processes to ensure optimal performance and availability, including utilizing the supervisor script.

Configuration Management: Performing basic and advanced configuration of Hue to effectively administer the application within the cluster environment. This may include setting up Hue for security, configuring authentication, and managing its integration with other services like Impala and HiveServer2.

Security Configuration: Implementing security measures, such as configuring user authorization and authentication within Hue, to protect data.

Troubleshooting: Identifying and resolving issues related to Hue operation, user access, and performance.

Communication: Excellent communication skills, both written and verbal, for interacting with users, other administrators, and potentially vendors.

Documentation: Maintaining clear documentation related to Hue configuration, user management procedures, and other administrative tasks.

Staying Current: Keeping abreast of new Hue features and best practices to ensure the platform is used effectively and efficiently.

Hue Administration: Strong understanding of Hue's features, administration tools, and configuration options.

User Management Expertise: Proficiency in managing users and groups within Hue and potentially other authentication systems like LDAP.

Permissions and Security Configuration: Ability to configure user permissions and access control within Hue to ensure data security.

Technical Skills: Comfortable with command-line interfaces for managing Hue processes and configuring its settings.

Troubleshooting and Problem Solving: Ability to diagnose and resolve issues that may arise with Hue functionality or performance.

Ability to work in a 24/7 support rotation and handle urgent production issues.

Responsibilities

Installation, configuration, and support of the Talend platform (including servers and components).

User and project management within the Talend Administration Center (TAC).

Managing platform licenses, users, and project authorizations.

Setting up and managing shared development environments and Git branches.

Enforcing platform governance and standards.

Deploying and executing Talend jobs, routes, and services.

Monitoring job execution, troubleshooting issues, and ensuring successful runs.

Managing job scheduling and dependencies using tools like Job Conductor in TAC.

Monitoring platform health, capacity planning, and resource allocation.

Using tools like the Activity Monitoring Console (AMC) and Talend Log Server for monitoring and troubleshooting.

Identifying and resolving performance bottlenecks in Talend jobs and the overall environment.

Implementing continuous improvement and automation opportunities.

Executing disaster recovery and password rotation exercises.

Providing technical support to development and data engineering teams using Talend.

Assisting with audit requests.

Creating and maintaining documentation for platform setup, configurations, and troubleshooting.

Cloud Specific Responsibilities (if applicable):

Managing Talend Cloud environments, including users, remote engines, and workspaces.

Publishing and managing artifacts within the cloud environment.

Managing and promoting environments and workspaces.

Qualification:

Education qualification: Any degree from a reputed college