Job
Description
Role Overview: You will be responsible for designing and configuring data pipelines using PySpark with GCP (Airflow, Dataproc, BigQuery). Your role will involve writing complex SQL queries for data analysis on various databases such as SQL Server, Oracle, and HIVE. Additionally, you will work on ETL processes, Unix, and scheduling tools like Control-M to ensure efficient data pipeline execution. Your expertise in SQL, Python, PySpark, and Hive will be essential in independently handling specialized assignments within project deliverables. Key Responsibilities: - Utilize deep ETL tool experience and PySpark with GCP pipeline expertise to configure data pipelines - Write complex SQL queries for data analysis on different databases - Work on ETL processes, Unix, and scheduling tools like Control-M - Independently handle specialized assignments within project deliverables - Design and implement solutions to enhance engineering efficiencies - Communicate effectively at a global level through oral, written, and visual communication - Deliver high-quality materials to management within tight deadlines - Adapt to rapidly changing priorities and work effectively under pressure Qualifications Required: - At least 3-5 years of experience in PySpark with GCP (Airflow, Dataproc, BigQuery) - Proficiency in SQL, Python, PySpark, Hive, ETL, Unix, and scheduling tools like Control-M - Strong problem-solving and analytical skills - Ability to critically evaluate information from multiple sources and apply technical domain knowledge - Experience in any of the ETL tools like Prophecy, Datastage 11.5/11.7, or Pentaho would be a plus (Note: The additional details of the company were not provided in the job description.) Role Overview: You will be responsible for designing and configuring data pipelines using PySpark with GCP (Airflow, Dataproc, BigQuery). Your role will involve writing complex SQL queries for data analysis on various databases such as SQL Server, Oracle, and HIVE. Additionally, you will work on ETL processes, Unix, and scheduling tools like Control-M to ensure efficient data pipeline execution. Your expertise in SQL, Python, PySpark, and Hive will be essential in independently handling specialized assignments within project deliverables. Key Responsibilities: - Utilize deep ETL tool experience and PySpark with GCP pipeline expertise to configure data pipelines - Write complex SQL queries for data analysis on different databases - Work on ETL processes, Unix, and scheduling tools like Control-M - Independently handle specialized assignments within project deliverables - Design and implement solutions to enhance engineering efficiencies - Communicate effectively at a global level through oral, written, and visual communication - Deliver high-quality materials to management within tight deadlines - Adapt to rapidly changing priorities and work effectively under pressure Qualifications Required: - At least 3-5 years of experience in PySpark with GCP (Airflow, Dataproc, BigQuery) - Proficiency in SQL, Python, PySpark, Hive, ETL, Unix, and scheduling tools like Control-M - Strong problem-solving and analytical skills - Ability to critically evaluate information from multiple sources and apply technical domain knowledge - Experience in any of the ETL tools like Prophecy, Datastage 11.5/11.7, or Pentaho would be a plus (Note: The additional details of the company were not provided in the job description.)
