In-person Hiring Drive - Sr. GCP Data Engineer - 11th July

Role & responsibilities

Understand the Business Problem and the Relevant Data

• Maintain and understanding of company and department strategy
• Translate analysis requirements into data requirements
• Identify and understand the data sources that are relevant to the business problem
• Develop conceptual models that capture relationships within the data
• Define the data-quality objectives for the solution
• Be a subject matter expert in data sources and reporting options

Architect Data Management Systems

• Use understanding of the business problem and the nature of the data to select appropriate data management systems (Big Data, OLTP, OLAP, etc.)
• Design and implement optimum data structures in the appropriate data management system (GCP BQ, Snowflake, SQL server etc.) to satisfy the data requirements
• Plan methods for archiving/deleting of information

Develop, Automate, and Orchestrate an Ecosystem of ETL Processes for Varying Volumes of Data

• Identify and select the optimum methods of access for each data source (real-time/streaming, batch, delayed, static)
• Determine transformation requirements and develop processes to bring structured and unstructured data from the source to a new physical data model
• Develop processes to efficiently load the transformed  data into the data management system

Prepare Data to Meet Analysis Requirements

• Work with the data scientist to implement strategies for cleaning and preparing data for analysis (e.g., outliers, missing data, etc.)
• Develop and code data extracts
• Follow standard methodologies to ensure data quality and data integrity
• Ensure that the data is fit to use for data science applications

Preferred candidate profile

• 4-7 years of experience developing, delivering, and/or supporting data engineering, advanced analytics or business intelligence solutions
• Ability to work with multiple operating systems and generic tools.
• Experienced in developing ETL/ELT processes using Apache Ni-Fi, Cloud solutions like GCP, Big Query and Snowflake or any equivalent etc.
• Significant experience with big data processing and/or developing applications and data sources using different cloud services etc.
• Experienced in integration with different Ingestion, Scheduling, logging, Alerting and Monitoring cloud services.
• Understanding of how distributed systems work
• Familiarity with software architecture (data structures, data schemas, etc.)
• Strong working knowledge of databases (Cloud DBs like BQ, Snowflake, AlloyDB or any equivalents etc.) including SQL and NoSQL.
• Strong mathematics background, analytical, problem solving, and organizational skills
• Strong communication skills (written, verbal and presentation)
• Experience working in a global, multi-functional environment
• Minimum of 2 years experience in any of the following: At least one high-level client, object-oriented language (e.g., C#, C++, JAVA, Python, Perl, etc.); at least one or more web programming language (PHP, MySQL, Python, Perl, JavaScript, ASP, etc.); one or more Data Extraction Tools (SSIS, Informatica etc.)
• Software development
• Ability to travel as needed