Apache Spark Professional

Roles and Responsibilities:

Skillset :Apache Spark Designer

Education

• 8 or more years of professional IT experience, including the majority of the following (specific experiences can be concurrent)
• BFSI domain experience is preferred
• AApache Spark Designer to create and manage large scale data processing solutions
• In this role, Candidate should have a background in data and business analysis
• Should have hands on Apache Spark where you have worked on design, development and deployment on cluster of servers
• Also have data related processing problem-solving aptitude

Responsibilities

• Translate business needs to technical specifications
• Architecting, Design, build and deploy data processing solutions
• Hands on tools to process, stage, store data Conduct recon of data
• Collaborate with teams to integrate systems Develop and execute Spark Queries, database queries and conduct analyses
• Create dataset, RDD, Data Frame for requested projects Develop and update technical documentation
• Sound knowledge about effort estimation methods for data processing solutions requirements
• Proven experience as Apache Spark Designer.
• Hands-on coding experience for Apache Spark designing
• Background in data processing design (e.g. large scale data processing with API based status) and data curation
• In-depth understanding of database management systems, Spark clustering, Spark executor, Driver, real time data processing platform, Java 1.8, and ETL (Extract, transform, load) framework
• Knowledge of SQL queries, Spark Engine and Spark Services, Experience of Spark query tuning and performance optimization

Soft skills

• Proven abilities to take initiative and be innovative
• Analytical mind with a problem-solving aptitude