7 Spark Architecture Jobs

Job Summary 5+ years of experience as a developer with experience in developing distributed systems Demonstrated ability to program in Python (pandas, numpy, pytest, pyspark modules and data manipulation techniques) Working knowledge/familiarity with Flask framework, ORM (Object Relational Mapper) and Object Oriented Programming Knowledge of data visualization software like Airflow Good understanding of API, microservices architecture. Design with Rest or GraphQL. Solid understanding of the development process pipeline, effective code version control & continuous Integration, Deployment. Experience on Kubernetes, Helm deployments. Writing reusable, testable, and efficient code and excellent ...

**Job Description:** **Role Overview:** You are an experienced developer with over 5 years of experience in developing distributed systems. You have a demonstrated ability to program in Python, utilizing modules such as pandas, numpy, pytest, and pyspark for data manipulation techniques. Additionally, you have a working knowledge of the Flask framework, ORM, and Object Oriented Programming. Your expertise extends to data visualization software like Airflow, API, microservices architecture, and design with Rest or GraphQL. You possess a solid understanding of the development process pipeline, effective code version control & continuous Integration, Deployment. Experience with Kubernetes, Helm...

We are hiring for one of our client company based in bangalore - Position : Senior Apache Spark Expert Experience : 6+ Years Client location: Bangalore Mode of work : Work from Office Mode of Interview : Virtual Qualification: Bachelors degree in computer science, Engineering, or a related field (preferred) Immediate joiners only Job Description Strong knowledge of Spark architecture, able to set up, manage and admin Spark cluster with Kubernetes, Strong on Scala, build efficient ETL and data pipelines. Experience in building end-to-end flows, orchestrate and optimize. Strong experience database RDBMS, SQL, Snowflake.

Hey Jobseekers, we are looking for AWS Data Engineer with glue & Pyspark experience Experience : 4 to 10 Years Location : Hyderabad / Chennai / Noida / Pune / Bangalore kindly reply with below details [HIDDEN TEXT] CTC :- ECTC :- Notice Period :- Preferred Location :- Hands on experience in Dimensional, Data vault and relation modelling Expertise in Data warehousing and ETL Design and implementation Hands on experience with Programming language like Python/Scala Good understanding of Spark architecture along with internals Hand on experience using AWS services like Glue(Pyspark), Lambda, S3, Athena, RDS, IAM, Lake formation Experience on Mulesoft is good to have Hands on experience on implem...

Role Overview: You will be responsible for ETL design, development, and business system designs. Additionally, you will need to have a good understanding of AWS networking concepts, proficiency in IAM concepts, hands-on experience in various AWS services, and the ability to write clean, modular, testable, and maintainable code. Your role will also involve working with programming languages such as Python, Scala, or Java, willingness to learn Python/PySpark if not already experienced, understanding Spark architecture, execution plans, and performance tuning, strong SQL skills, experience with Redshift and/or Athena, writing and maintaining Terraform scripts, familiarity with Git, GitHub workf...

Role Overview: You will be responsible for leveraging your expertise in any of the following Languages such as Scala, Python, or Java with a minimum of 5-10 years of experience. Your role will involve working on SPARK Architecture, demonstrating good problem-solving and analytical skills, and translating business requirements into technical specifications. Additionally, you will collaborate effectively with team members and vendors, possess familiarity with CI/CD pipelines, and have experience supporting existing strategic applications in an agile environment. Key Responsibilities: - Possess expertise in at least one of the following Languages: Scala/Python/Java - Demonstrate proficiency in ...

Role: Data Engineer Exp: 5 to 8 Years Location: Bangalore, Noida, and Hyderabad (Hybrid, weekly 2 Days office must) NP: Immediate to 15 Days (Try to find only immediate joiners) Note: Candidate must have experience in Python, Kafka Streams, Pyspark, and Azure Databricks. Not looking for candidates who have only Exp in Pyspark and not in Python. Job Title: SSE Kafka, Python, and Azure Databricks (Healthcare Data Project) Experience: 5 to 8 years Role Overview: We are looking for a highly skilled with expertise in Kafka, Python, and Azure Databricks (preferred) to drive our healthcare data engineering projects. The ideal candidate will have deep experience in real-time data streaming, cloud-ba...