15802 Pyspark Jobs - Page 6

Experience on Python, Python ML packages, TensorFlow, PyTorch. -Experience on Python, Flask, TensorFlow, PyTorch, SQL, NOSQL, Vector -Understanding of data structures, data modelling and data pipeline design in PySpark, Apache Airflow preferable A day in the life of an Infoscion As part of the Infosys delivery team, your primary role would be to ensure effective Design, Development, Validation and Support activities, to assure that our clients are satisfied with the high levels of service in the technology domain. You will gather the requirements and specifications to understand the client requirements in a detailed manner and translate the same into system requirements. You will play a key ...

• Minimum 5 years + working exp as Databricks Developer • Minimum 3 years + working exp on Redshift, Python, PySpark, and AWS • Associate should hold Databrick Certification and willing to join within 30 Days

Job role & responsibilities:- Understanding operational needs by collaborating with specialized teams Supporting key business operations. This involves architecture designing, building and deploying data systems, pipelines etc Build and support scalable and cost efficiency solution using multiple data services. Support on analysing data pipelines utilizing Azure data stack capabilities Close coordination with Command Centre which performs BAU monitoring and L1 Support Deep dive of failed pipelines to formulate root cause analysis, L2/L3 support Coordinate with Development team on analysis of incidents Technical skills, Qualification and experience required:- 5-7 years of experience in Azure ...

Roles and Responsibilities Design, develop, test, deploy, and maintain big data solutions using Spark (Scala) on Hadoop ecosystem. Collaborate with cross-functional teams to identify business requirements and design scalable data pipelines. Develop high-quality code that meets performance, security, and reliability standards. Troubleshoot complex issues related to big data processing and provide timely resolutions. Stay up-to-date with industry trends and best practices in big data technologies. Desired Candidate Profile 3+ years of experience in working with Spark (Scala) development. Strong understanding of Python programming language and its applications in big data processing. Experience...

Job Description Manager, Data Engineer The Opportunity Based in Hyderabad, join a global healthcare biopharma company and be part of a 130-year legacy of success backed by ethical integrity, forward momentum, and an inspiring mission to achieve new milestones in global healthcare. Be part of an organisation driven by digital technology and data-backed approaches that support a diversified portfolio of prescription medicines, vaccines, and animal health products. Drive innovation and execution excellence. Join a team that is passionate about using data, analytics, and insights to drive decision-making and create custom software, allowing us to tackle some of the world's greatest health threat...

Role: Lead Data Engineer Location: kochi/Trivandrum/ Remote Experience: 7+ years Skill Set: S3, GLue, Lambda, Step function, snowflake, pyspark Budget: Max 25lpa The work location will be Kochi or Trivandrum . Candidates are required to attend the final round of the interview in person . In exceptional cases, a remote interview can be considered; however, the selected candidate must work from the office during the first month. Job Overview We are seeking an experienced Senior Data Engineer to lead the development of a scalable data ingestion framework while ensuring high data quality and validation. The successful candidate will also be responsible for designing and implementing robust APIs ...

Python Lead Experience : 7-10 Years Location : Mumbai/Pune/Bangalore Must-Have: Python (Pandas, PySpark) Data engineering & workflow optimization Delta Tables, Parquet Good-to-Have: Databricks Apache Spark, DBT, Airflow Advanced Pandas optimizations PyTest/DBT testing frameworks Role & responsibilities Preferred candidate profile

Overview We are seeking an outstanding Software Engineer (Python + Database + Cloud) to join our Sustainability Application Development team. As part of a global team the candidate will need to develop productive working relationships. Responsibilities Deliver new functionalities for Operations platform and for upcoming OMDP applications for the ESG & Climate Business. Monitor and optimize application performance. Closely collaborate with Product Management, Research and Product Development, Quality Assurance, Data Operation and IT Infrastructure on all stages of software development life cycle. Having very good Hands-On working experience in Python and Fast API/Flask/Django or similar Rest ...

Educational Requirements Bachelor of Engineering Service Line Data & Analytics Unit Responsibilities Technical knowledge- has expertise in cloud technologies, specifically MS Azure, and services with hands on coding to Python Programming - Expert and Experienced - 4 -5 years DevOps Working knowledge with implementation experience - 1 or 2 projects a minimum Hands-On MS Azure Cloud knowledge Understand and take requirements on Operationalization of ML Models from Data Scientist Help team with ML Pipelines from creation to execution List Azure services required for deployment, Azure Data bricks and Azure DevOps Setup Assist team to coding standards (flake8 etc) Guide team to debug on issues wi...

Job Description Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client qu...

Hiring Data Engineer with strong expertise in Python & PySpark. Hands-on experience in data pipelines, ETL & big data processing. Exposure to any cloud platform (AWS/Azure/GCP) required. Good problem-solving & communication skills

Job Description Role Purpose The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters Do 1. Instrumental in understanding the requirements and design of the product/ software Develop software solutions by studying information needs, studying systems flow, data usage and work processes Investigating problem areas followed by the software development life cycle Facilitate root cause analysis of the system issues and problem statement Identify ideas to improve system performance and impact availability Analyze client requirements and convert re...

Hi, This is Riddhi from Silverlink Technologies. We have an excellent Job opportunity with TCS for the post of Data Engineer(AWS, Databricks) " at Bengaluru, Pune, Hyderabad and Kolkata Location. If interested, kindly forward me your word formatted updated resume ASAP on Riddhi@silverlinktechnologies.com, kindly fill in the below-mentioned details too. Full Name: Contact No: Email ID: DOB: Experience: Relevant Exp: Current Company: Notice Period: Current CTC: Expected CTC: Offer in hand: If yes then offered ctc: Date of joining: Company name: Grades -- 10th: 12th: Graduation: Full time/Part Time? University Name: Current Location: Preferred Location: Gap in education: Gap in employment: **Ma...

Job Description Role Purpose The purpose of the role is to support process delivery by ensuring daily performance of the Production Specialists, resolve technical escalations and develop technical capability within the Production Specialists. Do Oversee and support process by reviewing daily transactions on performance parameters Review performance dashboard and the scores for the team Support the team in improving performance parameters by providing technical support and process guidance Record, track, and document all queries received, problem-solving steps taken and total successful and unsuccessful resolutions Ensure standard processes and procedures are followed to resolve all client qu...

REQUIRED SKILLS: Bachelor's degree in Computer Science, Engineering, or a related field. Advanced degree preferred. Proven 6-8 years experience in designing, implementing solutions on the Azure ecosystem including expertise in Azure services such as Azure Data Factory, Azure Databricks, Azure Machine Learning, Azure Data Lake Storage, and Azure SQL Database, Azure Synapse, PowerBI etc... Experience with big data technologies such as Apache Spark, Hadoop, or similar frameworks. Solid understanding of cloud computing concepts and experience with cloud infrastructure management and provisioning. Solid understanding of network security concepts and technologies (such as firewalls, VPNs, intrusio...

Looking to onboard a skilled professional with 6-8 years of experience in Pyspark, SQL, and AWS Glue. The ideal candidate will have a strong background in these technologies and excellent problem-solving skills. This position is located across Pan India. Roles and Responsibility Design, develop, and implement data processing pipelines using Pyspark and AWS Glue. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data warehouses using Snowflake and Snowpark. Optimize data processing workflows for performance and scalability. Troubleshoot and resolve technical issues related to data processing and storage. Ensure data quali...

Exp: 5 - 14 Yrs Work Mode: Hybrid Location: Bangalore, Chennai, Kolkata, Pune and Gurgaon Primary Skills: Python, pyspark, Azure Data Factory, snowflake, snowpipe, snowsql, Snowsight, Snowpark, ETL, SQL, and Architect Designing. Snowpro certified is plus Primary Roles and Responsibilities: Developing Modern Data Warehouse solutions using Snowflake, Databricks and ADF. Ability to provide solutions that are forward-thinking in data engineering and analytics space Collaborate with DW/BI leads to understand new ETL pipeline development requirements. Triage issues to find gaps in existing pipelines and fix the issues Work with business to understand the need in the reporting layer and develop a d...

Exp: 5 - 14 Yrs Work Mode: Hybrid Location: Bangalore, Chennai, Kolkata, Pune and Gurgaon Primary Skills: Python, pyspark, Azure Data Factory, snowflake, snowpipe, snowsql, Snowsight, Snowpark, ETL, SQL, and Architect Designing. Snowpro certified is plus Primary Roles and Responsibilities: Developing Modern Data Warehouse solutions using Snowflake, Databricks and ADF. Ability to provide solutions that are forward-thinking in data engineering and analytics space Collaborate with DW/BI leads to understand new ETL pipeline development requirements. Triage issues to find gaps in existing pipelines and fix the issues Work with business to understand the need in the reporting layer and develop a d...

Exp: 5 - 14 Yrs Work Mode: Hybrid Location: Bangalore, Chennai, Kolkata, Pune and Gurgaon Primary Skills: Python, pyspark, Azure Data Factory, snowflake, snowpipe, snowsql, Snowsight, Snowpark, ETL, SQL, and Architect Designing. Snowpro certified is plus Primary Roles and Responsibilities: Developing Modern Data Warehouse solutions using Snowflake, Databricks and ADF. Ability to provide solutions that are forward-thinking in data engineering and analytics space Collaborate with DW/BI leads to understand new ETL pipeline development requirements. Triage issues to find gaps in existing pipelines and fix the issues Work with business to understand the need in the reporting layer and develop a d...

Role: Data Architect Experience: 8-15 years Location: Bangalore, Chennai, Gurgaon, Pune, and Kolkata Work Mode: Hybrid Mandatory Skills: Python, Pyspark, SQL, ETL, Pipelines, Azure Databricks, Azure Data Factory, & Architect Designing. Primary Roles and Responsibilities: Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack Ability to provide solutions that are forward-thinking in data engineering and analytics space Collaborate with DW/BI leads to understand new ETL pipeline development requirements. Triage issues to find gaps in existing pipelines and fix the issues Work with business to understand the need in reporting layer and develop data model to fulfill rep...

Role: Data Architect Experience: 8-15 years Location: Bangalore, Chennai, Gurgaon, Pune, and Kolkata Work Mode: Hybrid Mandatory Skills: Python, Pyspark, SQL, ETL, Pipelines, Azure Databricks, Azure Data Factory, & Architect Designing. Primary Roles and Responsibilities: Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack Ability to provide solutions that are forward-thinking in data engineering and analytics space Collaborate with DW/BI leads to understand new ETL pipeline development requirements. Triage issues to find gaps in existing pipelines and fix the issues Work with business to understand the need in reporting layer and develop data model to fulfill rep...

Role: Data Architect Experience: 8-15 years Location: Bangalore, Chennai, Gurgaon, Pune, and Kolkata Work Mode: Hybrid Mandatory Skills: Python, Pyspark, SQL, ETL, Pipelines, Azure Databricks, Azure Data Factory, & Architect Designing. Primary Roles and Responsibilities: Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack Ability to provide solutions that are forward-thinking in data engineering and analytics space Collaborate with DW/BI leads to understand new ETL pipeline development requirements. Triage issues to find gaps in existing pipelines and fix the issues Work with business to understand the need in reporting layer and develop data model to fulfill rep...

At Citi we’re not just building technology, we’re building the future of banking. Encompassing a broad range of specialties, roles, and cultures, our teams are creating innovations used across the globe. Citi is constantly growing and progressing through our technology, with laser focused on evolving the ways of doing things. As one of the world’s most global banks we’re changing how the world does business. Shape your Career with Citi We’re currently looking for a high caliber professional to join our team as 25900903 Officer, Big Data Engineer -C10- Hybrid based in Chennai, India. Being part of our team means that we’ll provide you with the resources to meet your unique needs, empower you ...

Hi, Wishes from GSN!!! Pleasure connecting with you!!! We been into Corporate Search Services for Identifying & Bringing in Stellar Talented Professionals for our reputed IT / Non-IT clients in India. We have been successfully providing results to various potential needs of our clients for the last 20 years. At present, GSN is hiring DATA ENGINEER for one of our leading MNC client. PFB the details for your better understanding : 1. WORK LOCATION : CHENNAI 2. Job Role: DATA ENGINEER 3. EXPERIENCE : 4.5 yrs (relevant) 4. CTC Range: 12LPA to 20 LPA 5. Work Type : WFO - Hybrid Who are we looking for : A Data Engineer focuses on enhancing and supporting existing Data & Analytics solutions by leve...

Chennai / HYD / Kolkata / Delhi / Pune / BLR Functional Skills: Experience in Credit Risk/Regulatory risk domain Technical Skills: Spark ,PySpark, Python, Hive, Scala, MapReduce, Unix shell scripting Machine learning tech.