Home
Jobs

3895 Pyspark Jobs - Page 20

Filter Interviews
Min: 0 years
Max: 25 years
Min: ₹0
Max: ₹10000000
Setup a job Alert
Filter
JobPe aggregates results for easy application access, but you actually apply on the job portal directly.

0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

Avant de postuler à un emploi, sélectionnez votre langue de préférence parmi les options disponibles en haut à droite de cette page. Découvrez votre prochaine opportunité au sein d'une organisation qui compte parmi les 500 plus importantes entreprises mondiales. Envisagez des opportunités innovantes, découvrez notre culture enrichissante et travaillez avec des équipes talentueuses qui vous poussent à vous développer chaque jour. Nous savons ce qu’il faut faire pour diriger UPS vers l'avenir : des personnes passionnées dotées d’une combinaison unique de compétences. Si vous avez les qualités, de la motivation, de l'autonomie ou le leadership pour diriger des équipes, il existe des postes adaptés à vos aspirations et à vos compétences d'aujourd'hui et de demain. Job Summary Fiche de poste : UPS Marketing team is looking for a talented and driven Data Scientist to drive its strategic objectives in the areas of pricing, revenue management, market analysis and evidence/data-based decision making. This role will work across multiple channels and teams to drive tangible results in the organization. You will focus on developing metrics for multiple channels and markets, applying advanced statistical modeling where appropriate and pioneering new analytical methods in a variety of fast paced and rapidly evolving consumer channels. This high visibility position will work with multiple levels of the organization, including senior leadership to bring analytical capabilities to the forefront of pricing, rate setting, and optimization of our go-to-market offers. You will contribute to rapidly evolving UPS Marketing analytical capabilities by working amongst a collaborative team of Data Scientists, Analysts and multiple business stakeholders. Responsibilities Become a subject matter expert on UPS business processes, data and analytical capabilities to help define and solve business needs using data and advanced statistical methods Analyze and extract insights from large-scale structured and unstructured data utilizing multiple platforms and tools. Understand and apply appropriate methods for cleaning and transforming data Work across multiple stake holders to develop, maintain and improve models in production Take the initiative to create and execute analyses in a proactive manner Deliver complex analytical and visualizations to broader audiences including upper management and executives Deliver analytics and insights to support strategic decision making Understand the application of AI/ML when appropriate to solve complex business problems Qualifications Expertise in R, SQL, Python. Strong analytical skills and attention to detail. Able to engage key business and executive-level stakeholders to translate business problems to high level analytics solution approach. Expertise with statistical techniques, machine learning or operations research and their application in business applications. Deep understanding of data management pipelines and experience in launching moderate scale advanced analytics projects in production at scale. Proficient in Azure, Google Cloud environment Experience implementing open-source technologies and cloud services; with or without the use of enterprise data science platforms. Solid oral and written communication skills, especially around analytical concepts and methods. Ability to communicate data through a story framework to convey data-driven results to technical and non-technical audience. Master’s Degree in a quantitative field of mathematics, computer science, physics, economics, engineering, statistics (operations research, quantitative social science, etc.), international equivalent, or equivalent job experience. Bonus Qualifications Experience with pricing methodologies and revenue management Experience using PySpark, Azure Databricks, Google BigQuery and Vertex AI Creating and implementing NLP/LLM projects Experience utilizing and applying neurals networks and other AI methodologies Familiarity with Data architecture and engineering Type De Contrat en CDI Chez UPS, égalité des chances, traitement équitable et environnement de travail inclusif sont des valeurs clefs auxquelles nous sommes attachés. Show more Show less

Posted 5 days ago

Apply

0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

Before you apply to a job, select your language preference from the options available at the top right of this page. Explore your next opportunity at a Fortune Global 500 organization. Envision innovative possibilities, experience our rewarding culture, and work with talented teams that help you become better every day. We know what it takes to lead UPS into tomorrow—people with a unique combination of skill + passion. If you have the qualities and drive to lead yourself or teams, there are roles ready to cultivate your skills and take you to the next level. Job Description Job Summary UPS Marketing team is looking for a talented and driven Data Scientist to drive its strategic objectives in the areas of pricing, revenue management, market analysis and evidence/data-based decision making. This role will work across multiple channels and teams to drive tangible results in the organization. You will focus on developing metrics for multiple channels and markets, applying advanced statistical modeling where appropriate and pioneering new analytical methods in a variety of fast paced and rapidly evolving consumer channels. This high visibility position will work with multiple levels of the organization, including senior leadership to bring analytical capabilities to the forefront of pricing, rate setting, and optimization of our go-to-market offers. You will contribute to rapidly evolving UPS Marketing analytical capabilities by working amongst a collaborative team of Data Scientists, Analysts and multiple business stakeholders. Responsibilities Become a subject matter expert on UPS business processes, data and analytical capabilities to help define and solve business needs using data and advanced statistical methods Analyze and extract insights from large-scale structured and unstructured data utilizing multiple platforms and tools. Understand and apply appropriate methods for cleaning and transforming data Work across multiple stake holders to develop, maintain and improve models in production Take the initiative to create and execute analyses in a proactive manner Deliver complex analytical and visualizations to broader audiences including upper management and executives Deliver analytics and insights to support strategic decision making Understand the application of AI/ML when appropriate to solve complex business problems Qualifications Expertise in R, SQL, Python. Strong analytical skills and attention to detail. Able to engage key business and executive-level stakeholders to translate business problems to high level analytics solution approach. Expertise with statistical techniques, machine learning or operations research and their application in business applications. Deep understanding of data management pipelines and experience in launching moderate scale advanced analytics projects in production at scale. Proficient in Azure, Google Cloud environment Experience implementing open-source technologies and cloud services; with or without the use of enterprise data science platforms. Solid oral and written communication skills, especially around analytical concepts and methods. Ability to communicate data through a story framework to convey data-driven results to technical and non-technical audience. Master’s Degree in a quantitative field of mathematics, computer science, physics, economics, engineering, statistics (operations research, quantitative social science, etc.), international equivalent, or equivalent job experience. Bonus Qualifications Experience with pricing methodologies and revenue management Experience using PySpark, Azure Databricks, Google BigQuery and Vertex AI Creating and implementing NLP/LLM projects Experience utilizing and applying neurals networks and other AI methodologies Familiarity with Data architecture and engineering Employee Type Permanent UPS is committed to providing a workplace free of discrimination, harassment, and retaliation. Show more Show less

Posted 5 days ago

Apply

0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

Join us as a Technology Business Analyst - PySpark Take on a critical role supporting the bank to achieve customer advocacy by eliciting and understanding the internal and external technology customer needs You'll be working in a vibrant, forward looking team and helping us to shape the future of our business It’s a great chance to hit the ground running, take advantage of our development opportunities, and advance your career We're offering this role at vice president level What you'll do In your new role, you’ll be producing business requirements to inform future state design, supporting teams in understanding the requirements and their linkages to the overall domain strategy. In the course of your work, we’ll also look to you to build, manage and maintain a positive working relationship with stakeholders and third party suppliers across multiple franchises and functions. You’ll Also Be Supporting others to articulate the ideas and requirements in the format required by the team, such as use cases, user journeys, user stories and illustrations Capturing, validating and documenting business and system requirements and making sure that they’re in line with key strategic principles Interacting with customers, key stakeholders and the team to obtain and document functional and non-functional needs Analysing existing system documentation to summarise existing system functionality as it relates to the work at hand Supporting the identification of team priorities based upon feedback from the customers, key stakeholders and the team The skills you'll need We’ll look to you to bring a good understanding of Agile values, principles and methodologies with experience of working within an Agile environment. You'll need at least twelve years of total experience with at least six years of experience in PySpark, AWS and SQL. Additionally, you’ll need to hold an appropriate business analyst certification. You’ll Also Need Good communication skills, with the ability to communicate complex technical concepts clearly to your peers and management level colleague Knowledge on credit risk models methodology and implementation Experience with data recovery and writing SQL queries Good collaboration and stakeholder management skills Show more Show less

Posted 5 days ago

Apply

0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

Join us as a Software Engineer - PySpark This is an opportunity for a driven Software Engineer to take on an exciting new career challenge Day-to-day, you'll be engineering and maintaining innovative, customer centric, high performance, secure and robust solutions It’s a chance to hone your existing technical skills and advance your career while building a wide network of stakeholders What you'll do In your new role, you’ll be working within a feature team to engineer software, scripts and tools, as well as liaising with other engineers, architects and business analysts across the platform. You’ll Also Be Producing complex and critical software rapidly and of high quality which adds value to the business Working in permanent teams who are responsible for the full life cycle, from initial development, through enhancement and maintenance to replacement or decommissioning Collaborating to optimise our software engineering capability Designing, producing, testing and implementing our working software solutions Working across the life cycle, from requirements analysis and design, through coding to testing, deployment and operations The skills you'll need To take on this role, you’ll need a background in software engineering, software design, and architecture, and an understanding of how your area of expertise supports our customers. You'll need four to seven years of experience in Pyspark, Python, AWS, SQL and Tableau. You'll also need experience in developing and supporting ETL pipelines and tableau reporting. You’ll Also Need Experience of working with development and testing tools, bug tracking tools and wikis Experience in multiple programming languages or low code toolsets Experience of DevOps and Agile methodology and associated toolsets A background in solving highly complex, analytical and numerical problems Experience of implementing programming best practice, especially around scalability, automation, virtualisation, optimisation, availability and performance Show more Show less

Posted 5 days ago

Apply

0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

Join us as a Data Engineering Lead This is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiences You’ll be simplifying the bank through developing innovative data driven solutions, inspiring to be commercially successful through insight, and keeping our customers’ and the bank’s data safe and secure Participating actively in the data engineering community, you’ll deliver opportunities to support our strategic direction while building your network across the bank We’re recruiting for multiple roles across a range to levels, up to and including experienced managers What you'll do We’ll look to you to demonstrate technical and people leadership to drive value for the customer through modelling, sourcing and data transformation. You’ll be working closely with core technology and architecture teams to deliver strategic data solutions, while driving Agile and DevOps adoption in the delivery of data engineering, leading a team of data engineers. We’ll Also Expect You To Be Working with Data Scientists and Analytics Labs to translate analytical model code to well tested production ready code Helping to define common coding standards and model monitoring performance best practices Owning and delivering the automation of data engineering pipelines through the removal of manual stages Developing comprehensive knowledge of the bank’s data structures and metrics, advocating change where needed for product development Educating and embedding new data techniques into the business through role modelling, training and experiment design oversight Leading and delivering data engineering strategies to build a scalable data architecture and customer feature rich dataset for data scientists Leading and developing solutions for streaming data ingestion and transformations in line with streaming strategy The skills you'll need To be successful in this role, you’ll need to be an expert level programmer and data engineer with a qualification in Computer Science or Software Engineering. You’ll also need a strong understanding of data usage and dependencies with wider teams and the end customer, as well as extensive experience in extracting value and features from large scale data. We'll also expect you to have knowledge of of big data platforms like Snowflake, AWS Redshift, Postgres, MongoDB, Neo4J and Hadoop, along with good knowledge of cloud technologies such as Amazon Web Services, Google Cloud Platform and Microsoft Azure You’ll Also Demonstrate Knowledge of core computer science concepts such as common data structures and algorithms, profiling or optimisation An understanding of machine-learning, information retrieval or recommendation systems Good working knowledge of CICD tools Knowledge of programming languages in data engineering such as Python or PySpark, SQL, Java, and Scala An understanding of Apache Spark and ETL tools like Informatica PowerCenter, Informatica BDM or DEI, Stream Sets and Apache Airflow Knowledge of messaging, event or streaming technology such as Apache Kafka Experience of ETL technical design, automated data quality testing, QA and documentation, data warehousing, data modelling and data wrangling Extensive experience using RDMS, ETL pipelines, Python, Hadoop and SQL Show more Show less

Posted 5 days ago

Apply

0 years

0 Lacs

Gurugram, Haryana, India

On-site

Linkedin logo

Join us as a Machine Learning Engineer In this role, you’ll be driving and embedding the deployment, automation, maintenance and monitoring of machine learning models and algorithms Day-to-day, you’ll make sure that models and algorithms work effectively in a production environment while promoting data literacy education with business stakeholders If you see opportunities where others see challenges, you’ll find that this solutions-driven role will be your chance to solve new problems and enjoy excellent career development What you’ll do Your daily responsibilities will include you collaborating with colleagues to design and develop advanced machine learning products which power our group for our customers. You’ll also codify and automate complex machine learning model productions, including pipeline optimisation. We’ll expect you to transform advanced data science prototypes and apply machine learning algorithms and tools. You’ll also plan, manage, and deliver larger or complex projects, involving a variety of colleagues and teams across our business. You’ll Also Be Responsible For Understanding the complex requirements and needs of business stakeholders, developing good relationships and how machine learning solutions can support our business strategy Working with colleagues to productionise machine learning models, including pipeline design and development and testing and deployment, so the original intent is carried over to production Creating frameworks to ensure robust monitoring of machine learning models within a production environment, making sure they deliver quality and performance Understanding and addressing any shortfalls, for instance, through retraining Leading direct reports and wider teams in an Agile way within multi-disciplinary data and analytics teams to achieve agreed project and Scrum outcomes The skills you’ll need To be successful in this role, you’ll need to have a good academic background in a STEM discipline, such as Mathematics, Physics, Engineering or Computer Science. You’ll also have the ability to use data to solve business problems, from hypotheses through to resolution. We’ll look to you to have experience of at least twelve years with machine learning on large datasets, as well as experience building, testing, supporting, and deploying advanced machine learning models into a production environment using modern CI/CD tools, including git, TeamCity and CodeDeploy. You’ll Also Need A good understanding of machine learning approaches and algorithms such as supervised or unsupervised learning, deep learning, NLP with a strong focus on model development, deployment, and optimization Experience using Python with libraries such as NumPy, Pandas, Scikit-learn, and TensorFlow or PyTorch An understanding of PySpark for distributed data processing and manipulation with AWS (Amazon Web Services) including EC2, S3, Lambda, SageMaker, and other cloud tools. Experience with data processing frameworks such as Apache Kafka, Apache Airflow and containerization technologies such as Docker and orchestration tools such as Kubernetes Experience of building GenAI solutions to automate workflows to improve productivity and efficiency Show more Show less

Posted 5 days ago

Apply

0 years

0 Lacs

Gurugram, Haryana, India

On-site

Linkedin logo

Join us as a Data Engineer We’re looking for someone to build effortless, digital first customer experiences to help simplify our organisation and keep our data safe and secure Day-to-day, you’ll develop innovative, data-driven solutions through data pipelines, modelling and ETL design while inspiring to be commercially successful through insights If you’re ready for a new challenge, and want to bring a competitive edge to your career profile by delivering streaming data ingestions, this could be the role for you We're offering this role at associate vice president level What you’ll do Your daily responsibilities will include you developing a comprehensive knowledge of our data structures and metrics, advocating for change when needed for product development. You’ll also provide transformation solutions and carry out complex data extractions. We’ll expect you to develop a clear understanding of data platform cost levels to build cost-effective and strategic solutions. You’ll also source new data by using the most appropriate tooling before integrating it into the overall solution to deliver it to our customers. You’ll Also Be Responsible For Driving customer value by understanding complex business problems and requirements to correctly apply the most appropriate and reusable tools to build data solutions Participating in the data engineering community to deliver opportunities to support our strategic direction Carrying out complex data engineering tasks to build a scalable data architecture and the transformation of data to make it usable to analysts and data scientists Building advanced automation of data engineering pipelines through the removal of manual stages Leading on the planning and design of complex products and providing guidance to colleagues and the wider team when required The skills you’ll need To be successful in this role, you’ll have an understanding of data usage and dependencies with wider teams and the end customer. You’ll also have experience in SQL and NoSQL databases to support diverse data requirements We’ll expect you to have a minimum of eight years of experience in ETL technical design, data quality testing, cleansing and monitoring, data sourcing, exploration and analysis, and data warehousing and data modelling capabilities. You’ll Also Need Experience in developing and maintaining high-quality, reusable code in Pyspark and Spark SQL Experience in development using technologies such as Spark and Kafka Great communication skills with the ability to collaborate with software engineering teams to integrate data solutions into existing applications Show more Show less

Posted 5 days ago

Apply

0 years

0 Lacs

Gurugram, Haryana, India

On-site

Linkedin logo

Join us as a Technology Business Analyst - PySpark Take on a critical role supporting the bank to achieve customer advocacy by eliciting and understanding the internal and external technology customer needs You'll be working in a vibrant, forward looking team and helping us to shape the future of our business It’s a great chance to hit the ground running, take advantage of our development opportunities, and advance your career We're offering this role at vice president level What you'll do In your new role, you’ll be producing business requirements to inform future state design, supporting teams in understanding the requirements and their linkages to the overall domain strategy. In the course of your work, we’ll also look to you to build, manage and maintain a positive working relationship with stakeholders and third party suppliers across multiple franchises and functions. You’ll Also Be Supporting others to articulate the ideas and requirements in the format required by the team, such as use cases, user journeys, user stories and illustrations Capturing, validating and documenting business and system requirements and making sure that they’re in line with key strategic principles Interacting with customers, key stakeholders and the team to obtain and document functional and non-functional needs Analysing existing system documentation to summarise existing system functionality as it relates to the work at hand Supporting the identification of team priorities based upon feedback from the customers, key stakeholders and the team The skills you'll need We’ll look to you to bring a good understanding of Agile values, principles and methodologies with experience of working within an Agile environment. You'll need at least twelve years of total experience with at least six years of experience in PySpark, AWS and SQL. Additionally, you’ll need to hold an appropriate business analyst certification. You’ll Also Need Good communication skills, with the ability to communicate complex technical concepts clearly to your peers and management level colleague Knowledge on credit risk models methodology and implementation Experience with data recovery and writing SQL queries Good collaboration and stakeholder management skills Show more Show less

Posted 5 days ago

Apply

0 years

0 Lacs

Gurugram, Haryana, India

On-site

Linkedin logo

Join us as a Data Engineering Lead This is an exciting opportunity to use your technical expertise to collaborate with colleagues and build effortless, digital first customer experiences You’ll be simplifying the bank through developing innovative data driven solutions, inspiring to be commercially successful through insight, and keeping our customers’ and the bank’s data safe and secure Participating actively in the data engineering community, you’ll deliver opportunities to support our strategic direction while building your network across the bank We’re recruiting for multiple roles across a range to levels, up to and including experienced managers What you'll do We’ll look to you to demonstrate technical and people leadership to drive value for the customer through modelling, sourcing and data transformation. You’ll be working closely with core technology and architecture teams to deliver strategic data solutions, while driving Agile and DevOps adoption in the delivery of data engineering, leading a team of data engineers. We’ll Also Expect You To Be Working with Data Scientists and Analytics Labs to translate analytical model code to well tested production ready code Helping to define common coding standards and model monitoring performance best practices Owning and delivering the automation of data engineering pipelines through the removal of manual stages Developing comprehensive knowledge of the bank’s data structures and metrics, advocating change where needed for product development Educating and embedding new data techniques into the business through role modelling, training and experiment design oversight Leading and delivering data engineering strategies to build a scalable data architecture and customer feature rich dataset for data scientists Leading and developing solutions for streaming data ingestion and transformations in line with streaming strategy The skills you'll need To be successful in this role, you’ll need to be an expert level programmer and data engineer with a qualification in Computer Science or Software Engineering. You’ll also need a strong understanding of data usage and dependencies with wider teams and the end customer, as well as extensive experience in extracting value and features from large scale data. We'll also expect you to have knowledge of of big data platforms like Snowflake, AWS Redshift, Postgres, MongoDB, Neo4J and Hadoop, along with good knowledge of cloud technologies such as Amazon Web Services, Google Cloud Platform and Microsoft Azure You’ll Also Demonstrate Knowledge of core computer science concepts such as common data structures and algorithms, profiling or optimisation An understanding of machine-learning, information retrieval or recommendation systems Good working knowledge of CICD tools Knowledge of programming languages in data engineering such as Python or PySpark, SQL, Java, and Scala An understanding of Apache Spark and ETL tools like Informatica PowerCenter, Informatica BDM or DEI, Stream Sets and Apache Airflow Knowledge of messaging, event or streaming technology such as Apache Kafka Experience of ETL technical design, automated data quality testing, QA and documentation, data warehousing, data modelling and data wrangling Extensive experience using RDMS, ETL pipelines, Python, Hadoop and SQL Show more Show less

Posted 5 days ago

Apply

3.0 years

0 Lacs

Hyderabad, Telangana, India

On-site

Linkedin logo

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Databricks Unified Data Analytics Platform, Oracle Procedural Language Extensions to SQL (PLSQL), PySpark Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As an Application Developer, you will engage in the design, construction, and configuration of applications tailored to fulfill specific business processes and application requirements. Your typical day will involve collaborating with team members to understand project needs, developing innovative solutions, and ensuring that applications are optimized for performance and usability. You will also participate in testing and debugging processes to ensure the applications function as intended, while continuously seeking ways to enhance application efficiency and user experience. Roles & Responsibilities: - Expected to perform independently and become an SME. - Required active participation/contribution in team discussions. - Contribute in providing solutions to work related problems. - Collaborate with cross-functional teams to gather requirements and translate them into technical specifications. - Participate in code reviews to ensure adherence to best practices and coding standards. Professional & Technical Skills: -- Backend Engineer who is good on niche backend skills preferably on Databricks, integration and Reporting skillset - Microservices Architecture and Rest patterns using leading industry recommended security frameworks. - Cloud and related technologies such as AWS, Google, Azure. - Test Automation Skills using Behavioral Driven Development. - Data Integration (batch, real-time) following Enterprise Integration Patterns. - Relational Database, No SQL Database, DynamoDB and Data Modeling, - Database development & tuning (PL/SQL/XQuery). - Performance (threading, indexing, clustering, caching). - Document-centric data architecture (XML DB/NoSQL). Additional Skills: Tableau, Angular, Performance Tuning Additional Information: - The candidate should have minimum 5 years of experience in Databricks Unified Data Analytics Platform. - This position is based at our Hyderabad office. - A 15 years full time education is required. Show more Show less

Posted 5 days ago

Apply

3.0 years

0 Lacs

Hyderabad, Telangana, India

On-site

Linkedin logo

Project Role : Application Developer Project Role Description : Design, build and configure applications to meet business process and application requirements. Must have skills : Databricks Unified Data Analytics Platform, Microsoft Azure Databricks, PySpark Good to have skills : NA Minimum 3 Year(s) Of Experience Is Required Educational Qualification : 15 years full time education Summary: As an Application Developer, you will design, build, and configure applications to meet business process and application requirements. A typical day involves collaborating with team members to understand project needs, developing application features, and ensuring that the applications are aligned with business objectives. You will also engage in testing and troubleshooting to enhance application performance and user experience, while continuously seeking opportunities for improvement and innovation in application development. Roles & Responsibilities: - Expected to perform independently and become an SME. - Required active participation/contribution in team discussions. - Contribute in providing solutions to work related problems. - Assist in the documentation of application processes and workflows. - Engage in code reviews to ensure quality and adherence to best practices. Professional & Technical Skills: - Must To Have Skills: Proficiency in Databricks Unified Data Analytics Platform, Microsoft Azure Databricks, PySpark. - Strong understanding of data integration techniques and ETL processes. - Experience with cloud-based application development and deployment. - Familiarity with agile development methodologies and practices. - Ability to troubleshoot and optimize application performance. Additional Information: - The candidate should have minimum 3 years of experience in Databricks Unified Data Analytics Platform. - This position is based at our Hyderabad office. - A 15 years full time education is required. Show more Show less

Posted 5 days ago

Apply

5.0 - 9.0 years

12 - 22 Lacs

Hyderabad, Bengaluru

Hybrid

Naukri logo

Position : PySpark Data Engineer Location : Bangalore / Hyderabad Experience : 5 to 9 Yrs Job Type : On Role Job Description: PySpark Data Engineer:- 1. API Development : Design, develop, and maintain robust APIs using FastAPI and RESTful principles for scalable backend systems. 2. Big Data Processing : Leverage PySpark to process and analyze large datasets efficiently, ensuring optimal performance in big data environments. 3. Full-Stack Integration : Develop seamless backend-to-frontend feature integrations, collaborating with front-end developers for cohesive user experiences. 4. CI/CD Pipelines : Implement and manage CI/CD pipelines using GitHub Actions and Azure DevOps to streamline deployments and ensure system reliability. 5. Containerization : Utilize Docker for building and deploying containerized applications in development and production environments. 6. Team Leadership : Lead and mentor a team of developers, providing guidance, code reviews, and support to junior team members to ensure high-quality deliverables. 7. Code Optimization : Write clean, maintainable, and efficient Python code, with a focus on scalability, reusability, and performance. 8. Cloud Deployment : Deploy and manage applications on cloud platforms like Azure , ensuring high availability and fault tolerance. 9. Collaboration : Work closely with cross-functional teams, including product managers and designers, to translate business requirements into technical solutions. 10. Documentation : Maintain thorough documentation for APIs, processes, and systems to ensure transparency and ease of maintenance. Highlighted Skillset:- Big Data : Strong PySpark skills for processing large datasets. DevOps : Proficiency in GitHub Actions , CI/CD pipelines , Azure DevOps , and Docker . Integration : Experience in backend-to-frontend feature connectivity. Leadership : Proven ability to lead and mentor development teams. Cloud : Knowledge of deploying and managing applications in Azure or other cloud environments. Team Collaboration : Strong interpersonal and communication skills for working in cross-functional teams. Best Practices : Emphasis on clean code, performance optimization, and robust documentation. Interested candidates kindly share your CV and below details to usha.sundar@adecco.com 1) Present CTC (Fixed + VP) - 2) Expected CTC - 3) No. of years experience - 4) Notice Period - 5) Offer-in hand - 6) Reason of Change - 7) Present Location -

Posted 5 days ago

Apply

2.0 years

0 Lacs

Coimbatore, Tamil Nadu, India

Remote

Linkedin logo

Job Description Job Title: Data Support Specialist Location: Remote Candidate Expectation Candidate should have 2+ years of experience in Data support. Job Description Candidate should have 2+ years of experience as a data or quality assurance analyst, ideally working with SQL, PySpark, and/or Python Should have strong attention to detail and are a methodical problem-solver Should have excellent oral and written communication skills, with the ability to interact effectively with internal teams across time zones and cultures Should strive to make tasks as efficient as possible Should be enthusiastic about making a big impact at a rapidly growing company Should have experience working with web-scraped data, transaction data, or email data, though this is not required. Skills Required RoleData Support Specialist - Remote Industry TypeIT/ Computers - Software Functional Area Required Education B E Employment TypeFull Time, Permanent Key Skills DATA SUPPORT PY SPARK PYT HO N Other Information Job CodeGO/JC/166/2025 Recruiter NameDevikala D Show more Show less

Posted 5 days ago

Apply

0.0 - 2.0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

The Data Analytics Analyst 2 is a developing professional role. Applies specialty area knowledge in monitoring, assessing, analyzing and/or evaluating processes and data. Identifies policy gaps and formulates policies. Interprets data and makes recommendations. Researches and interprets factual information. Identifies inconsistencies in data or results, defines business issues and formulates recommendations on policies, procedures or practices. Integrates established disciplinary knowledge within own specialty area with basic understanding of related industry practices. Good understanding of how the team interacts with others in accomplishing the objectives of the area. Develops working knowledge of industry practices and standards. Limited but direct impact on the business through the quality of the tasks/services provided. Impact of the job holder is restricted to own team. Responsibilities: Identifies policy gaps and formulates policies. Interprets data and make recommendations. Integrates established disciplinary knowledge within own specialty area with basic understanding of related industry practices. Makes judgments and recommendations based on analysis and specialty area knowledge. Researches and interprets factual information. Identifies inconsistencies in data or results, define business issues and formulate recommendations on policies, procedures or practices. Exchanges information in a concise and logical way as well as be sensitive to audience diversity. Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency. Qualifications: 0-2 years experience using tools for statistical modeling of large data sets Education: Bachelor’s/University degree or equivalent experience This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required. 2-3 years of experience as a python developer with expertise in automation testing to design, develop, and automate robust software solutions and testing frameworks like Pytest, Behave etc. 2-4 years of experience as Big Data Engineer to develop, optimize, and manage large-scale data processing systems and analytics platforms. 3-4 years of experience in distributed data processing & near real-time data analytics using Springboot/PySpark. 2-5 years of experience in designing, managing, and optimizing Kafka-based messaging systems for real-time data streaming. 3-5 years of experience in tools such as Selenium, Cucumber BDD, Karate, Rest Assured and programming languages such as Java and python for Web application automation and API automation. Familiarity with CI/CD pipelines, version control systems (e.g., Git), and DevOps practic ------------------------------------------------------ Job Family Group: Technology ------------------------------------------------------ Job Family: Data Analytics ------------------------------------------------------ Time Type: Full time ------------------------------------------------------ Most Relevant Skills Please see the requirements listed above. ------------------------------------------------------ Other Relevant Skills For complementary skills, please see above and/or contact the recruiter. ------------------------------------------------------ Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi. View Citi’s EEO Policy Statement and the Know Your Rights poster. Show more Show less

Posted 5 days ago

Apply

2.0 - 6.0 years

5 - 15 Lacs

Chennai

Hybrid

Naukri logo

Job Summary: We are seeking a highly skilled and passionate Data Scientist with 3-6 years of experience to join our dynamic team in Chennai. The ideal candidate will possess a strong background in machine learning and deep learning methodologies, coupled with expert-level proficiency in PySpark and SQL for large-scale data manipulation and analysis. You will be instrumental in transforming complex data into actionable insights, building predictive models, and deploying robust data-driven solutions that directly impact our business objectives. Key Responsibilities: Data Analysis & Feature Engineering: Perform extensive exploratory data analysis (EDA) to identify trends, patterns, and anomalies in large, complex datasets. Develop and implement robust data preprocessing, cleaning, and feature engineering pipelines using PySpark and SQL to prepare data for model training. Work with structured and unstructured data, ensuring data quality and integrity. Model Development & Implementation (Machine Learning & Deep Learning): Design, develop, and implement advanced Machine Learning (ML) and Deep Learning (DL) models to solve complex business problems, such as prediction, classification, recommendation, and anomaly detection. Apply a wide range of ML algorithms (e.g., Regression, Classification, Clustering, Ensemble methods) and DL architectures (e.g., CNNs, RNNs, Transformers) as appropriate for the problem at hand. Optimize and fine-tune models for performance, accuracy, and scalability. Experience with ML/DL frameworks such as TensorFlow, PyTorch, Scikit-learn, etc. Big Data Processing: Leverage PySpark extensively for distributed data processing, ETL operations, and running machine learning algorithms on big data platforms (e.g., Hadoop, Databricks, Spark clusters). Write efficient and optimized SQL queries for data extraction, transformation, and loading from relational and non-relational databases. Deployment & MLOps: Collaborate with MLOps engineers and software development teams to integrate and deploy machine learning and deep learning models into production environments. Monitor model performance, identify degradation, and implement retraining strategies to ensure sustained accuracy and relevance. Contribute to building CI/CD pipelines for ML model deployment. Insights & Communication: Translate complex analytical findings and model results into clear, concise, and actionable insights for both technical and non-technical stakeholders. Create compelling data visualizations and reports to effectively communicate findings and recommendations. Act as a subject matter expert, guiding business teams on data-driven decision-making. Research & Innovation: Stay abreast of the latest advancements in data science, machine learning, deep learning, and big data technologies. Proactively identify opportunities to apply new techniques and tools to enhance existing solutions or develop new capabilities. Required Qualifications: Bachelor's or Master's degree in Computer Science, Statistics, Mathematics, Engineering, or a related quantitative field. 3-6 years of hands-on experience as a Data Scientist or in a similar role. Expert proficiency in Python for data science, including libraries such as Pandas, NumPy, Scikit-learn. Strong expertise in PySpark for large-scale data processing and machine learning. Advanced SQL skills with the ability to write complex, optimized queries for data extraction and manipulation. Proven experience in applying Machine Learning algorithms to real-world problems. Solid understanding and hands-on experience with Deep Learning frameworks (e.g., TensorFlow, PyTorch) and architectures. Experience with big data technologies like Hadoop or Spark ecosystem. Strong understanding of statistical concepts, hypothesis testing, and experimental design. Excellent problem-solving, analytical, and critical thinking skills. Ability to work independently and collaboratively in a fast-paced environment. Strong communication and presentation skills, with the ability to explain complex technical concepts to diverse audiences.

Posted 5 days ago

Apply

7.0 years

0 Lacs

Greater Kolkata Area

On-site

Linkedin logo

Job Description Job Title: Automation Tester - Selenium, python, databricks Candidate Specification: 7 + years, Immediate to 30 days. Job Description Experience with Automated Testing. Ability to code and read a programming language (Python). Experience in pytest, selenium(python). Experience working with large datasets and complex data environments. Experience with airflow, Databricks, Data lake, Pyspark. Knowledge and working experience in Agile methodologies. Experience in CI/CD/CT methodology. Experience in Test methodologies. Skills Required RoleAutomation Tester Industry TypeIT/ Computers - Software Functional Area Required Education B Tech Employment TypeFull Time, Permanent Key Skills SELENIUM PYTHON DATABRICKS Other Information Job CodeGO/JC/100/2025 Recruiter NameSheena Rakesh Show more Show less

Posted 5 days ago

Apply

2.0 years

0 Lacs

Bengaluru, Karnataka, India

On-site

Linkedin logo

Job Description Your Responsibilities Do you want to join a team passionate about engineering data solutions to inform business decisions and transform business performance? ADM is seeking a Data Engineer, someone who thrives in an environment continuously integrating and deploying data products and analytics driving business value. The Data Engineer is a team player who sees problems as challenges to be creatively solved in collaboration with other data engineers, data scientists, and business colleagues. The ideal candidate is a driven learner who develops and sharpens skills building data pipelines, transforming raw data into useful data systems, and optimizing the data delivery architecture. Learn to design, build, refactor, and maintain data pipelines using Microsoft Azure, Databricks, SAP Datasphere, SQL, Azure Data Factory , Python, and PySpark to meet business requirements for reporting, analysis, and data science Participate in designing, and integrating fault tolerance and enhancements into data pipelines to improve quality and performance Monitor data pipelines using analytic tools to develop actionable insights into performance issues Perform root cause analysis and solve problems using analytical and technical skills to optimize data delivery and reduce costs Adhere to code standards and DataOps and MLOps best practices to accelerate and continuously improve data system performance Your Profile 2+ years proven Data Engineering experience Bachelor’s degree in computer science, software engineering, information technology or equivalent combination of data engineering professional experience and education. Knowledge of Microsoft Azure, SQL, Databricks, SAP Datasphere, Azure Data Factory, Python, PySpark, Power BI or other cloud-based data systems Knowledge of Azure DevOps, GitHub, CI/CD are a plus Working knowledge of relational database systems Task management and organizational skills Knowledge of or demonstrated experience building cloud ETL pipelines using code or ETL platforms utilizing database connections, APIs, or file-based Knowledge of data manipulations and processing techniques to extract value from large, disconnected datasets Continuous learning to upskill data engineering techniques and business acumen #IncludingYou Diversity, equity, inclusion and belonging are cornerstones of ADM’s efforts to continue innovating, driving growth, and delivering outstanding performance. We are committed to attracting and retaining a diverse workforce and create welcoming, truly inclusive work environments — environments that enable every ADM colleague to feel comfortable on the job, make meaningful contributions to our success, and grow their career. We respect and value the unique backgrounds and experiences that each person can bring to ADM because we know that diversity of perspectives makes us better, together. For more information regarding our efforts to advance Diversity, Equity, Inclusion & Belonging, please visit our website here: Diversity, Equity and Inclusion | ADM. About ADM At ADM, we unlock the power of nature to provide access to nutrition worldwide. With industry-advancing innovations, a complete portfolio of ingredients and solutions to meet any taste, and a commitment to sustainability, we give customers an edge in solving the nutritional challenges of today and tomorrow. We’re a global leader in human and animal nutrition and the world’s premier agricultural origination and processing company. Our breadth, depth, insights, facilities and logistical expertise give us unparalleled capabilities to meet needs for food, beverages, health and wellness, and more. From the seed of the idea to the outcome of the solution, we enrich the quality of life the world over. Learn more at www.adm.com. Req/Job ID 97994BR Ref ID Show more Show less

Posted 5 days ago

Apply

2.0 - 6.0 years

7 - 17 Lacs

Chennai

Hybrid

Naukri logo

We are seeking a highly skilled and motivated Azure Data Engineer to join our growing data team. In this role, you will be responsible for designing, developing, and maintaining scalable and robust data pipelines and data solutions within the Microsoft Azure ecosystem. You will work closely with data scientists, analysts, and business stakeholders to understand data requirements and translate them into effective data architectures. The ideal candidate will have a strong background in data warehousing, ETL/ELT processes, and a deep understanding of Azure data services. Responsibilities: Design, build, and maintain scalable and efficient data pipelines using Azure Data Factory, Azure Synapse Analytics, Azure Databricks, or other relevant Azure services. Develop and optimize data ingestion processes from various source systems (on-premises, cloud, third-party APIs) into Azure data platforms. Implement data warehousing solutions, including dimensional modeling and data lake strategies, using Azure Synapse Analytics, Azure Data Lake Storage Gen2, or Azure SQL Database. Write, optimize, and maintain complex SQL queries, stored procedures, and data transformation scripts. Develop and manage data quality checks, data validation processes, and data governance policies. Monitor and troubleshoot data pipeline issues, ensuring data accuracy and availability. Collaborate with data scientists and analysts to support their data needs for reporting, analytics, and machine learning initiatives. Implement security best practices for data storage and access within Azure. Participate in code reviews, contribute to architectural discussions, and promote best practices in data engineering. Stay up-to-date with the latest Azure data technologies and trends, proposing and implementing improvements where applicable. Document data flows, architectures, and operational procedures. Qualifications: Required: Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. 3 to 5 years of professional experience as a Data Engineer, with a strong focus on Microsoft Azure data platforms. Proven experience with Azure Data Factory for orchestration and ETL/ELT. Solid understanding and hands-on experience with Azure Synapse Analytics (SQL Pool, Spark Pool) or Azure SQL Data Warehouse. Proficiency in SQL and experience with relational databases. Experience with Azure Data Lake Storage Gen2. Familiarity with data modeling, data warehousing concepts (e.g., Kimball methodology), and ETL/ELT processes. Strong programming skills in Python or Spark (PySpark). Experience with version control systems (e.g., Git). Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills.

Posted 5 days ago

Apply

2.0 - 5.0 years

0 Lacs

Pune, Maharashtra, India

On-site

Linkedin logo

The Data Analytics Intmd Analyst is a developing professional role. Deals with most problems independently and has some latitude to solve complex problems. Integrates in-depth specialty area knowledge with a solid understanding of industry standards and practices. Good understanding of how the team and area integrate with others in accomplishing the objectives of the subfunction/ job family. Applies analytical thinking and knowledge of data analysis tools and methodologies. Requires attention to detail when making judgments and recommendations based on the analysis of factual information. Typically deals with variable issues with potentially broader business impact. Applies professional judgment when interpreting data and results. Breaks down information in a systematic and communicable manner. Developed communication and diplomacy skills are required in order to exchange potentially complex/sensitive information. Moderate but direct impact through close contact with the businesses' core activities. Quality and timeliness of service provided will affect the effectiveness of own team and other closely related teams. Responsibilities: Integrates in-depth data analysis knowledge with a solid understanding of industry standards and practices. Demonstrates a Good understanding of how data analytics teams and area integrate with others in accomplishing objectives. Applies project management skills. Applies analytical thinking and knowledge of data analysis tools and methodologies. Analyzes factual information to make accurate judgments and recommendations focused on local operations and broader impacts. Applies professional judgment when interpreting data and results breaking down information in a systematic and communicable manner. Employs developed communication and diplomacy skills to exchange potentially complex/sensitive information. Demonstrates attention to quality and timeliness of service to ensure the effectiveness of the team and group. Provides informal guidance or on-the-job-training to new team members. Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency. Qualifications: 2-5 years relevant experience Knowledge of tools for statistical modeling of large data sets Ability to effectively use complex analytical, interpretive and problem solving techniques Demonstrated interpersonal, verbal and written communication skills Education: Bachelor’s/University degree or equivalent experience This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required. 2-4 years of experience as a python developer with expertise in automation testing to design, develop, and automate robust software solutions and testing frameworks like Pytest, Behave etc. 2-4 years of experience as Big Data Engineer to develop, optimize, and manage large-scale data processing systems and analytics platforms. 3-4 years of experience in distributed data processing & near real-time data analytics using PySpark. 2-5 years of experience in designing, managing, and optimizing Kafka-based messaging systems for real-time data streaming. Familiarity with CI/CD pipelines, version control systems (e.g., Git), and DevOps practic ------------------------------------------------------ Job Family Group: Technology ------------------------------------------------------ Job Family: Data Analytics ------------------------------------------------------ Time Type: Full time ------------------------------------------------------ Most Relevant Skills Please see the requirements listed above. ------------------------------------------------------ Other Relevant Skills For complementary skills, please see above and/or contact the recruiter. ------------------------------------------------------ Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi. View Citi’s EEO Policy Statement and the Know Your Rights poster. Show more Show less

Posted 5 days ago

Apply

0 years

0 Lacs

Chennai, Tamil Nadu, India

On-site

Linkedin logo

Join us as a Software Engineer - PySpark This is an opportunity for a driven Software Engineer to take on an exciting new career challenge Day-to-day, you'll be engineering and maintaining innovative, customer centric, high performance, secure and robust solutions It’s a chance to hone your existing technical skills and advance your career while building a wide network of stakeholders What you'll do In your new role, you’ll be working within a feature team to engineer software, scripts and tools, as well as liaising with other engineers, architects and business analysts across the platform. You’ll Also Be Producing complex and critical software rapidly and of high quality which adds value to the business Working in permanent teams who are responsible for the full life cycle, from initial development, through enhancement and maintenance to replacement or decommissioning Collaborating to optimise our software engineering capability Designing, producing, testing and implementing our working software solutions Working across the life cycle, from requirements analysis and design, through coding to testing, deployment and operations The skills you'll need To take on this role, you’ll need a background in software engineering, software design, and architecture, and an understanding of how your area of expertise supports our customers. You'll need four to seven years of experience in Pyspark, Python, AWS, SQL and Tableau. You'll also need experience in developing and supporting ETL pipelines and tableau reporting. You’ll Also Need Experience of working with development and testing tools, bug tracking tools and wikis Experience in multiple programming languages or low code toolsets Experience of DevOps and Agile methodology and associated toolsets A background in solving highly complex, analytical and numerical problems Experience of implementing programming best practice, especially around scalability, automation, virtualisation, optimisation, availability and performance Show more Show less

Posted 5 days ago

Apply

6.0 - 10.0 years

14 - 24 Lacs

Hyderabad

Work from Office

Naukri logo

Role & responsibilities Job Title: Data Engineer Years of experience: 6 to 10 years (Minimum 5 years of relevant experience) Work Mode: Work From Office Hyderabad Notice Period-Immediate to 30 Days only Key Skills: Python, SQL, AWS, Spark, Databricks - ( Mandate) Airflow- Good to have

Posted 5 days ago

Apply

2.0 - 10.0 years

0 Lacs

Noida, Uttar Pradesh, India

On-site

Linkedin logo

About Veersa - Veersa Technologies is a US-based IT services and AI enablement company founded in 2020, with a global delivery center in Noida (Sector 142). Founded by industry leaders with an impressive 85% YoY growth A profitable company since inception Team strength: Almost 400 professionals and growing rapidly Our Services Include Digital & Software Solutions: Product Development, Legacy Modernization, Support Data Engineering & AI Analytics: Predictive Analytics, AI/ML Use Cases, Data Visualization Tools & Accelerators: AI/ML-embedded tools that integrate with client systems Tech Portfolio Assessment: TCO analysis, modernization roadmaps, etc. Tech Stack - * AI/ML, IoT, Blockchain, MEAN/MERN stack, Python, GoLang, RoR, Java Spring Boot, Node.js Databases: PostgreSQL, MySQL, MS SQL, Oracle Cloud: AWS & Azure (Serverless Architecture) Website: https://veersatech.com LinkedIn: Feel free to explore our company profile About The Role We are seeking a highly skilled and experienced Data Engineer & Lead Data Engineer to join our growing data team. This role is ideal for professionals with 2 to 10 years of experience in data engineering, with a strong foundation in SQL, Databricks, Spark SQL, PySpark, and BI tools like Power BI or Tableau. As a Data Engineer, you will be responsible for building scalable data pipelines, optimizing data processing workflows, and enabling insightful reporting and analytics across the organization. Key Responsibilities Design and develop robust, scalable data pipelines using PySpark and Databricks. Write efficient SQL and Spark SQL queries for data transformation and analysis. Work closely with BI teams to enable reporting through Power BI or Tableau. Optimize performance of big data workflows and ensure data quality. Collaborate with business and technical stakeholders to gather and translate data requirements. Implement best practices for data integration, processing, and governance. Required Qualifications Bachelor’s degree in Computer Science, Engineering, or a related field. 2–10 years of experience in data engineering or a similar role. Strong experience with SQL, Spark SQL, and PySpark. Hands-on experience with Databricks for big data processing. Proven experience with BI tools such as Power BI and/or Tableau. Strong understanding of data warehousing and ETL/ELT concepts. Good problem-solving skills and the ability to work in cross-functional teams. Nice To Have Experience with cloud data platforms (Azure, AWS, or GCP). Familiarity with CI/CD pipelines and version control tools (e.g., Git). Understanding of data governance, security, and compliance standards. Exposure to data lake architectures and real-time streaming data pipelines. Show more Show less

Posted 5 days ago

Apply

2.0 - 10.0 years

0 Lacs

Ghaziabad, Uttar Pradesh, India

On-site

Linkedin logo

About Veersa - Veersa Technologies is a US-based IT services and AI enablement company founded in 2020, with a global delivery center in Noida (Sector 142). Founded by industry leaders with an impressive 85% YoY growth A profitable company since inception Team strength: Almost 400 professionals and growing rapidly Our Services Include Digital & Software Solutions: Product Development, Legacy Modernization, Support Data Engineering & AI Analytics: Predictive Analytics, AI/ML Use Cases, Data Visualization Tools & Accelerators: AI/ML-embedded tools that integrate with client systems Tech Portfolio Assessment: TCO analysis, modernization roadmaps, etc. Tech Stack - * AI/ML, IoT, Blockchain, MEAN/MERN stack, Python, GoLang, RoR, Java Spring Boot, Node.js Databases: PostgreSQL, MySQL, MS SQL, Oracle Cloud: AWS & Azure (Serverless Architecture) Website: https://veersatech.com LinkedIn: Feel free to explore our company profile About The Role We are seeking a highly skilled and experienced Data Engineer & Lead Data Engineer to join our growing data team. This role is ideal for professionals with 2 to 10 years of experience in data engineering, with a strong foundation in SQL, Databricks, Spark SQL, PySpark, and BI tools like Power BI or Tableau. As a Data Engineer, you will be responsible for building scalable data pipelines, optimizing data processing workflows, and enabling insightful reporting and analytics across the organization. Key Responsibilities Design and develop robust, scalable data pipelines using PySpark and Databricks. Write efficient SQL and Spark SQL queries for data transformation and analysis. Work closely with BI teams to enable reporting through Power BI or Tableau. Optimize performance of big data workflows and ensure data quality. Collaborate with business and technical stakeholders to gather and translate data requirements. Implement best practices for data integration, processing, and governance. Required Qualifications Bachelor’s degree in Computer Science, Engineering, or a related field. 2–10 years of experience in data engineering or a similar role. Strong experience with SQL, Spark SQL, and PySpark. Hands-on experience with Databricks for big data processing. Proven experience with BI tools such as Power BI and/or Tableau. Strong understanding of data warehousing and ETL/ELT concepts. Good problem-solving skills and the ability to work in cross-functional teams. Nice To Have Experience with cloud data platforms (Azure, AWS, or GCP). Familiarity with CI/CD pipelines and version control tools (e.g., Git). Understanding of data governance, security, and compliance standards. Exposure to data lake architectures and real-time streaming data pipelines. Show more Show less

Posted 5 days ago

Apply

2.0 - 10.0 years

0 Lacs

Delhi, India

On-site

Linkedin logo

About Veersa - Veersa Technologies is a US-based IT services and AI enablement company founded in 2020, with a global delivery center in Noida (Sector 142). Founded by industry leaders with an impressive 85% YoY growth A profitable company since inception Team strength: Almost 400 professionals and growing rapidly Our Services Include Digital & Software Solutions: Product Development, Legacy Modernization, Support Data Engineering & AI Analytics: Predictive Analytics, AI/ML Use Cases, Data Visualization Tools & Accelerators: AI/ML-embedded tools that integrate with client systems Tech Portfolio Assessment: TCO analysis, modernization roadmaps, etc. Tech Stack - * AI/ML, IoT, Blockchain, MEAN/MERN stack, Python, GoLang, RoR, Java Spring Boot, Node.js Databases: PostgreSQL, MySQL, MS SQL, Oracle Cloud: AWS & Azure (Serverless Architecture) Website: https://veersatech.com LinkedIn: Feel free to explore our company profile About The Role We are seeking a highly skilled and experienced Data Engineer & Lead Data Engineer to join our growing data team. This role is ideal for professionals with 2 to 10 years of experience in data engineering, with a strong foundation in SQL, Databricks, Spark SQL, PySpark, and BI tools like Power BI or Tableau. As a Data Engineer, you will be responsible for building scalable data pipelines, optimizing data processing workflows, and enabling insightful reporting and analytics across the organization. Key Responsibilities Design and develop robust, scalable data pipelines using PySpark and Databricks. Write efficient SQL and Spark SQL queries for data transformation and analysis. Work closely with BI teams to enable reporting through Power BI or Tableau. Optimize performance of big data workflows and ensure data quality. Collaborate with business and technical stakeholders to gather and translate data requirements. Implement best practices for data integration, processing, and governance. Required Qualifications Bachelor’s degree in Computer Science, Engineering, or a related field. 2–10 years of experience in data engineering or a similar role. Strong experience with SQL, Spark SQL, and PySpark. Hands-on experience with Databricks for big data processing. Proven experience with BI tools such as Power BI and/or Tableau. Strong understanding of data warehousing and ETL/ELT concepts. Good problem-solving skills and the ability to work in cross-functional teams. Nice To Have Experience with cloud data platforms (Azure, AWS, or GCP). Familiarity with CI/CD pipelines and version control tools (e.g., Git). Understanding of data governance, security, and compliance standards. Exposure to data lake architectures and real-time streaming data pipelines. Show more Show less

Posted 5 days ago

Apply

0.0 - 2.0 years

0 Lacs

Pune, Maharashtra, India

On-site

Linkedin logo

The Data Analytics Analyst 2 is a developing professional role. Applies specialty area knowledge in monitoring, assessing, analyzing and/or evaluating processes and data. Identifies policy gaps and formulates policies. Interprets data and makes recommendations. Researches and interprets factual information. Identifies inconsistencies in data or results, defines business issues and formulates recommendations on policies, procedures or practices. Integrates established disciplinary knowledge within own specialty area with basic understanding of related industry practices. Good understanding of how the team interacts with others in accomplishing the objectives of the area. Develops working knowledge of industry practices and standards. Limited but direct impact on the business through the quality of the tasks/services provided. Impact of the job holder is restricted to own team. Responsibilities: Identifies policy gaps and formulates policies. Interprets data and make recommendations. Integrates established disciplinary knowledge within own specialty area with basic understanding of related industry practices. Makes judgments and recommendations based on analysis and specialty area knowledge. Researches and interprets factual information. Identifies inconsistencies in data or results, define business issues and formulate recommendations on policies, procedures or practices. Exchanges information in a concise and logical way as well as be sensitive to audience diversity. Appropriately assess risk when business decisions are made, demonstrating particular consideration for the firm's reputation and safeguarding Citigroup, its clients and assets, by driving compliance with applicable laws, rules and regulations, adhering to Policy, applying sound ethical judgment regarding personal behavior, conduct and business practices, and escalating, managing and reporting control issues with transparency. Qualifications: 0-2 years experience using tools for statistical modeling of large data sets Education: Bachelor’s/University degree or equivalent experience This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required. Experience as a python developer with expertise in automation testing to design, develop, and automate robust software solutions and testing frameworks like Pytest, Behave etc. 2-4 years of experience as Big Data Engineer to develop, optimize, and manage large-scale data processing systems and analytics platforms. 3-4 years of experience in distributed data processing & near real-time data analytics using PySpark. Familiarity with CI/CD pipelines, version control systems (e.g., Git), and DevOps practices ------------------------------------------------------ Job Family Group: Technology ------------------------------------------------------ Job Family: Data Analytics ------------------------------------------------------ Time Type: Full time ------------------------------------------------------ Most Relevant Skills Please see the requirements listed above. ------------------------------------------------------ Other Relevant Skills For complementary skills, please see above and/or contact the recruiter. ------------------------------------------------------ Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law. If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi. View Citi’s EEO Policy Statement and the Know Your Rights poster. Show more Show less

Posted 5 days ago

Apply

Exploring PySpark Jobs in India

PySpark, a powerful data processing framework built on top of Apache Spark and Python, is in high demand in the job market in India. With the increasing need for big data processing and analysis, companies are actively seeking professionals with PySpark skills to join their teams. If you are a job seeker looking to excel in the field of big data and analytics, exploring PySpark jobs in India could be a great career move.

Top Hiring Locations in India

Here are 5 major cities in India where companies are actively hiring for PySpark roles: 1. Bangalore 2. Pune 3. Hyderabad 4. Mumbai 5. Delhi

Average Salary Range

The estimated salary range for PySpark professionals in India varies based on experience levels. Entry-level positions can expect to earn around INR 6-8 lakhs per annum, while experienced professionals can earn upwards of INR 15 lakhs per annum.

Career Path

In the field of PySpark, a typical career progression may look like this: 1. Junior Developer 2. Data Engineer 3. Senior Developer 4. Tech Lead 5. Data Architect

Related Skills

In addition to PySpark, professionals in this field are often expected to have or develop skills in: - Python programming - Apache Spark - Big data technologies (Hadoop, Hive, etc.) - SQL - Data visualization tools (Tableau, Power BI)

Interview Questions

Here are 25 interview questions you may encounter when applying for PySpark roles:

  • Explain what PySpark is and its main features (basic)
  • What are the advantages of using PySpark over other big data processing frameworks? (medium)
  • How do you handle missing or null values in PySpark? (medium)
  • What is RDD in PySpark? (basic)
  • What is a DataFrame in PySpark and how is it different from an RDD? (medium)
  • How can you optimize performance in PySpark jobs? (advanced)
  • Explain the difference between map and flatMap transformations in PySpark (basic)
  • What is the role of a SparkContext in PySpark? (basic)
  • How do you handle schema inference in PySpark? (medium)
  • What is a SparkSession in PySpark? (basic)
  • How do you join DataFrames in PySpark? (medium)
  • Explain the concept of partitioning in PySpark (medium)
  • What is a UDF in PySpark? (medium)
  • How do you cache DataFrames in PySpark for optimization? (medium)
  • Explain the concept of lazy evaluation in PySpark (medium)
  • How do you handle skewed data in PySpark? (advanced)
  • What is checkpointing in PySpark and how does it help in fault tolerance? (advanced)
  • How do you tune the performance of a PySpark application? (advanced)
  • Explain the use of Accumulators in PySpark (advanced)
  • How do you handle broadcast variables in PySpark? (advanced)
  • What are the different data sources supported by PySpark? (medium)
  • How can you run PySpark on a cluster? (medium)
  • What is the purpose of the PySpark MLlib library? (medium)
  • How do you handle serialization and deserialization in PySpark? (advanced)
  • What are the best practices for deploying PySpark applications in production? (advanced)

Closing Remark

As you explore PySpark jobs in India, remember to prepare thoroughly for interviews and showcase your expertise confidently. With the right skills and knowledge, you can excel in this field and advance your career in the world of big data and analytics. Good luck!

cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Featured Companies