1569 Data Pipeline Jobs - Page 10

IOT Cloud Engineer -JD and Assessment : Required Skills and Qualifications: IoT and Cloud Expertise o Strong experience with IoT platforms like Azure IoT Hub, AWS IoT Core, or GCP IoT Core. Proficiency in protocols such as MQTT, AMQP, CoAP, or HTTP for IoT communication. Hands-on experience with device provisioning, telemetry, and digital twins. Real-time Streaming and Data Pipeline Skills o Expertise in Apache Kafka, Kafka Streams, or equivalent streaming platforms. Experience in designing real-time applications using Apache Flink, Spark Streaming, or Azure Stream Analytics. o Knowledge of data pipeline orchestration tools like Airflow, Prefect, or Data Factory. Programming and Tools o Prof...

Job Title: Data & GenAI AWS Specialist Experience: 1-4 Years Location: Bangalore Mandatory Qualification: B.E./ B.Tech/ M.Tech/ MS from IIT or IISc ONLY Job Overview: We are seeking a seasoned Data & GenAI Specialist with deep expertise in AWS Managed Services (PaaS) to join our innovative team. The ideal candidate will have extensive experience in designing sophisticated, scalable architectures for data pipelines and Generative AI (GenAI) solutions leveraging cloud services. Key to this role is the ability to articulate architecture solutions clearly and effectively to customers, helping them conceptualize and implement advanced GenAI-driven applications tailored precisely to their business...

Hiring a Python Developer for a 6 month Project who can: *Design the data models *Build the backend APIs & web scrapers. * GCP experience will be a big plus. * Work with Data Pipelines, Web API, Integration. Required Candidate profile * 6 to 8 years or above experience in Python. * Knowledge of APIs & Web scrapers. * The role is majorly remote but initially have to work from office. * This is a 6 month project to start with.

Dear candidate, Greeting from BMW Tech works!! Join our Data Engineering Team to design and industrialize robust data pipelines and semantic layers for advanced analytics and AI-driven solutions. you will work across all phases of development from data ingestion and preparation to pipeline implementation, dashboard enablement, and deployment on cloud platforms. Youll collaborate in an international, interdisciplinary environment with an agile mindset. Our infrastructure leverages AWS services and big data technologies to process large-scale datasets and deliver high-performance solutions for enterprise use cases. What should you bring along? Strong programming skills in Python and Py Spark ,...

Role Overview: JMAN Group, a fast-growing data engineering and data science consultancy, is looking for a talented and experienced Data Engineer to join their team. As a Data Engineer at JMAN Group, you will be responsible for managing and leading a team of data engineers, diagnosing existing architecture, building analytics tools, ensuring data quality, and providing actionable insights into key business performance metrics. Your role will involve working with a variety of data warehouse platforms, data visualization tools, and cloud technologies to create innovative solutions for clients. Key Responsibilities: - Manage and lead a team of data engineers, providing guidance, mentorship, and ...

Roles and Responsibility Design, develop, and implement data pipelines and architectures. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems and databases. Ensure data quality and integrity through data validation and testing procedures. Optimize data processing workflows for improved performance and efficiency. Troubleshoot and resolve technical issues related to data engineering. Disclaimer: This job description has been sourced from a public domain and may have been modified by Naukri.com to improve clarity for our users. We encourage job seekers to verify all details directly with the employer via their o...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

We are looking for a self-starter to join our Data Engineering team. You will work in a fast-paced environment where you will get an opportunity to build and contribute to the full lifecycle development and maintenance of the data engineering platform. With the Data Engineering team you will get an opportunity to - Design and implement data engineering solutions that is scalable, reliable and secure on the Cloud environment Understand and translate business needs into data engineering solutions Build large scale data pipelines that can handle big data sets using distributed data processing techniques that supports the efforts of the data science and data application teams Partner with cross-...

Skill: GCP Data Engineer Experience : 4 to 13 Years Location : AIA - Gurgaon Mandatory Skills : GCP, DBT, Google Dataform, BigQuery, PySpark/Spark SQL, Advanced SQL, Agile Methodologies . Key Responsibilities : Design, build, and optimize scalable data pipelines using BigQuery, and PySpark. Leverage GCP-native services like Cloud Storage, Pub/Sub, Dataproc, Cloud Functions, and Composer for ETL/ELT workflows. Collaborate with cross-functional teams including Infra, Security, and DataOps for reliable, secure, and high-quality data delivery. Lead code reviews, mentor junior engineers, and enforce best practices in data engineering. Participate in Agile sprints, backlog grooming, and Jira-based...

We are seeking a skilled and experienced Data Engineer to join our team, for designing, implementing, and maintaining our data infrastructure, pipelines, and systems.

Job Description Q2 is seeking a Senior Data Engineer to join our Data Insights organization . As part of the Data Engineering team, you will build and maintain the Cloud infrastructure for developing and deploying data analytic products and core data transformation jobs. Our team collaborates closely with data scientists and other product engineering teams. Our goal is to enable Data Scientists to focus on creating data products through discovery and development of transformation jobs and machine-learning models, while ensuring their quick deployment to production. Our team supports the data and machine-learning aspects for multiple Q2 products including Online Banking, PrecisionLender and F...

Design, build a BI&analytics solution Implement batch & near real time data movement design patterns define best practices in DE Design &develop optimal cloud data solutions (lakes, warehouses ,marts, analytics) by collaborating with diverse IT teams Required Candidate profile Exp in designing, implementing, managing large-scale & ETL solutions, BI data modelling, DE, ETL, multi-dimensional DW, cubes, with expertise in relevant languages and frameworks like SQL, Python

6+ years of experience in data engineering, specifically in cloud environments like AWS. Proficiency in Python and PySpark for data processing and transformation tasks. Solid experience with AWS Glue for ETL jobs and managing data workflows. Hands-on experience with AWS Data Pipeline (DPL) for workflow orchestration. Strong experience with AWS services such as S3, Lambda, Redshift, RDS, and EC2. Technical Skills : Deep understanding of ETL concepts and best practices.. Strong knowledge of SQL for querying and manipulating relational and semi-structured data. Experience with Data Warehousing and Big Data technologies, specifically within AWS. Additional Skills: Experience with AWS Lambda for ...

6+ years of experience in data engineering, specifically in cloud environments like AWS. Proficiency in Python and PySpark for data processing and transformation tasks. Solid experience with AWS Glue for ETL jobs and managing data workflows. Hands-on experience with AWS Data Pipeline (DPL) for workflow orchestration. Strong experience with AWS services such as S3, Lambda, Redshift, RDS, and EC2. Technical Skills : Deep understanding of ETL concepts and best practices.. Strong knowledge of SQL for querying and manipulating relational and semi-structured data. Experience with Data Warehousing and Big Data technologies, specifically within AWS. Additional Skills: Experience with AWS Lambda for ...

We are looking for a highly skilled and experienced Software Engineer-Data Engineer to join our team at CGI Information Systems and Management Consultants. The ideal candidate will have a strong background in software engineering and data analysis, with excellent problem-solving skills. Roles and Responsibility Design, develop, and test software applications for data processing and analysis. Collaborate with cross-functional teams to identify business requirements and develop solutions. Develop and maintain large-scale databases and data systems. Analyze complex data sets to identify trends and patterns. Implement data quality checks and validation processes. Optimize database performance an...

We are looking for skilled ML Engineer with strong expertise in Python, data engineering, model development, and API deployment. should be able to work from data extraction cleaning to model building, MLOps, production deployment using FastAPI.

Role & responsibilities Proven experience as a Senior Data Engineer, with a focus on Google Cloud Platform (GCP). Strong proficiency in GCP services such as BigQuery, Dataproc, Dataflow, Pub/Sub, Cloud Composer, and Cloud Storage. Expertise in SQL and database management. Experience with ETL processes and tools. Proficiency in Python programming language. Knowledge of data modeling, data warehousing, and data lakes. Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills. Preferred Skills: Experience with machine learning and AI frameworks. Knowledge of containerization and orchestration tools. Experience with data visualization tools (Tableau,...

We are seeking a Senior Data Analytics Engineer who can take ownership of analytics ecosystems end-to-end ( full-stack) from raw data ingestion to business-ready dashboards. The ideal candidate has a blend of data engineering skills and an eye for storytelling through data. You"ll work with Python , Databricks , ADF , Snowflake , MS Fabric and SQL to build robust data pipelines, and use Power BI, DAX, and Power Query to deliver actionable insights. You will play a critical role in designing, developing, and maintaining end-to-end data solutions, from building robust data pipelines and models to delivering insightful dashboards and visualizations that empower informed decision-making across t...

Role & responsibilities: Architect and maintain structured databases for tick-level and MFT data. Develop efficient ingestion & ETL pipelines using Python/C++, handling large volumes in near-real time. Write optimised SQL queries and manage schemas for time-series and reference data. Implement automated data validation, reconciliation, and version control. Work with quants to expose clean datasets for research and live trading systems. Ensure data integrity across time zones, instruments, and asset classes. Manage historical data archives and set up policies for retention, compression, and retrieval. Collaborate with the execution team to align live feeds with stored data. Requirements : Str...

Greetings!!! TCS is hiring... Role: AWS Data Engineer EXP: 5- 8 YEARS LOCATION: Hyderabad/ Kolkata **** Notice Period- 30 Day s *** Desired Skills -Must-Have AWS Python , AWS Glue, Lambda, Athena, AWS Pyspark Good hands-on experience in Python programming and Pyspark/Scala Data Engineering experience using AWS core services (Lambda, Glue, EMR, S3 and RedShift) Good knowledge on any of the ETL tools and SQLs. Inviting applications with updated resume.

Greetings!!! TCS is hiring... Role: Snowflake Developer EXP: 6 - 10 YEARS LOCATION: Hyderabad/ Bangalore / Chennai/ Pune **** Notice Period- Immediate to Max 30 Day s *** Desired Skills -Must-Have SQL, Snowflake, Snowflake modeling roles, databases, schemas, ETL , python Migration to AWS Snowflake platform and Knowledge in S3 buckets Knowledge of SQL language and cloud-based technologies Data warehousing concepts, data modelling, metadata management Data lakes, multi-dimensional models, data dictionaries Migration to AWS Snowflake platform and Knowledge in S3 buckets Performance tuning and setting up resource monitors Snowflake modelling – roles, databases, schemas (Hands on must) SQL perfor...

Job Title: Data Engineer Location: Kolkata, India Experience: Minimum 5 years in Data Engineering Joining: Immediate Job Description: We are looking for a skilled Data Engineer to design, build, and maintain scalable data architectures and pipelines. The ideal candidate will have hands-on experience in data transformation, data lake solutions, and analytics enablement, along with strong problem-solving and collaboration skills. Key Responsibilities: Develop & Maintain: Design, build, and maintain scalable data architectures and pipelines to support analytics and business requirements. Ensure proper documentation of solutions. Monitor & Troubleshoot: Support stakeholders with data issues, mon...

Lead the development and implementation of data engineering solutions using AWS. Collaborate with cross-functional teams to design and implement scalable data pipelines. Develop and maintain large-scale data architectures and systems. Ensure high-quality data processing and analysis across the organization. Mentor junior engineers and provide technical guidance. Stay up-to-date with industry trends and emerging technologies in data engineering. Job Requirements Strong understanding of data engineering principles and practices. Experience working with large datasets and distributed computing environments. Excellent leadership and communication skills. Ability to work collaboratively with cros...

Design, develop, and implement data pipelines and architectures. Collaborate with cross-functional teams to identify and prioritize project requirements. Develop and maintain large-scale data systems and databases. Ensure data quality and integrity through data validation and testing procedures. Optimize data processing workflows for improved performance and efficiency. Troubleshoot and resolve technical issues related to data engineering. Job Requirements Strong understanding of data engineering principles and practices. Experience with data modeling, database design, and data warehousing concepts. Proficiency in programming languages such as Python or Java. Excellent problem-solving skills...

Job Description_ Senior Data Engineer, Secure ETL & Ingestion Platform/Data Architect Why would you like to join us? TransOrg Analytics specializes in Data Science, Data Engineering and Generative AI, providing advanced analytics solutions to industry leaders and Fortune 500 companies across India, US, APAC and the Middle East. We leverage data science to streamline, optimize, and accelerate our clients' businesses. Visit at www.transorg.com to know more about us. Function: Data Engineering / Data Platform Role Overview - We are looking for a Senior Data Engineer to design and build our next-generation ETL / ingestion pipeline with a strong focus on security, performance, reliability, and go...