1166 Data Pipeline Jobs - Page 7

Overview Azure Data Architect Bangalore Aptean is changing Our ERP solutions are transforming a huge range of global businesses, from food producers to manufacturers In a world of generic enterprise software, we provide targeted solutions that bring together the very best technology and drive greater results With over 4500 employees, 90 different products and a global client base, theres no better time to advance your career at Aptean, Are you ready for whats next, nowWe are! If being part of a dynamic, high growth organization excites you and you are a Senior Data Architect, and eager to learn and grow, then this opportunity is for you! Our fast-paced environment and dynamic global R&D depa...

Position: Snowflake Data Engineer Work Location: Hyderabad/ Bangalore(WFO) Experience : 3+ About the opportunity : We are seeking a highly skilled and experienced Snowflake Developer with a strong background in SQL, Python, and a minimum of 3 years of hands-on experience with Snowflake. The ideal candidate will be a Snowflake Certified with a proven track record in data warehousing, data modelling, and implementing ETL/ELT pipelines using industry-standard tools. Primary Roles & Responsibilities : • Design, develop, and optimize data pipelines and ETL/ELT processes in Snowflake. • Develop and optimize complex SQL queries for data extraction, transformation, and reporting. • Write robust Pyth...

Project Duration 3 months and can get extended. Data Engineer (Databricks experience is mandatory) Exp: 3-8 Years Qualification: B. Tech/B.Sc. (IT) Expertise in Python, PySpark , and Apache Spark for data engineering Strong experience in building and optimizing ETL/data transformation pipelines Proficient in Databricks and Delta Lake storage format (Databricks experience is mandatory) Cloud platforms like AWS (Glue, EMR), Snowflake, Dataiku, Alteryx Skilled in managing large-scale, multi-dimensional datasets and designing scalable data architectures Solid understanding of data modeling and data warehouse concepts (On-Prem & Cloud) Familiar with modern data tools and continually updated on em...

Hi, Hope you are doing well. Please find the below Job description. Kindly share resume along with your LWD or Notice time join and your Current CTC and Expected CTC. Exact area in Bangalore and share it to manju@nam-it.com for immediate Response. Send resumes with details for immediate response to manju@nam-it.com Job Title: Data Lake Delivery Lead (Snowflake/Databricks on AWS) Company: NAM India Pvt. Ltd. Location: Bangalore, India Experience: 1214 Years Employment Type: Full-time Mode: All 5 Days Onsite ( Banashankari 2nd Stage location of work) Need Immediate Joiners only locals to Bangalore only. Required Skills & Experience 1214 years of overall experience in data engineering, analytic...

Responsibilities: * Design, develop & maintain cloud data pipelines using AWS/Azure & Python. * Optimize ETL processes with PySpark, SQL & DataBricks. * Collaborate on data warehouse architecture & implementation.

Job Summary: We are seeking a skilled Data Analyst with 3 to 5 years of experience to join our dynamic team. The successful candidate will be responsible for managing data pipelines, writing Spark-SQL code, and resolving stakeholder inquiries about data. This role requires a strong analytical mindset, excellent problem-solving skills, and the ability to communicate effectively with various stakeholders. Key Responsibilities: Data Pipeline Management: Design, build, and maintain efficient and reliable data pipelines to support business needs. Ensure data quality and integrity throughout the data lifecycle. Spark-SQL Development: Write and optimize Spark-SQL code to process large datasets effi...

We are seeking a talented Python Developer to join our dynamic team. The ideal candidate will have a strong background in data manipulation using Pandas and experience in building high-performance APIs with FastAPI . If you're passionate about technology and eager to contribute to impactful projects, we want to hear from you! Key Responsibilities: Design, develop, and maintain robust APIs using FastAPI . Utilize Pandas for data manipulation, analysis, and transformation. Collaborate with cross-functional teams to gather requirements and deliver solutions. Write clean, maintainable, and efficient code. Conduct code reviews and provide constructive feedback to peers. Troubleshoot and debug app...

Role Overview: You will be a highly motivated Product Manager with a strong interest in investment technologies, quantitative research, and project management to support the Quant research and development functions. As an Associate Product Manager on the Quantitative Product team, your main responsibility will be to partner with quants, technology, and portfolio managers to deliver advanced portfolio optimization and trade-validation tools. You will own project plans, user stories, and release tracks end-to-end to ensure the investment-tech roadmap advances on time and with high quality. The ideal candidate will have a strong project management mindset, experience working with different stak...

Tool required: ETL, Data Warehouse, Data Modeling, SQL, BI, Cloud Services, Python, Scalable Cloud Pipelines Experience - 4.5 Yrs and Above (Hiring Across Levels) What youll do Design/implement complex and scalable enterprise data processing and BI reporting solutions. Design, build, and optimize ETL pipelines or underlying code to enhance data warehouse systems. Work towards optimizing the overall costs incurred due to system infrastructure, operations, change management, etc. Deliver end-to-end data solutions across multiple infrastructures and applications. Coach, mentor, and manage a team of junior associates to help them plan tasks effectively. Demonstrate overall client stakeholder and...

Job Description: As a Data Engineer, you will develop and maintain scalable data transformation and analytics infrastructure using Snowflake, DBT (Data Build Tool) , and modern development practices. You will work closely with data analysts, data scientists, and business stakeholders to deliver high-quality, tested, and documented data models that drive business decision-making. You will also contribute to data model optimization, performance tuning, and data quality improvements. JOB REQUIREMENTS Education and Certifications Bachelor's or master's degree in computer science engineering or a related field Certification in Snowflake is preferred Required Experience 3-5 years of total experien...

SUMMARY Wissen Technology is Hirin g for Data Engineer About Wissen Technology: At Wissen Technology, we deliver niche, custom-built products that solve complex business challenges across industries worldwide. Founded in 2015, our core philosophy is built around a strong product engineering mindset ensuring every solution is architected and delivered right the first time. Today, Wissen Technology has a global footprint with 2000+ employees across offices in the US, UK, UAE, India, and Australia . Our commitment to excellence translates into delivering 2X impact compared to traditional service providers. How do we achieve this? Through a combination of deep domain knowledge, cutting-edge tech...

The Data Architect Lead utilizes extensive technical expertise to spearhead the design and implementation of data architecture solutions focused on orchestrating data pipelines, ingestion, transformation, and accessibility within a hybrid-cloud environment. Acting as a liaison between business stakeholders and leadership in our dealer network, this role validates requirements for data process changes and recommends scalable, governance-aligned solutions. The position leads the design, development, maintenance, and testing of cloud solution components and data pipelines to ensure alignment with business strategy and compliance requirements. Key Accountabilities Data Architecture Design Lead a...

2+ Yrs of relevant experience Strong Python programming skills Experience with Generative AI / LLMs Hands-on with LangChain / Hugging Face / OpenAI APIs Knowledge of ML workflows & data pipelines Opportunity to work on cutting-edge AI solutions

Key Responsibilities: Expertise in design and implementation of SnapLogic-based integrations for enterprise applications. Develop and optimize data pipelines for real-time and batch processing using Kafka, KSQL, JSON/XML. Integrate data across platforms including Salesforce, Marketo, SAP, Snowflake, Oracle, SQL Server. Collaborate with cross-functional teams to gather requirements and deliver integration solutions. Implement CI/CD pipelines for seamless code migration and deployment. Provide production support , troubleshoot issues, and ensure SLAs are met. Monitor system performance using tools like Splunk and DataDog. Mentor junior team members and conduct training sessions on SnapLogic be...

? Mentor and guide engineering teams in all aspects of the SDLC. ? Lead technical design sessions and translate ideas into robust technical architecture. ? Develop and maintain scalable, high-performance data pipelines and streaming systems. ? Use Python, SQL, Scala, NodeJS, and work with relational databases, NoSQL stores, and Kafka. ? Refactor and optimize legacy codebases for improved performance and scalability. ? Collaborate with product and engineering teams to deliver innovative solutions in an agile environment. ? Support technical support teams with ad-hoc data queries and operational troubleshooting. ? Ensure data pipelines and APIs meet enterprise-grade performance and usability s...

? Design, develop, and maintain scalable data pipelines and streaming systems. ? Work with Python, SQL, Scala, NodeJS, and various data stores (relational, NoSQL, Kafka). ? Analyze healthcare and marketing datasets using SQL. ? Refactor and optimize existing codebases for performance improvements. ? Collaborate closely within an agile team to deliver creative technical solutions. ? Support ad-hoc data requests and assist technical support teams as needed. ? Ensure high performance of data ingestion, transformation, and API consumption layers. ? Follow best practices for coding, testing, CI/CD, and cloud deployments

Please find the JD below: Essential Skills: Experience: 6 to 8 yrs 1. Technical Expertise: Proficiency in AWS services such as Amazon S3, Redshift, EMR, Glue, Lambda, and Kinesis. Strong skills in SQL and experience with scripting languages like Python or Java. 2. Data Engineering Experience: Hands on experience in building and maintaining data pipelines, data modeling, and working with big data technologies. 3. Problem-Solving Skills: Ability to analyze complex data issues and develop effective solutions to optimize data processing and storage. 4. Communication and Collaboration: Strong interpersonal skills to work effectively with cross-functional teams and communicate technical concepts t...

We are seeking a skilled Data Engineer with hands-on experience in building, optimizing, and maintaining large-scale data pipelines and cloud-based data solutions. The ideal candidate will have strong expertise in AWS services , DBT , and Power BI , with a deep understanding of data modeling, transformation, and analytics for manufacturing data systems. Roles and Responsibilities Design, develop, and maintain ETL/ELT pipelines for data ingestion, transformation, and integration. Work extensively with AWS Glue , Athena , and S3 to manage and optimize data workflows. Develop and maintain SQL transformation code in DBT (Data Build Tool) . Ensure data quality, reliability, and performance optimi...

CANDIDATE WITH LESS THAN 6YRS OF EXPERIENCE PLEASE DO NOT APPLY FOR ROLE. Data Engineer Location-Bangalore& Pune L1 Virtual ; L2 face to face Experience -6+ Yrs Job Mode : C2H IMMEDIATE JOINERS ONLY Must have experience in Pyspark , SQL, ETL , data pipelines, AWS , Python JD Candidate should have 6 + years of experience in Data Engineering Must have experience in Pyspark , SQL, ETL , data pipelines, AWS , Python Nice to Have Life science domain experience (Pharma), Dremio, Redshift, Nifi ingestion, BI Tools (Tableau/PowerBI) Designing, creating, testing and maintaining the complete data management & processing systems. Working closely with the stakeholders & solution architect. Ensuring arch...

Salary: 15 to 25 LPA Exp: 4 to 7 years Location: Gurgaon/Pune/Bengalore Notice: Immediate to 30 days..!! Job Profile: Experienced Data Engineer with a strong foundation in designing, building, and maintaining scalable data pipelines and architectures. Skilled in transforming raw data into clean, structured formats for analytics and business intelligence. Proficient in modern data tools and technologies such as SQL, T-SQL, Python, Databricks, and cloud platforms (Azure). Adept at data wrangling, modeling, ETL/ELT development, and ensuring data quality, integrity, and security. Collaborative team player with a track record of enabling data-driven decision-making across business units. As a Dat...

KPI Partners is seeking a highly skilled and motivated Senior Data Engineer to join our dynamic team. As a Sr Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines and architectures. You will work closely with data scientists, analysts, and other stakeholders to ensure that our data solutions meet business requirements and drive strategic decision-making. Key Responsibilities: - Design, build, and maintain efficient and reliable data pipelines for processing large volumes of data. - Collaborate with cross-functional teams to understand data requirements and translate them into technical specifications. - Ensure data quality and integrity by...

Urgent Hiring: AWS Data Engineer, Senior Data Engineers & Lead Data Engineers Apply Now: Send your resume to heena.ruchwani@gspann.com Location: Bangalore (6+ Years Experience) Company: GSPANN Technologies, Inc. GSPANN Technologies is seeking talented professionals with 4+ years of experience to join our team in Bangalore. We are looking for immediate joiners who are passionate about data engineering and eager to take on exciting challenges. Key Skills & Experience: 6+ years of hands-on experience with AWS Data Services (Glue, Redshift, S3, Lambda, EMR, Athena, etc.) Strong expertise in Big Data Technologies (Spark, Hadoop, Kafka) Proficiency in SQL, Python, and Scala Hands-on experience wit...

Team Manager/ Technical Documentation/ Individual Contributor Location: Hyderabad Nanakramguda Work Mode: Hybrid (2 days office) Shift Timings: 2 PM IST 11 PM IST (Day Shift) Experience Required: 5 to 8 years About the Role We are looking for an experienced Technical Documentation Specialist (Individual Contributor) to join our Operations team in Hyderabad. The role involves creating and managing technical and business documentation related to data pipelines, data flow processes, and business review documents (BRDs). The ideal candidate should have a strong background in technical writing, a clear understanding of data pipelines and ETL processes, and the ability to convert complex data insi...

Collaborate with software engineers, business stake holders and/or domain experts to translate business requirements into product features, tools, projects. ? Develop, implement, and deploy ETL solutions. ? Preprocess and analyze large datasets to identify patterns, trends, and insights. ? Evaluate, validate, and optimize data models to ensure efficiency, and generalizability. ? Monitor and maintain the performance of data pipeline, data models in production environments, identifying opportunities for improvement and update as needed. ? Document development processes, results, and lessons learned to facilitate knowledge sharing and continuous improvement.

Job Description Senior Consultant Delivery ( Data Engineer) Architect and implement scalable, secure, and cost-efficient data pipelines leveraging GCP data products (BigQuery, Dataflow, Composer, Dataproc, Cloud Storage, etc,), Translate business requirements into technical designs for datamarts, data models, and analytics solutions, Oversee development standards, coding guidelines, and CI/CD practices to ensure code quality and maintainability, Drive adoption of data governance, lineage, and security policies across the engineering team, Collaborate closely with Product Owners, Data Architects, and stakeholders to align deliverables with business goals, Stay updated with the latest GCP adva...