787 Etl Pipelines Jobs - Page 23

Role & responsibilities Design, implement and operate ETL workflows (Airflow, AWS Glue). Administer PostgreSQL, MySQL or SQL Server: performance tuning, backups, restores. Manage schema migrations via Flyway/Liquibase and version control. Provision and maintain data infrastructure with Terraform & Ansible. Automate scaling, snapshots and cost controls for RDS clusters. Secure data environment with IAM, network policies and encryption. Participate in 247 on-call rotations; author runbooks and postmortems. Collaborate on data modeling, indexing strategies and query optimization. Document all data flows, tuning guides, and runbooks in Confluence. Preferred candidate profile B.Tech/BE in CS, Inf...

Key Responsibilities Design conformed star & snowflake schemas , implement SCD2 dimensions and fact tables. Lead Spark (PySpark/Scala) or AWSGlue ELT pipelines from RDSZeroETL/S3 into Redshift. Tune RA3 clusterssort/dist keys, WLM queues, Spectrum partitionsfor subsecond BI queries. Establish dataquality, lineage, and costgovernance dashboards using CloudWatch & Terraform/CDK. Collaborate with Product & Analytics to translate HR KPIs into selfservice data marts. Mentor junior engineers; drive documentation and coding standards. MustHave Skills AmazonRedshift (sort & dist keys, RA3, Spectrum) Spark on EMR/Glue (PySpark or Scala) Dimensional modelling (Kimball), star schema, SCD2 Advanced SQL ...

As a seasoned IT professional with over 15 years of progressive experience, you will be responsible for providing strategic and technical leadership in key areas such as Application & Data Modernization, AI, and BI technologies. Your role will involve defining and developing new service offerings that are aligned with emerging technologies and market demands. You will lead high-performing teams to deliver enterprise-grade software products using modern tech stacks such as Java, Spring, Python, Flutter, Angular, React, Vue, microservices, Kafka, Azure, AWS, and GCP. You will also be tasked with scaling and managing application development and data engineering practices across different geogra...

> Strong programming expertise in PySpark and Python. > Solid understanding of Spark internals, DAG optimization, partitioning, broadcast joins, etc. > Hands-on experience with one or more cloud platforms > Experience with API integrations Required Candidate profile The ideal candidate has strong expertise in PySpark optimization, API integration, and big data ingestion using AWS, GCP, or Azure. A solid foundation in SQL.

As a Senior Software Engineer Java at SAP, you will be a key member of the App2App Integration team within SAP Business Data Cloud. Your primary focus will be to accelerate the integration of SAP's application ecosystem with its unified data fabric, facilitating low-latency, secure, and scalable data exchange processes. Your responsibilities will include designing and developing core integration frameworks using Java, RESTful APIs, and messaging frameworks like Apache Kafka. You will also be tasked with building and maintaining scalable data processing and ETL pipelines that support real-time and batch data flows. Additionally, you will work on integrating data engineering workflows with too...

As a Data Engineer for our data-rich e-commerce platform catering to the life sciences sector, your primary responsibility will be to support infrastructure, develop data pipelines, and deploy pricing logic. You will play a crucial role in ensuring the usability and interface design of internal tools that facilitate experimentation, pricing configuration, and real-time monitoring. Your key responsibilities will include: - Building and maintaining ETL pipelines for pricing, shipping, and behavioral datasets - Collaborating with data scientists and product managers to facilitate model development and experimentation - Developing APIs or backend logic to implement dynamic pricing algorithms - C...

BambooBox is a SaaS-based software start-up with 4+ years of expertise, offering a platform for growth marketers. With its advanced AI and ML-driven platform, BambooBox helps companies deliver untapped marketing intelligence to business teams and increase the marketing contribution to the overall revenue. The company collaborates with clients like Airtel Business, Darwinbox, Acalvio, and other global organizations. BambooBox is backed by investors such as Peak XV (earlier Sequoia Surge), Emergent Ventures, and Arc180. Currently, BambooBox is in the process of scaling the organization and is in a high-growth phase. Located in Bangalore, BambooBox is seeking a talented developer to join its te...

You are a talented SAP HANA Developer sought after by RiskInsight Consulting Pvt Ltd, a company specializing in providing innovative risk management solutions to clients across finance and insurance sectors. Your role in this growing team involves designing and implementing sophisticated SAP HANA applications to enhance business value and efficiency. As a key member of the team, you will be responsible for developing and maintaining data models in SAP HANA, designing, coding, and testing applications, optimizing SQL queries for high performance, and collaborating with business analysts to translate requirements into technical specifications. Additionally, you will implement data integration ...

Company Overview Incedo is a US-based consulting, data science and technology services firm with over 3000 people helping clients from our six offices across US, Mexico and India. We help our clients achieve competitive advantage through end-to-end digital transformation. Our uniqueness lies in bringing together strong engineering, data science, and design capabilities coupled with deep domain understanding. We combine services and products to maximize business impact for our clients in telecom, Banking, Wealth Management, product engineering and life science & healthcare industries. Working at Incedo will provide you an opportunity to work with industry leading client organizations, deep te...

Role : Enterprise Architect (Data) Location : Bangalore Role overview We are seeking an experienced Enterprise Data Architect to join our team, focusing primarily on Data engineering, Data Analytics/Visualization and cloud engineering. The ideal candidate will play a crucial role in shaping our technology landscape, participating in project delivery, and contributing to presales activities during lean periods. 14 to 20 years of total experience in IT, with a minimum of 3 years in an Enterprise Data Architect capacity. Strong expertise in data and cloud technologies, with hands-on experience in data architecture, cloud migrations, and modern data platforms. Knowledge of design patterns and ar...

We are looking for a Senior Data Engineer with deep experience in SnapLogic, SQL, ETL pipelines, and data warehousing, along with hands-on experience with Databricks.in designing scalable data solutions and working across cloud and big data .

We are seeking experienced Data Analysts / Data Engineers with strong expertise in U.S. pharmaceutical commercial datasets to support critical Data Operations initiatives. This role will be focused on onboarding third-party data, ensuring data quality , and implementing outlier detection techniques . Familiarity with ML/A I approaches for anomaly detection is highly desirable. Key Responsibilities: Pharma Data Integration: Work extensively with U.S. pharmaceutical commercial datasets. Ingest and onboard third-party data sources such as IQVIA, Symphony Health, Komodo Health etc. Ensure alignment of data schemas, dictionary mapping, and metadata integrity. Data Quality & Governance: Design and...

Responsibilities: * Design, develop & maintain data pipelines using Python, AWS & SQL. * Collaborate with cross-functional teams on ETL projects. * Optimize performance & scalability of data systems.

Handling of petabytes of data and experience required in design and delivery of distributed system in cloud by using server less architecture & in Data lakes and data sourcing platforms and in python and pyspark immediate joiners are prefer.

As an Experienced Senior Data Engineer at Adobe, you will utilize Big Data and Google Cloud technologies to develop large-scale, on-cloud data processing pipelines and data warehouses. Your role will involve consulting with customers worldwide on their data engineering needs around Adobe's Customer Data Platform and supporting pre-sales discussions regarding complex and large-scale cloud data engineering solutions. You will design custom solutions on cloud by integrating Adobe's solutions in a scalable and performant manner. Additionally, you will deliver complex, large-scale, enterprise-grade on-cloud data engineering and integration solutions in a hands-on manner. To be successful in this ...

We are seeking an experienced Project Manager specializing in Data & Analytics to oversee and supervise data-driven projects spanning various business domains. The successful candidate will possess significant expertise in managing projects related to data engineering, business intelligence, and analytics, demonstrating a track record of collaborating effectively with technical and business stakeholders to deliver impactful solutions within specified timeframes and budgets. Key Responsibilities: - Lead the end-to-end project management of data and analytics initiatives. - Define project scope, objectives, deliverables, schedules, and resource allocations. - Collaborate closely with data engi...

As a Data Analyst with deep experience in the US Healthcare industry and skills around Machine Learning (ML), you will play a vital role in conducting comprehensive data analysis using statistical and exploratory methods to uncover patterns and insights that drive data-driven decision-making in the healthcare domain. Leveraging your knowledge of healthcare industry metrics such as HEDIS, CMS Star Ratings, risk adjustment models, and revenue cycle data, you will optimize analytics strategies to ensure efficient data quality, availability, and reliability for AI/ML-driven healthcare analytics solutions. Your responsibilities will include designing and maintaining data pipelines for the ingesti...

As a Senior Consultant - Data Engineer at AstraZeneca, you will be an integral part of the R&D IT team, a driving force behind technological advancement in areas such as Drug Research and Development. With our global presence spanning across multiple countries, including North America, the UK, Sweden, India, and Mexico, we are united by our passion for innovation. The Data & Analytics team, specializing in cutting-edge fields like Data Engineering, Generative AI, and Data Science, is at the heart of our mission. Join us to not only keep up with the latest technological trends but to set them. In this role, you will work collaboratively in a dynamic, globally distributed team to design and im...

HCLTech is seeking a Data and AI Principal / Senior Manager (Generative AI) for their Noida location. As a global technology company with a workforce of over 218,000 employees in 59 countries, HCLTech specializes in digital, engineering, cloud, and AI solutions. The company collaborates with clients across various industries such as Financial Services, Manufacturing, Life Sciences, Healthcare, Technology, Telecom, Retail, and Public Services, offering innovative technology services and products. With consolidated revenues of $13.7 billion as of the 12 months ending September 2024, HCLTech aims to drive progress and transformation for its clients globally. Key Responsibilities: In this role, ...

The ideal candidate for this position in Ahmedabad should be a graduate with at least 3 years of experience. At Bytes Technolab, we strive to create a cutting-edge workplace infrastructure that empowers our employees and clients. Our focus on utilizing the latest technologies enables our development team to deliver high-quality software solutions for a variety of businesses. You will be responsible for leveraging your 3+ years of experience in Machine Learning and Artificial Intelligence to contribute to our projects. Proficiency in Python programming and relevant libraries such as NumPy, Pandas, and scikit-learn is essential. Hands-on experience with frameworks like PyTorch, TensorFlow, Ker...

About CLOUDKEEPER: CloudKeeper is a cloud cost optimization partner that combines the power of group buying & commitments management, expert cloud consulting & support, and an enhanced visibility & analytics platform to reduce cloud cost & help businesses maximize the value from AWS, Microsoft Azure, & Google Cloud. A certified AWS Premier Partner, Azure Technology Consulting Partner, Google,Cloud Partner, and FinOps Foundation Premier Member, CloudKeeper has helped 400+ global companies save an average of 20% on their cloud bills, modernize their cloud set-up and maximize value all while maintaining flexibility and avoiding any long-term commitments or cost. CloudKeeper hived off from TO TH...

HCLTech is looking for a Data and AI Principal / Senior Manager (Generative AI) to join their team in Noida. As a global technology company with a strong presence in 59 countries and over 218,000 employees, HCLTech is a leader in digital, engineering, cloud, and AI services. They collaborate with clients in various industries such as Financial Services, Manufacturing, Life Sciences, Healthcare, Technology, Telecom, Media, Retail, and Public Services. With consolidated revenues of $13.7 billion, HCLTech aims to provide industry-leading capabilities to drive progress for their clients. In this role, you will be responsible for providing hands-on technical leadership and oversight. This include...

As a Senior Consultant - Data Engineer at AstraZeneca, you will be an integral part of our R&D IT team, a driving force of technological advancement in the field of Drug Research, Development, and beyond. Our global presence across North America, the UK, Sweden, India, and Mexico unites us in our passion for innovation. Join our Data & Analytics team, specializing in cutting-edge fields like Data Engineering, Generative AI, and Data Science, to not just keep up with the newest technological trends but to set them. You will play a crucial role in revolutionizing drug discovery by harnessing the power of remarkable technology and advanced AI capabilities. Collaborate with our dynamic, globally...

The Healthcare Analytics Specialist position based in Gurugram requires 3 to 5 years of experience. You will be responsible for developing, maintaining, and optimizing data pipelines, including ingestion, transformation, and loading of internal and external sources. Collaboration with Operations is essential to design scalable, secure, and high-performing data workflows. Best practices in data governance, version control, security, and documentation must be implemented. Analytical models for cost, quality, and utilization metrics will be built and maintained using tools like Python, R, or SQL-based BI tools. Reports will be developed to communicate findings to stakeholders across the organiz...

Job Title Data Engineer Job Description Job Duties and Responsibilities: We are looking for a self-starter to join our Data Engineering team. You will work in a fast-paced environment where you will get an opportunity to build and contribute to the full lifecycle development and maintenance of the data engineering platform. With the Data Engineering team you will get an opportunity to - Design and implement data engineering solutions that is scalable, reliable and secure on the Cloud environment Understand and translate business needs into data engineering solutions Build large scale data pipelines that can handle big data sets using distributed data processing techniques that supports the e...