74 Apache Beam Jobs - Page 2

You will have the opportunity to work at Capgemini, a company that empowers you to shape your career according to your preferences. You will be part of a collaborative community of colleagues worldwide, where you can reimagine what is achievable and contribute to unlocking the value of technology for leading organizations to build a more sustainable and inclusive world. Your Role: - You should have a very good understanding of current work, tools, and technologies being used. - Comprehensive knowledge and clarity on Bigquery, ETL, GCS, Airflow/Composer, SQL, Python are required. - Experience with Fact and Dimension tables, SCD is necessary. - Minimum 3 years of experience in GCP Data Enginee...

Dear Candidate, Greetings of the day!! My name is Arumugam Veera, and I'm reaching out to you regarding an exciting opportunity with TechMango. You can connect with me on below platform. LinkedIn https://www.linkedin.com/in/arumugamv/ Techmango Technology Services is a full-scale software development services company founded in 2014 with a strong focus on emerging technologies. It holds a primary objective of delivering strategic solutions towards the goal of its business partners in terms of technology. We are a full-scale leading Software and Mobile App Development Company. Techmango is driven by the mantra Clients Vision is our Mission. We have a tendency to stick on to the current statem...

Data Engineer (GCP) Company: Moptra Infotech Pvt. Ltd. Location: Noida, Sector 63 Website: https://moptra.com/ Work Schedule: 5 days working (Saturday & Sunday fixed off) About Moptra Infotech Moptra Infotech Pvt. Ltd. is a fast-growing IT services and consulting company headquartered in Noida, India. We specialize in delivering data engineering, cloud solutions, business intelligence, and advanced analytics services to global enterprises. With a strong focus on innovation, scalability, and client satisfaction , Moptra partners with leading organizations across industries to build reliable, future-ready digital ecosystems. Job Responsibilities Develop and maintain robust data pipelines and E...

The GCP Data Engineer will be responsible for designing, developing, and maintaining data pipelines and data infrastructure on Google Cloud Platform (GCP). This role requires expertise in data engineering best practices, cloud architecture, and big data technologies. The ideal candidate will work closely with data scientists, analysts, and other stakeholders to ensure the availability, reliability, and efficiency of data systems, enabling data-driven decision-making across the organization. Key Responsibilities Data Pipeline Development Design, develop, and maintain scalable and efficient ETL/ELT pipelines on GCP. Implement data ingestion processes from various data sources (e.g., APIs, data...

This role is for a GCP Data Engineer who can build cloud analytics platforms to meet expanding business requirements with speed and quality using lean Agile practices. You will work on analysing and manipulating large datasets supporting the enterprise by activating data assets to support Enabling Platforms and Analytics in the GCP. You will be responsible for designing the transformation and modernization on GCP. Experience with large scale solutions and operationalizing of data warehouses, data lakes and analytics platforms on Google Cloud Platform or other cloud environment is a must. We are looking for candidates who have a broad set of technology skills across these areas and who can de...

As a Technical Specialist - Data Engineer at Pune, India, you will be responsible for designing, developing, and delivering significant components of engineering solutions to achieve business goals. Your key responsibilities include actively participating in the design and development of new application features, ensuring solutions are maintainable and integrated successfully, and assisting junior team members as needed. You will develop and deploy source code, provide development activities for projects and technical infrastructure components, debug and provide support to the L3 team, and contribute to quality assurance through unit testing. Additionally, you will ensure architectural chang...

The company StatusNeo is a global consulting firm that specializes in cutting-edge AI, automation, and cloud-first digital solutions. With a focus on product & platform engineering, the team at StatusNeo includes top-notch software engineers and product designers who are experts in their respective fields. Located in Gurugram, StatusNeo is a Great Place To Work certified organization that values innovation and collaboration. As a GCP Data Engineer at StatusNeo, your primary responsibility will be designing, developing, and maintaining scalable data pipelines and architectures on Google Cloud Platform. Working alongside data scientists, analysts, and other stakeholders, you will ensure that d...

Dear Candidate, Greetings of the day!! My name is Arumugam Veera, and I'm reaching out to you regarding an exciting opportunity with TechMango. You can connect with me on below platform. LinkedIn https://www.linkedin.com/in/arumugamv/ Mobile Not available on calls & WhatsApp Only: (+91) [+91 6369 002 769] Email: arumugam.veera@techmango.net Techmango Technology Services is a full-scale software development services company founded in 2014 with a strong focus on emerging technologies. It holds a primary objective of delivering strategic solutions towards the goal of its business partners in terms of technology. We are a full-scale leading Software and Mobile App Development Company. Techmango...

You have more than 7 years of IT experience and have expertise in working with at least two structural databases such as SQL, Oracle, or Postgres, and one NoSQL database. You are capable of collaborating with the Presales team to propose optimal solutions and architectures. Additionally, you possess design experience with BQ, Redshift, and Synapse. Your responsibilities include managing the entire product life cycle, from proposal to delivery, and continuously evaluating architecture enhancements with the delivery team. You are well-versed in security protocols for in-transit data, as well as encryption and decryption of PII data. Moreover, you have a strong understanding of analytics tools ...

The Senior GCP Data Engineer role at Deutsche Bank in Bangalore, India, offers you the opportunity to be part of an ambitious initiative focused on Sustainable Finance, ESG Risk Mitigation, and Corporate Sustainability. As Climate Change presents new challenges and opportunities, the bank is investing in developing a Sustainability Technology Platform, Sustainability data products, and sustainability applications to support its goals. To contribute to this initiative, we are seeking a highly motivated and experienced Senior GCP Data Engineer who is passionate about Climate Change and eager to leverage their Technology Skillset in Cloud/Hybrid Architecture. In this role, you will be responsib...

Role & responsibilities (Apache Beam) . Develop and manage orchestration workflows using Cloud Composer (Airflow) . Ingest, transform, and process large-scale data with a focus on performance, scalability, and compliance. Collaborate with business analysts and healthcare SMEs to understand workflows and translate them into data solutions. Optimize data pipelines for cost efficiency, performance, and scalability. Ensure data quality, lineage, and governance across claims datasets. Integrate structured and unstructured data sources into data lakes/warehouses . Implement data security and HIPAA compliance standards in all processes. Build reusable frameworks for ETL/ELT processes in a regulated...

As a GCP Data Engineer-Technical Lead at Birlasoft Office in Bengaluru, India, you will be responsible for designing, building, and maintaining scalable data pipelines and platforms on Google Cloud Platform (GCP) to support business intelligence, analytics, and machine learning initiatives. With a primary focus on Python and GCP technologies such as BigQuery, Dataproc, and Data Flow, you will develop ETL and ELT pipelines while ensuring optimal data manipulation and performance tuning. Your role will involve leveraging data manipulation libraries like Pandas, NumPy, and PySpark, along with SQL expertise for efficient data processing in BigQuery. Additionally, your experience with tools such ...

As a Data Engineer at our company, you will be responsible for designing scalable and robust AI/ML systems in production, focusing on high-performance and cost-effective solutions. Your expertise in various technologies, including GCP services like BigQuery, Cloud Dataflow, Pub/Sub, Dataproc, and Cloud Storage, along with programming languages such as Python, Java/Scala, and SQL, will be crucial for the success of our projects. Additionally, your experience with data processing tools like Apache Beam, Apache Kafka, and Cloud Dataprep, as well as orchestration tools like Apache Airflow and Terraform, will play a significant role in implementing efficient data pipelines. Knowledge of security ...

You should have at least 7-10 years of industry experience, with a minimum of 5 years specifically in machine learning roles. Your proficiency should be advanced in Python and common ML libraries such as TensorFlow, PyTorch, and Scikit-learn. Experience in distributed training, model optimization (quantization, pruning), and inference at scale is essential. It is expected that you have hands-on experience with cloud ML platforms like AWS (SageMaker), GCP (Vertex AI), or Azure ML. Additionally, familiarity with MLOps tooling such as MLflow, TFX, Airflow, or Kubeflow, as well as data engineering frameworks like Spark, dbt, or Apache Beam is required. You should possess a strong understanding o...

You should have at least 10 years of overall development experience with at least 5 years in a Data Engineering role. Your responsibilities will include building and optimizing big data pipelines, architectures, and data sets. You should have a strong background in writing SQL statements and experience in Spring/Spring Boot framework. Additionally, you should have experience in relational databases like Postgres, Oracle, SnowFlake, BigQuery, and other cloud databases. Experience in implementing web services such as SOAP and RESTful web services is required. Knowledge of frontend frameworks like Angular, jQuery, and Bootstrap is also expected. You should be familiar with real-time and batch d...

As a GCP DBT Manager, your primary responsibility will be to collaborate with the team in designing, building, and maintaining data pipelines and transformations using Google Cloud Platform (GCP) and the Data Build Tool (dbt). This role will involve utilizing tools such as BigQuery, Cloud Composer, and Python, requiring a strong foundation in SQL skills and knowledge of data warehousing concepts. Additionally, you will play a crucial role in ensuring data quality, optimizing performance, and working closely with cross-functional teams. Your key responsibilities will include: Data Pipeline Development: - Designing, building, and maintaining ETL/ELT pipelines using dbt and GCP services like Bi...

About Us: Fission Labs is a leading software development company headquartered in Sunnyvale, with offices in Dallas and Hyderabad. We specialize in crafting flexible, agile, and scalable solutions that drive businesses forward. Our comprehensive services include product development, cloud engineering, big data analytics, QA, DevOps consulting, and AI/ML solutions, empowering clients to achieve sustainable digital transformation aligned with their business goals. Fission Labs Website: https://www.fissionlabs.com/ Work Location: Hyderabad Notice Period: Immediate to 30 Days Role Overview: Omada is dedicated to developing next-gen intelligent systems that seamlessly integrate real-time APIs, cl...

As a member of the Platform Observability Engineering team within Ford's Data Platforms and Engineering (DP&E) organization, you will contribute to building and maintaining a top-tier platform for monitoring and observability. This platform focuses on the four golden signalslatency, traffic, errors, and saturationproviding essential data to support operations, root cause analysis, continuous improvement, and cost optimization. You will collaborate with platform architects to help design, develop, and maintain a scalable and reliable platform, ensuring smooth integration with systems used across various teams. Your contributions will be key in improving MTTR and MTTX through increased visibil...

You should possess a minimum of 7-10 years of industry experience, out of which a minimum of 5 years should have been in machine learning roles. Your proficiency in Python and popular ML libraries such as TensorFlow, PyTorch, and Scikit-learn should be advanced. Furthermore, you should have hands-on experience in distributed training, model optimization including quantization and pruning, and inference at scale. Experience with cloud ML platforms like AWS (SageMaker), GCP (Vertex AI), or Azure ML is essential. It is expected that you are familiar with MLOps tooling such as MLflow, TFX, Airflow, or Kubeflow, and data engineering frameworks like Spark, dbt, or Apache Beam. A solid understandin...

As a Data Engineer at Synoptek, you will be responsible for designing, developing, and maintaining robust and scalable data pipelines on the Google Cloud Platform (GCP). You will leverage your hands-on experience with GCP services such as BigQuery, Jitterbit, Cloud Dataflow, Cloud Pub/Sub, and Cloud Storage to build efficient data processing solutions. Collaborating with cross-functional teams, you will translate their data needs into technical requirements, ensuring data quality, integrity, and security throughout the data lifecycle. Your role will involve developing and optimizing ETL/ELT processes to extract, transform, and load data from various sources into data warehouses and data lake...

As a Data Pipeline Architect at our company, you will be responsible for designing, developing, and maintaining optimal data pipeline architecture. You will monitor incidents, perform root cause analysis, and implement appropriate actions to ensure smooth operations. Additionally, you will troubleshoot issues related to abnormal job execution and data corruption, and automate jobs, notifications, and reports for efficiency. Your role will also involve optimizing existing queries, reverse engineering for data research and analysis, and calculating the impact of issues on downstream processes for effective communication. You will support failures, address data quality issues, and ensure the ov...

As the Architect IT Strategy, VP at Deutsche Bank Group based in Pune, India, your role involves designing applications, defining modernization approaches, developing application code, and implementing technical solutions to address business problems and meet specified requirements and design principles. Under our flexible scheme, you will enjoy benefits such as a best-in-class leave policy, gender-neutral parental leaves, childcare assistance benefit reimbursement, flexible working arrangements, sponsorship for industry-relevant certifications and education, employee assistance program, comprehensive hospitalization and life insurance, and health screening for individuals aged 35 years and ...

As a GCP Data Engineer in Australia, you will be responsible for leveraging your experience in Google Cloud Platform (GCP) to handle various aspects of data engineering. Your role will involve working on data migration projects from legacy systems such as SQL and Oracle. You will also be designing and building ETL pipelines for data lake and data warehouse solutions on GCP. In this position, your expertise in GCP data and analytics services will be crucial. You will work with tools like Cloud Dataflow, Cloud Dataprep, Apache Beam/Cloud composer, Cloud BigQuery, Cloud Fusion, Cloud PubSub, Cloud storage, and Cloud Functions. Additionally, you will utilize Cloud Native GCP CLI/gsutil for opera...

The ideal candidate for the position will have the responsibility of designing, developing, and maintaining an optimal data pipeline architecture. You will be required to monitor incidents, perform root cause analysis, and implement appropriate actions to solve issues related to abnormal job execution and data corruption conditions. Additionally, you will automate jobs, notifications, and reports to improve efficiency. You should possess the ability to optimize existing queries, reverse engineer for data research and analysis, and calculate the impact of issues on the downstream side for effective communication. Supporting failures, data quality issues, and ensuring environment health will a...

As a Founding Engineer at Colrows, you will play a crucial role in shaping and scaling our generative analytics platform. We are dedicated to revolutionizing how enterprises engage with data, aiming to make insights as accessible as conversing with a colleague. Our platform integrates robust data foundations, agentic workflows, and enterprise-grade governance to deliver a seamless, self-serve experience. Join us in driving the development of a powerful execution engine that lies at the core of our platform. In this role, you will serve as a hands-on Principal Software Engineer, leveraging your architectural expertise and practical experience in constructing resilient systems using Java and o...