8 Pipeline Optimization Jobs

As a Principal Engineer/Architect at our company, you will play a crucial role in spearheading the development and expansion of our Data & AI practice. This senior leadership position requires a blend of technical proficiency, strategic acumen, and client-oriented leadership. You will be responsible for leading intricate data migration and transformation initiatives, establishing best practices and reusable frameworks, and influencing decision-making processes at both Fortune 500 companies and innovative startups. Your expertise will be utilized across modern cloud technologies such as AWS, Azure, GCP, Databricks, and advanced Data & AI tools. Your primary responsibilities will include designing and architecting scalable, high-performance data solutions on cloud platforms, leading enterprise-scale Data & AI projects from inception to delivery, crafting data architecture blueprints and reusable frameworks, as well as optimizing data pipelines for enhanced efficiency. Additionally, you will be instrumental in building and nurturing a high-performing Data & AI team by recruiting, mentoring, and training skilled professionals, developing standardized processes and best practices, and driving internal product development initiatives to create reusable tools. In your role, you will act as a trusted advisor to clients, offering strategic guidance and technical expertise, collaborating with sales and pre-sales teams to propose solutions for complex data projects, and representing the company at industry forums and technical conferences to showcase thought leadership and innovation. Moreover, you will provide thought leadership in data governance, data integration, and cloud data transformations, stay abreast of emerging trends and technologies in Data & AI, and foster a culture of continuous improvement and innovation within the team. To excel in this position, you should possess strong expertise in at least one cloud platform (AWS, Azure, or GCP), demonstrate the ability to design and implement high-performing data architectures, have experience in data engineering, data governance, and pipeline optimization, and be proficient in tools like Databricks and Apache Spark for large-scale data processing. Your track record should include successful data platform migrations and the development of reusable frameworks for large-scale deployments. Additionally, you should have a background in Enterprise Information Management programs or large cloud data migration initiatives, excellent client communication skills, and hands-on experience in MLOps, observability frameworks, and cloud security best practices. Success in this role will be defined by your ability to design and deliver scalable data solutions, build and lead a high-impact team, drive standardization and reusability through frameworks and tools, establish yourself as a trusted advisor and thought leader in the Data & AI domain, and contribute to revenue growth through collaboration on pre-sales and solutioning activities. Joining our team will offer you competitive compensation, performance bonuses, a 401k plan with employer matching, comprehensive health and wellness benefits, professional development and certification assistance, a flexible work environment with remote/hybrid options, exposure to industry-leading Data & AI projects, leadership and career growth opportunities, and the chance to work with cutting-edge technologies and innovative teams.,

Key Responsibilities: Design, implement, and operate scalable data pipelines for business intelligence and machine learning. Develop and maintain core data frameworks and key infrastructures. Create and support ETL processes to ingest data from various sources into our data warehouse. Design and model data warehouses for efficient and cost-effective reporting. Collaborate with data analysts and scientists to optimize data warehouse structures for reporting and insights. Improve software development processes and team productivity continuously. Define and enforce data governance processes, including data discovery, lineage, access control, and quality assurance. Perform code reviews and ensure data quality across all pipelines and processes. Qualifications: Experience: 35 years overall, with at least 2+ years in data engineering/infrastructure. Technology Stack: Big Data Technologies: Hive, Spark, PySpark (Batch & Streaming), Airflow, Redshift, Delta Lake Cloud: AWS (Strong architectural experience preferred) Data Tools: Databricks (strong plus), BI tools (Metabase, Power BI – a plus) DevOps knowledge is preferable but not mandatory Strong understanding of data warehousing and data ecosystem concepts Proven experience building and maintaining scalable data pipelines Experience with APIs for business application data ingestion Familiarity with data governance and quality assurance best practices Experience in product-based companies or startups is highly preferred

Job description Description: The Senior Data Engineer is responsible for leading the design, development, and optimization of data infrastructure and systems to meet complex business needs. They play a strategic role in developing and managing large-scale data pipelines, ensuring scalability, reliability, and performance. As a key technical expert, the Senior Data Engineer collaborates with cross-functional teams to deliver innovative data solutions and ensures adherence to best practices in data management and governance. Responsibilities Leads the design and development of advanced data pipelines, ensuring seamless ETL/ELT of data into enterprise storage systems. Oversees data integration from diverse sources, maintaining data consistency, integrity, and compliance with enterprise standards. Transforms raw data into usable formats through cleansing, aggregation, filtering, and enrichment techniques. Optimizes data pipelines and workflows for scalability, performance, and cost efficiency. Establishes robust data validation, quality checks, and error-handling mechanisms to ensure enterprise data integrity. Mentors junior data engineers, promoting technical growth and team development. Implements and enforces best practices in data governance, including metadata management and lineage tracking. Drives the adoption of real-time data streaming and processing workflows to enable advanced analytics and decision-making. Partners with stakeholders to identify and address complex data challenges, delivering scalable and innovative solutions. Stays updated with emerging data technologies, evaluating their potential to enhance enterprise capabilities. Develops and enforces data security protocols to protect sensitive information. Job Requirements Education A bachelor s degree in computer science, data science, software engineering, information systems, or related quantitative field; master s degree preferred Experience 8+ years of data engineering experience, including expertise in data integration, pipeline optimization, and enterprise-scale data solutions. Proven leadership in implementing big data solutions (e.g., Snowflake, Databricks) and distributed data systems (e.g., Apache Spark, Flink). Skills Advanced proficiency in Apache technologies (Kafka, Airflow, Spark) and programming languages (Python, Java, Scala). Expertise in data query tools (SQL, Hive) and database technologies (NoSQL, Hadoop, Teradata). Strong knowledge of cloud platforms (AWS, Azure, GCP) and modern data architectures. Experience with real-time streaming data workflows and AI/ML analytics initiatives. Exceptional analytical and problem-solving skills with debugging expertise in complex systems. Excellent communication skills to effectively collaborate with technical and non-technical stakeholders. Strong leadership abilities to mentor teams and influence cross-functional initiatives.

Job description Description: The Senior Data Engineer is responsible for leading the design, development, and optimization of data infrastructure and systems to meet complex business needs. They play a strategic role in developing and managing large-scale data pipelines, ensuring scalability, reliability, and performance. As a key technical expert, the Senior Data Engineer collaborates with cross-functional teams to deliver innovative data solutions and ensures adherence to best practices in data management and governance. Responsibilities Leads the design and development of advanced data pipelines, ensuring seamless ETL/ELT of data into enterprise storage systems. Oversees data integration from diverse sources, maintaining data consistency, integrity, and compliance with enterprise standards. Transforms raw data into usable formats through cleansing, aggregation, filtering, and enrichment techniques. Optimizes data pipelines and workflows for scalability, performance, and cost efficiency. Establishes robust data validation, quality checks, and error-handling mechanisms to ensure enterprise data integrity. Mentors junior data engineers, promoting technical growth and team development. Implements and enforces best practices in data governance, including metadata management and lineage tracking. Drives the adoption of real-time data streaming and processing workflows to enable advanced analytics and decision-making. Partners with stakeholders to identify and address complex data challenges, delivering scalable and innovative solutions. Stays updated with emerging data technologies, evaluating their potential to enhance enterprise capabilities. Develops and enforces data security protocols to protect sensitive information. Job Requirements Education A bachelor s degree in computer science, data science, software engineering, information systems, or related quantitative field; master s degree preferred Experience 8+ years of data engineering experience, including expertise in data integration, pipeline optimization, and enterprise-scale data solutions. Proven leadership in implementing big data solutions (e.g., Snowflake, Databricks) and distributed data systems (e.g., Apache Spark, Flink). Skills Advanced proficiency in Apache technologies (Kafka, Airflow, Spark) and programming languages (Python, Java, Scala). Expertise in data query tools (SQL, Hive) and database technologies (NoSQL, Hadoop, Teradata). Strong knowledge of cloud platforms (AWS, Azure, GCP) and modern data architectures. Experience with real-time streaming data workflows and AI/ML analytics initiatives. Exceptional analytical and problem-solving skills with debugging expertise in complex systems. Excellent communication skills to effectively collaborate with technical and non-technical stakeholders. Strong leadership abilities to mentor teams and influence cross-functional initiatives.

Job description Description: The Senior Data Engineer is responsible for leading the design, development, and optimization of data infrastructure and systems to meet complex business needs. They play a strategic role in developing and managing large-scale data pipelines, ensuring scalability, reliability, and performance. As a key technical expert, the Senior Data Engineer collaborates with cross-functional teams to deliver innovative data solutions and ensures adherence to best practices in data management and governance. Responsibilities Leads the design and development of advanced data pipelines, ensuring seamless ETL/ELT of data into enterprise storage systems. Oversees data integration from diverse sources, maintaining data consistency, integrity, and compliance with enterprise standards. Transforms raw data into usable formats through cleansing, aggregation, filtering, and enrichment techniques. Optimizes data pipelines and workflows for scalability, performance, and cost efficiency. Establishes robust data validation, quality checks, and error-handling mechanisms to ensure enterprise data integrity. Mentors junior data engineers, promoting technical growth and team development. Implements and enforces best practices in data governance, including metadata management and lineage tracking. Drives the adoption of real-time data streaming and processing workflows to enable advanced analytics and decision-making. Partners with stakeholders to identify and address complex data challenges, delivering scalable and innovative solutions. Stays updated with emerging data technologies, evaluating their potential to enhance enterprise capabilities. Develops and enforces data security protocols to protect sensitive information. Job Requirements Education A bachelor s degree in computer science, data science, software engineering, information systems, or related quantitative field; master s degree preferred Experience 8+ years of data engineering experience, including expertise in data integration, pipeline optimization, and enterprise-scale data solutions. Proven leadership in implementing big data solutions (e.g., Snowflake, Databricks) and distributed data systems (e.g., Apache Spark, Flink). Skills Advanced proficiency in Apache technologies (Kafka, Airflow, Spark) and programming languages (Python, Java, Scala). Expertise in data query tools (SQL, Hive) and database technologies (NoSQL, Hadoop, Teradata). Strong knowledge of cloud platforms (AWS, Azure, GCP) and modern data architectures. Experience with real-time streaming data workflows and AI/ML analytics initiatives. Exceptional analytical and problem-solving skills with debugging expertise in complex systems. Excellent communication skills to effectively collaborate with technical and non-technical stakeholders. Strong leadership abilities to mentor teams and influence cross-functional initiatives.

You are a high-performing and experienced Sales & Marketing Specialist, ready to take charge, nurture relationships with enterprise clients, and secure high-value contracts. Your motivation stems from achieving results, working with cutting-edge products, and aiming for top-tier incentives. This opportunity promises a dynamic environment where you can excel. Your responsibilities include leading strategic outreach to enterprise clients, managing the full sales cycle from lead generation to closing contracts, identifying new business opportunities in SaaS and software services, creating tailored proposals, and building consultative relationships with key decision-makers. You will leverage market intelligence to refine strategies, analyze sales data for optimization, collaborate with marketing for impactful campaigns, and oversee a high-performing team. What makes this role unique is the uncapped performance-based bonuses, competitive commission structure, exclusive rewards for top performers, equity options, and the chance to sell in fast-growing verticals like headless eCommerce and custom software development. You will collaborate with a team delivering enterprise solutions, benefit from ongoing sales training, certifications, and leadership development, and enjoy a hybrid work setup with in-person collaboration at the Noida office. To qualify for this role, you should have at least 3 years of experience in B2B/B2C sales, a track record of closing high-ticket deals, knowledge of eCommerce, SaaS, or software development services, strong communication and negotiation skills, proficiency in CRM platforms and sales automation tools, motivation for target-driven environments, and the ability to adapt to evolving client needs and tech trends. If you are ready to lead deals, dominate pipelines, and close high-value contracts in a challenging yet rewarding environment, this opportunity awaits you.,

The role of Technical Artist involves developing and implementing high-quality content in a team environment to support the delivery of games at the highest quality. As a creative professional with development experience, the Technical Artist is expected to become a subject-matter expert in their discipline by contributing to the success of various commercially released products within the gaming industry. The Technical Artist is responsible for mastering technical issues such as rendering, performance, streaming, and memory management to create visually appealing yet optimized assets. They maintain art and animation production workflows by ensuring smooth workflows across different departments and implementing tools to streamline tasks. The Technical Artist also learns custom toolsets and engine constraints specific to the company to provide efficient solutions to common problems. Problem-solving and innovation are key aspects of the role, as the Technical Artist proactively tackles art-related technological challenges and collaborates with the engineering team to develop creative solutions. Additionally, they contribute to proof-of-concept presentations for game features, support the art team with tools and assistance, and maintain the After Effects export pipeline. The Technical Artist provides desktop support and troubleshooting to ensure artists can work without technical interruptions. They modify artwork to meet technical requirements, optimize art data production processes, and maintain organized project data. Conducting asset quality assurance, participating in team critiques, and enforcing file structure and naming conventions are also part of the Technical Artist's responsibilities. Essential skills for this role include technical art expertise, problem-solving, strong collaboration skills, tool development experience, pipeline optimization, and attention to detail. Candidates should have degree or diploma qualifications in animation or a related discipline and at least 5 years of experience in a professional animation/technical art role. Aristocrat, a world leader in gaming content and technology, offers a robust benefits package, global career opportunities, and a diverse work environment where individual differences are valued. The company values responsible gameplay, governance, employee wellbeing, and sustainability, and encourages applications from individuals of all backgrounds. Please note that travel is not expected for this position. Candidates must be authorized to work in the job posting location without the need for current or future visa sponsorship.,

We are looking for a talented Talend Developer with hands-on experience in Talend Management Console on Cloud and Snowflake to join our growing team. The ideal candidate will play a key role in building and optimizing ETL/ELT data pipelines, integrating complex data systems, and ensuring high performance across cloud environments. While experience with Informatica is a plus, it is not mandatory for this role. As a Talend Developer, you will be responsible for designing, developing, and maintaining data integration solutions to meet the organizations growing data needs. You will collaborate with business stakeholders, data architects, and other data professionals to ensure the seamless and secure movement of data across platforms, ensuring scalability and performance. Key Responsibilities: Develop and maintain ETL/ELT data pipelines using Talend Management Console on Cloud to integrate data from various on-premises and cloud-based sources. Design, implement, and optimize data flows for data ingestion, processing, and transformation in Snowflake to support analytical and reporting needs. Utilize Talend Management Console on Cloud to manage, deploy, and monitor data integration jobs, ensuring robust pipeline management and process automation. Collaborate with data architects to ensure that the data integration solutions align with business requirements and follow best practices. Ensure data quality, performance, and scalability of Talend-based data solutions. Troubleshoot, debug, and optimize existing ETL processes to ensure smooth and efficient data integration. Document data integration processes, including design specifications, mappings, workflows, and performance optimizations. Collaborate with the Snowflake team to implement best practices for data warehousing and data transformation. Implement error-handling and data validation processes to ensure high levels of accuracy and data integrity. Provide ongoing support for Talend jobs, including post-deployment monitoring, troubleshooting, and optimization. Participate in code reviews and collaborate in an agile development environment. Required Qualifications: 2+ years of experience in Talend development, with a focus on using the Talend Management Console on Cloud for managing and deploying jobs. Strong hands-on experience with Snowflake data warehouse, including data integration and transformation. Expertise in developing ETL/ELT workflows for data ingestion, processing, and transformation. Experience with SQL and working with relational databases to extract and manipulate data. Experience working in cloud environments (e.g., AWS, Azure, or GCP) with integration of cloud-based data platforms. Strong knowledge of data integration, data quality, and performance optimization in Talend. Ability to troubleshoot and resolve issues in data integration jobs and processes. Solid understanding of data modeling concepts and best practices for building scalable data pipelines. Preferred Qualifications: Experience with Informatica is a plus but not mandatory. Experience with scripting languages such as Python or Shell scripting for automation. Familiarity with CI/CD pipelines and working in DevOps environments for continuous integration of Talend jobs. Knowledge of data governance and data security practices in cloud environments. Role & responsibilities Preferred candidate profile