20 Dagster Jobs

How is this team contributing to vision of Providence Ensure a continual and seamless service for customers regardless of which services they access. What will you be responsible for End-to-end development and quality of solutions and services in conversational & Agentic AI capability Understanding and Implementing advanced techniques in Agentic AI & Conversational AI Frameworks. Introduce best practices in AI Frameworks. What would your day look like Lead the design and development of Agentic AI Frameworks, models and systems, ensuring high performance and scalability. Good in Python, RestAPI and data structure skills Fine-tune and optimize pre-trained large language models (LLMs) such as GPT, LLAMA, Falcon, and Mistral. Implement and manage model quantization techniques (e.g., GPTQ, AWQ, 8-bit/4-bit models) to enhance model efficiency. Utilize Langchain, Dagster etc for orchestration and memory management in agentic applications. Conduct thorough model evaluations using metrics such as BLEU, ROUGE, and perplexity to ensure model quality and performance. Develop and maintain systems for intent recognition, entity extraction, and generative dialogue management. Stay updated with the latest advancements in NLP and AI technologies and apply them to ongoing projects. Mentor and guide junior engineers, fostering a culture of continuous learning and innovation. Who are we looking for Minimum 6 years of experience in Programming Languages & AI Platforms Strong understanding of deep learning fundamentals, including LSTM, RNN, ANN, POS tagging, and Transformer models. Strong understanding and proficiency in Agentic AI Frameworks. Proven experience in fine-tuning pre-trained LLMs (e.g., GPT, LLAMA, Falcon, Mistral) and Quantization. Knowledge of model quantization techniques (GPTQ, AWQ, 8-bit/4-bit models). Proficiency in using orchestration frameworks, state and memory management. Experience with model evaluation metrics such as BLEU, ROUGE, and perplexity. Understanding of intent recognition, entity extraction, and generative dialogue management systems. Excellent problem-solving skills and the ability to work in a fast-paced, collaborative environment. Strong communication skills and the ability to convey complex technical concepts to non-technical stakeholders. Good Understanding with Azure AI Foundary, Azure AI search, Indexing Knowledge in Kubernetes is good to have. Knowledge of Microsoft Event Hubs or Kafka Technologies Experience with load testing and unit testing tools. Proficient understanding source control and code versioning tools such as GIT, Azure DevOps etc. Familiarity with CICD and DevOps.

As an experienced Software Engineer specialized in Internal Tooling and Support, you will be responsible for strengthening the ingestion infrastructure and internal operations tooling at the intersection of Engineering and Operations. Your role will involve ensuring smooth data ingestion, reliable scraping infrastructure, and efficient tooling for Ops teams. Additionally, you will act as the first layer of triage between Ops and Engineering Oncall, driving faster resolutions and building long-term automation to reduce operational overhead. - **Maintain and optimize the ingestion and scraping infrastructure, ensuring high reliability and performance. - **Design, build, and improve internal tools and automation to streamline Ops workflows and reduce manual intervention. - **Act as a bridge between Operations and Engineering, owning first-level triage, debugging, and resolution of recurring issues. - **Partner with Ops to understand pain points and proactively deliver tooling/solutions that improve efficiency. - **Collaborate with Engineering teams to escalate and resolve critical issues, while driving knowledge sharing and documentation. - **Monitor system performance, identify bottlenecks, and implement solutions for scalability and resilience. Qualifications Required: - **5+ years of experience in software engineering, devops, or infrastructure-focused roles. - **Strong programming skills in Python, Go, or JavaScript(preferred) with a focus on tooling and automation. - **Hands-on experience with data ingestion pipelines, scraping frameworks, or similar infra-heavy systems. - **Solid understanding of debugging, logging, and monitoring practices (Grafana, Prometheus, Datadog, or similar). - **Experience building internal tools for non-technical stakeholders (Ops, Support, etc. ). - **Strong problem-solving mindset with the ability to quickly triage and resolve issues. - **Excellent communication skills to work effectively across Engineering and Ops. In addition, it is nice to have experience with web scraping frameworks (e.g., Scrapy, Puppeteer, Playwright), knowledge of cloud infrastructure (AWS, GCP, Azure) and containerization (Docker, Kubernetes), prior experience in a Support Engineering or Internal Tools role, and familiarity with data pipelines and workflow orchestration tools (Airflow, Prefect, Dagster, etc.). Please note that this job opportunity was posted by Roshan Muniraj from Loop AI.,

As an Infoscion, your primary role would be to lead the engagement effort of providing high quality and value-adding consulting solutions to customers at different stages from problem definition to diagnosis to solution design development and deployment. You will review the proposals prepared by consultants, provide guidance, and analyze the solutions defined for the client's business problems to identify any potential risks and issues. Additionally, you will identify change management requirements and propose a structured approach to the client for managing the change using multiple communication mechanisms. Your responsibilities also include coaching, creating a vision for the team, providing subject matter training for your focus areas, and motivating and inspiring team members through effective and timely feedback and recognition for high performance. You will be a key contributor in unit-level and organizational initiatives with an objective of providing high-quality, value-adding consulting solutions to customers adhering to the guidelines and processes of the organization. If you think you fit right in to help our clients navigate their next in their digital transformation journey, this is the place for you. - Good knowledge of software configuration management systems - Strong business acumen, strategy, and cross-industry thought leadership - Awareness of latest technologies and industry trends - Logical thinking and problem-solving skills along with an ability to collaborate - Two or three industry domain knowledge - Understanding of the financial processes for various types of projects and the various pricing models available - Client interfacing skills - Knowledge of SDLC and agile methodologies - Project and team management The company prefers candidates with technology expertise in Big Data - ALL.,

About Chargebee: Chargebee is a subscription billing and revenue management platform powering some of the fastest-growing brands around the world today, including Calendly, Hopin, Pret-a-Manger, Freshworks, Okta, Study.com and others. Thousands of SaaS and subscription-first businesses process over billions of dollars in revenue every year through the Chargebee platform. Headquartered in San Francisco, USA, our 500+ team members work remotely throughout the world, including India, the Netherlands, Paris, Spain, Australia, and the USA. Chargebee has raised over $480 million in capital and is funded by Accel, Tiger Global, Insight Partners, Steadview Capital, and Sapphire Ventures. And were on a mission to push the boundaries of subscription revenue operations. Not just ours, but every customer and prospective business on a recurring revenue model. Our team builds high-quality and innovative software to enable our customers to grow their revenues powered by the state-of-the-art subscription management platform. Key Roles & Responsibilities Productionise ML workflows : build and maintain data pipelines for feature generation, ML model training, batch scoring, and real?time inference using modern orchestration and container frameworks. Own model serving infrastructure : implement fast, reliable APIs / batch jobs; manage autoscaling, versioning, and rollback strategies Feature?store development : design and operate feature stores and corresponding data pipelines to guarantee trainingserving consistency. CI/CD & DevEx : automate testing, deployment, and monitoring of data and model artefacts; provide templated repos and documentation that let data scientists move from notebook to prod quickly. Observability & quality : instrument data?drift, concept?drift, and performance metrics; set up alerting dashboards to ensure model health. Collaboration & review : work closely with data scientists on model experimentation, production?harden their code, review PRs, and evangelise MLOps best practices across the organisation. Required Skills & Experience 3+ years as a ML / Data Engineer working on large-scale, data-intensive systems in cloud environments (AWS, GCP, or Azure), with proven experience partnering closely with ML teams to deploy models at scale. Proficient in Python plus one of Go / Java / Scala; strong software?engineering fundamentals (testing, design patterns, code review). Hands on experience in Spark and familiarity with streaming frameworks (Kafka, Flink, Spark Structured Streaming) Hands-on experience with workflow orchestrators (Airflow, Dagster, Kubeflow Pipelines, etc.) and container platforms (Docker + Kubernetes/EKS/ECS). Practical knowledge of ML algorithms like XGBoost, LightGBM, transformers and deep learning frameworks like pytorch is preferred Experience with experiment?tracking / ML model?management tools (MLflow, SageMaker, Vertex AI, Weights & Biases) is a plus Benefits: Want to know what it means to work for a company that genuinely cares about you Check out just a few of the benefits we give our employees: We are Globally Local With a diverse team across four continents, and customers in over 60 countries, you get to work closely with a global perspective right from your own neighborhood. We value Curiosity We believe the next great idea might just be around the corner. Perhaps its that random thought you had ten minutes ago. We believe in creating an ecosystem that fosters a desire to seek out hard questions, and then figure out answers to them. Customer! Customer! Customer! Everything we do is driven towards enabling our customers growth. This means no matter what you do, you will always be adding real value to a real business problem. Its a lot of responsibility, but also a lot of fun. If you resonate with Chargebee, have a monstrous appetite for curiosity, and an insatiable urge to learn and build new things, were waiting for you! We value people from all backgrounds and are dedicated to hiring and employing a diverse and inclusive workplace. Come be a part of the Chargebee tribe! Show more Show less

This is your chance to contribute to building the foundational backend systems for "Citi Assist", a Generative AI assistant that aims to reach every Citi employee globally. As a core builder on this high-visibility platform, you will play a key role in setting the standard for how a global bank adopts this technology safely and at scale. In this hands-on role, your primary responsibility will be to build, code, and deliver within our Python backend. Your focus will be on creating high-quality, tested code and efficiently moving features from concept to production. Taking ownership of the features you develop is essential. This involves not only writing the code but also determining the best path forward, asking pertinent questions, and seeing your work through to completion independently. Building with a security-first mindset is paramount. Ensuring the safety, reliability, and compliance of the systems you create will be a non-negotiable part of your craft. You will implement security best practices in your code, addressing technical hardening and considering the evolving challenges of AI safety. Collaboration is key to success in this role. As a valuable teammate, you will actively participate in our engineering culture by engaging in constructive code reviews, assisting your peers when needed, and sharing knowledge. The team's success is contingent on our collective efforts. Upholding our Developer Manifesto is essential. We prioritize engineering discipline, emphasizing clean code, meaningful commits, tidy branches, and rigorous CI/CD practices in our work. You will be responsible for adhering to these principles in every commit you make. Initiative and clear communication are highly valued. You will be expected to take ownership of problems, seek clarifications when needed, and drive your work forward with autonomy. Proactive and concise communication of your progress and challenges is crucial, ensuring that the team remains informed without needing to inquire about status updates. To excel in this role, you should be a pragmatic builder who takes pride in crafting durable software solutions. A solutions-first mindset, a bias for action, a sense of ownership and effective communication, a drive for technical excellence, and a collaborative spirit are all qualities we seek in potential candidates. Additionally, hands-on experience in building, deploying, and maintaining production-grade Python services, familiarity with security considerations, proficiency in software design patterns, and experience with CI/CD pipelines are highly desirable. If you are passionate about contributing to a team dedicated to building cutting-edge technology that positively impacts a vast audience, while fostering a culture of learning and collaboration, this opportunity may be the perfect fit for you.,

You will be responsible for managing and optimizing PLM and MES systems to enhance manufacturing processes. This includes handling large volumes of structured and unstructured data, identifying and overcoming technical difficulties and integration issues in previous projects, and implementing robust data security measures to protect sensitive information. Your role will also involve designing and maintaining data warehouse architecture and optimizing Power BI performance, focusing on storage mode and response time. In addition, you will need to have a good understanding of manufacturing workflows, production planning, and quality control. You will be expected to generate comprehensive reports based on previous project requirements and communicate effectively with business users to understand and meet their needs. Managing activities related to Google Cloud Platform (GCP) and Extract, Transform, Load (ETL) processes will also be part of your responsibilities. The ideal candidate should have familiarity with Google Cloud Platform (GCP) and e-commerce analytics, along with a minimum of 3 years of experience in data engineering, data analysis, or a similar role, preferably in e-commerce. Proven experience in managing data within manufacturing processes, strong knowledge of PLM and MES systems, and proficiency in data security implementation and data warehouse architecture are mandatory requirements. Excellent communication skills and the ability to work with business users are essential. You should possess strong knowledge in GCP and ETL processes, as well as proficiency in SQL and Python (or R) for data extraction, transformation, and analysis. Experience with Snowflake, DBT data modelling, Dagster, and ETL/ELT processes is preferred. Advanced proficiency in Power BI for creating interactive dashboards and reports is also necessary. Familiarity with GA4, Google Ads, Meta Ads, Meta S2S, and Braze Marketing APIs, as well as experience with cloud platforms and data integration tools, including Google BigQuery or AWS, will be advantageous. Experience with e-commerce analytics, such as funnel analysis, cohort analysis, and attribution modelling, is a plus. Qualifications required for this role include a Bachelor's degree in a relevant field (e.g., Computer Science, Engineering, Data Management). Please note that candidates from Tamilnadu location are preferred. Only shortlisted profiles will be contacted for further consideration.,

Line of Service Advisory Industry/Sector Not Applicable Specialism Data, Analytics & AI Management Level Senior Associate Job Description & Summary At PwC, our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics techniques to help clients optimise their operations and achieve their strategic goals. In business intelligence at PwC, you will focus on leveraging data and analytics to provide strategic insights and drive informed decision-making for clients. You will develop and implement innovative solutions to optimise business performance and enhance competitive advantage. *Why PWC At PwC, you will be part of a vibrant community of solvers that leads with trust and creates distinctive outcomes forour clients and communities. This purpose-led and values-driven work, powered by technology in an environment that drives innovation, will enable you to make a tangible impact in the real world. We reward your contributions, support your wellbeing, and offer inclusive benefits, flexibility programmes and mentorship that will help you thrive in work and life. Together, we grow, learn, care, collaborate, and create a future of infinite experiences foreach other. Learn more about us . At PwC, we believe in providing equal employment opportunities, without any discrimination on the grounds of gender, ethnic background, age, disability, marital status, sexual orientation, pregnancy, gender identity or expression, religion or other beliefs, perceived differences and status protected by law. We strive to create an environment where each one of our people can bring their true selves and contribute to their personal growth and the firms growth. To enable this, we have zero tolerance for any discrimination and harassment based on the above considerations. " Responsibilities: Design, develop, and maintain data transformation pipelines using dbt/IICS on Snowflake. Write optimized SQL and Python scripts for complex data modeling and processing tasks. Collaborate with data analysts, engineers, and business teams to implement scalable ELT workflows. Create and manage data models, schemas, and documentation in dbt. Optimize Snowflake performance using best practices (clustering, caching, virtual warehouses). Manage data integration from data lakes, external systems, and cloud sources. Ensure data quality, lineage, version control, and compliance across all environments. Participate in code reviews, testing, and deployment activities using CI/CD pipelines. Mandatory skill sets: Python and SQL, along with hands-on experience in Snowflake, dbt (Data Build Tool), IICS or any leading ETL tools, Preferred skill sets: Experience with Airflow, Dagster, or other orchestration tools. Knowledge of cloud platforms like AWS, Azure, or GCP. Exposure to BI tools like Power BI, Tableau, or Looker. Understanding Data Governance, Security, and Compliance. Experience in leading a development team Years of experience required: 7 10 yrs Education qualification: B.tech/MBA/MCA Education (if blank, degree and/or field of study not specified) Degrees/Field of Study required: Bachelor of Technology, Master of Business Administration Degrees/Field of Study preferred: Certifications (if blank, certifications not specified) Required Skills Microsoft Azure Optional Skills Accepting Feedback, Accepting Feedback, Active Listening, Analytical Thinking, Business Case Development, Business Data Analytics, Business Intelligence and Reporting Tools (BIRT), Business Intelligence Development Studio, Communication, Competitive Advantage, Continuous Process Improvement, Creativity, Data Analysis and Interpretation, Data Architecture, Database Management System (DBMS), Data Collection, Data Pipeline, Data Quality, Data Science, Data Visualization, Embracing Change, Emotional Regulation, Empathy, Inclusion, Industry Trend Analysis + 16 more Desired Languages (If blank, desired languages not specified) Travel Requirements Not Specified Available for Work Visa Sponsorship No Government Clearance Required No Job Posting End Date Show more Show less

Key Responsibilities : Plan, and execute the end-to-end migration of large-scale datasets and data pipelines from PostgreSQL to Snowflake, ensuring minimal downtime and data integrity. – Design, build, and optimize robust, scalable, and automated ETL/ELT data pipelines using Python and modern data engineering technologies. – Guide and mentor other data engineers, fostering a culture of technical excellence, collaboration, and knowledge sharing. Provide code reviews and architectural oversight. – Take full ownership of data engineering projects from conception through to deployment and ongoing maintenance. Be accountable for the quality, reliability, and timeliness of deliverables. – Work closely with the team, and business stakeholders to understand their data needs and deliver high-quality data solutions that drive business value. – Tackle complex data challenges, troubleshoot production issues, and implement performance optimizations within our data warehouse and pipeline infrastructure. Skills & Qualifications: – Bachelor's or Master's degree in Computer Science, Engineering, or a related quantitative field. – 5+ years of relevant professional experience in a data engineering role. – Proven, hands-on experience leading at least one significant, large-scale data migration from PostgreSQL to Snowflake. – Expert-level proficiency in Python for data processing and pipeline orchestration (e.g., using libraries like Pandas, SQLAlchemy, and frameworks like Airflow or Dagster). – Deep expertise in advanced SQL, data modeling, and data warehousing concepts. – Strong understanding of the Snowflake architecture, features, and best practices. – Familiarity with cloud services (AWS, GCP, or Azure) and their data-related offerings. – Excellent problem-solving skills, a meticulous attention to detail, and a proven ability to manage multiple projects with tight deadlines. – Strong communication and teamwork skills, with a collaborative mindset and a genuine willingness to help others succeed in a fast-paced, innovative environment. Other Key expectations: • Ideal candidate is expected to work on-site (Hyderabad or Gurugram location) 12 days per month or 3 days per week. • Ideal candidate should have strong communication and teamwork skills, with a collaborative spirit and a willingness to help others when needed.

As a member of the Infosys consulting team, your primary responsibility will be to lead engagement efforts in delivering high-quality consulting solutions to customers. Your role will involve guiding the team through problem definition, diagnosis, solution design, development, and deployment stages. You will review proposals, provide guidance, analyze solutions, and identify potential risks and issues for client business problems. Additionally, you will suggest change management strategies and communication mechanisms to facilitate smooth transitions. Your duties will include coaching team members, providing subject matter training, and fostering a vision for the team. Motivating and inspiring team members through timely feedback and recognition will be essential. You will play a crucial role in unit-level and organizational initiatives aimed at delivering top-notch consulting solutions while adhering to organizational guidelines and processes. If you are passionate about helping clients navigate their digital transformation journey, this opportunity is tailored for you. The ideal candidate should possess technical proficiency in Python, SQL, Dagster, and Databricks. Moreover, having a good understanding of software configuration management systems, business strategy, industry trends, and problem-solving skills is essential. Familiarity with industry domain knowledge, financial processes, pricing models, SDLC, agile methodologies, project management, and client interfacing skills is advantageous. Furthermore, preferred skills include expertise in Big Data technologies. Join us and contribute to our mission of providing valuable consulting solutions to clients while staying abreast of technological advancements and industry best practices.,

As the Lead Python Software Engineer at Citi, you will have the exciting opportunity to build the foundational backend systems for "Citi Assist", a Generative AI assistant that is set to reach every Citi employee globally. Joining at the ground floor, you will play a pivotal role in developing a high-visibility platform that establishes the standard for the safe and scalable adoption of this technology by a global bank. In this hands-on role, your primary responsibility will be to build, code, and deliver high-quality, tested Python backend features efficiently. Taking ownership of the features you develop, you will drive them from conception to completion, ensuring security and compliance are integral parts of your work. Collaboration and teamwork are essential as you contribute to our engineering culture through constructive code reviews, assisting peers, and sharing knowledge. Operating with a security-first mindset, you will implement best practices to address emerging challenges in AI safety. Proactive communication and autonomy are key aspects of this role, requiring you to take initiative, seek clarifications, and provide progress updates without prompting. Upholding engineering discipline, you will adhere to clean code practices, meaningful commits, and rigorous CI/CD processes in every aspect of your work. The ideal candidate for this role is a pragmatic builder who takes pride in crafting enduring software solutions. With a bias for action and a focus on delivering value, you will excel in a collaborative environment where technical excellence is valued. Your deep experience in building, deploying, and maintaining production-grade Python services, coupled with familiarity with security considerations and software design patterns, will be invaluable assets in this role. Furthermore, your proficiency in managing applications in containerized environments, experience with CI/CD pipelines, and expertise in relational databases like Postgres will be crucial for success in this position. If you have a solutions-first mindset, a drive for technical excellence, and a passion for collaborative problem-solving, we invite you to join our team at Citi and be part of our mission to build "Citi Assist" and enable the safe adoption of Generative AI across the organization.,

You have over 8 years of experience and are located in Balewadi, Pune. You possess a strong understanding of Data Architecture and have led data-driven projects. Your expertise includes knowledge of Data Modelling paradigms like Kimball, Inmon, Data Marts, Data Vault, Medallion, etc. Experience with Cloud Based data strategies, particularly AWS, is preferred. Designing data pipelines for ETL with expert knowledge on ingestion, transformation, and data quality is a must, along with hands-on experience in SQL. In-depth understanding of PostGreSQL development, query optimization, and designing indexes is a key requirement. Proficiency in Postgres PL/SQL for complex warehouse workflows is necessary. You should be able to manipulate intermediate to complex SQL and use advanced SQL concepts like RANK, DENSE_RANK, and apply advanced statistical concepts through SQL. Working experience with PostGres SQL extensions like PostGIS is desired. Expertise in writing ETL pipelines combining Python + SQL is required, as well as understanding of data manipulation libraries in Python like Pandas, Polars, DuckDB. Experience in designing Data visualization with tools such as Tableau and PowerBI is desirable. Your responsibilities include participation in designing and developing features in the existing Data Warehouse, providing leadership in establishing connections between Engineering, product, and analytics/data scientists team. Designing, implementing, and updating existing/new batch ETL pipelines, defining and implementing data architecture, and working with various data orchestration tools like Apache Airflow, Dagster, Prefect, and others. Collaboration with engineers and data analysts to build reliable datasets that can be trusted and used by the company is essential. You should be comfortable in a fast-paced start-up environment, passionate about your job, and enjoy a dynamic international working environment. Background or experience in the telecom industry is a plus, though not mandatory. You should have a penchant for automating tasks and enjoy monitoring processes.,

The ideal candidate should possess extensive expertise in SQL, data modeling, ETL/ELT pipeline development, and cloud-based data platforms like Databricks or Snowflake. You will be responsible for designing scalable data models, managing reliable data workflows, and ensuring the integrity and performance of critical financial datasets. Collaboration with engineering, analytics, product, and compliance teams is a key aspect of this role. Responsibilities: - Design, implement, and maintain logical and physical data models for transactional, analytical, and reporting systems. - Develop and oversee scalable ETL/ELT pipelines to process large volumes of financial transaction data. - Optimize SQL queries, stored procedures, and data transformations for enhanced performance. - Create and manage data orchestration workflows using tools like Airflow, Dagster, or Luigi. - Architect data lakes and warehouses utilizing platforms such as Databricks, Snowflake, BigQuery, or Redshift. - Ensure adherence to data governance, security, and compliance standards (e.g., PCI-DSS, GDPR). - Work closely with data engineers, analysts, and business stakeholders to comprehend data requirements and deliver solutions. - Conduct data profiling, validation, and quality assurance to maintain clean and consistent data. - Maintain comprehensive documentation for data models, pipelines, and architecture. Required Skills & Qualifications: - Proficiency in advanced SQL, including query tuning, indexing, and performance optimization. - Experience in developing ETL/ELT workflows with tools like Spark, dbt, Talend, or Informatica. - Familiarity with data orchestration frameworks such as Airflow, Dagster, Luigi, etc. - Hands-on experience with cloud-based data platforms like Databricks, Snowflake, or similar technologies. - Deep understanding of data warehousing principles like star/snowflake schema, slowly changing dimensions, etc. - Knowledge of cloud services (AWS, GCP, or Azure) and data security best practices. - Strong analytical and problem-solving skills in high-scale environments. Preferred Qualifications: - Exposure to real-time data pipelines like Kafka, Spark Streaming. - Knowledge of data mesh or data fabric architecture paradigms. - Certifications in Snowflake, Databricks, or relevant cloud platforms. - Familiarity with Python or Scala for data engineering tasks.,

Job Role: We are seeking a highly skilled and experienced Data Architect with expertise in designing and building data platforms in cloud environments. The ideal candidate will have a strong background in either AWS Data Engineering or Azure Data Engineering, along with proficiency in distributed data processing systems like Spark. Additionally, proficiency in SQL, data modeling, building data warehouses, and knowledge of ingestion tools and data governance are essential for this role. TheData Architect will also need experience with orchestration tools such as Airflow or Dagster and proficiency in Python, with knowledge of Pandas being beneficial. ? Why Choose Ideas2IT Ideas2IT has all the good attributes of a product startup and a services company. Since we launch our products, you will have ample opportunities to learn and contribute. However, single-product companies stagnate in the technologies they use. In our multiple product initiatives and customer-facing projects, you will have the opportunity to work on various technologies. AGI is going to change the world. Big companies like Microsoft are betting heavily on this (seehereandhere). We are following suit. ? Whats in it for you? You will get to work on impactful products instead of back-office applications for the likes of customers like Facebook, Siemens, Roche, and more You will get to work on interesting projects like the Cloud AI platform for personalized cancer treatment Opportunity to continuously learn newer technologies Freedom to bring your ideas to the table and make a difference, instead of being a small cog in a big wheel Showcase your talent in Shark Tanks and Hackathons conducted in the company ? ?Heres what youll bring? Experience in designing and building data platforms in any cloud. Strong expertise in either AWS Data Engineering or Azure Data Engineering Develop and optimize data processing pipelines using distributed systems like Spark. Create and maintain data models to support efficient storage and retrieval. Build and optimize data warehouses for analytical and reporting purposes, utilizing technologies such as Postgres, Redshift, Snowflake, etc. Knowledge of ingestion tools such as Apache Kafka, Apache Nifi, AWS Glue, or Azure DataFactory. Establish and enforce data governance policies and procedures to ensure data quality and security. Utilize orchestration tools like Airflow or Dagster to schedule and manage data workflows. Develop scripts and applications in Python to automate tasks and processes. Collaborate with stakeholders to gather requirements and translate them into technical specifications. Communicate technical solutions effectively to clients and stakeholders. Familiarity with multiple cloud ecosystems such as AWS, Azure, and Google Cloud Platform(GCP). Experience with containerization and orchestration technologies like Docker and Kubernetes. Knowledge of machine learning and data science concepts. Experience with data visualization tools such as Tableau or Power BI. Understanding of DevOps principles and practices. Show more Show less

As a Senior Python Engineer at our company, you will leverage your deep expertise in data engineering and API development to drive technical excellence and autonomy. Your primary responsibility will be leading the development of scalable backend systems and data infrastructure that power AI-driven applications across our platform. You will design, develop, and maintain high-performance APIs and microservices using Python frameworks such as FastAPI and Flask. Additionally, you will build and optimize scalable data pipelines, ETL/ELT processes, and orchestration frameworks, ensuring the utilization of AI development tools like GitHub Copilot, Cursor, or CodeWhisperer to enhance engineering velocity and code quality. In this role, you will architect resilient and modular backend systems integrated with databases like PostgreSQL, MongoDB, and Elasticsearch. Managing workflows and event-driven architectures using tools such as Airflow, Dagster, or Temporal.io will be essential, as you collaborate with cross-functional teams to deliver production-grade systems in cloud environments (AWS/GCP/Azure) with high test coverage, observability, and reliability. To be successful in this position, you must have at least 5 years of hands-on experience in Python backend/API development, a strong background in data engineering, and proficiency in AI-enhanced development environments like Copilot, Cursor, or equivalent tools. Solid experience with Elasticsearch, PostgreSQL, and scalable data solutions, along with familiarity with Docker, CI/CD, and cloud-native deployment practices is crucial. You should also demonstrate the ability to take ownership of features from idea to production. Nice-to-have qualifications include experience with distributed workflow engines like Temporal.io, background in AI/ML systems (PyTorch or TensorFlow), familiarity with LangChain, LLMs, and vector search tools (e.g., FAISS, Pinecone), and exposure to weak supervision, semantic search, or agentic AI workflows. Join us to build infrastructure for cutting-edge AI products and work in a collaborative, high-caliber engineering environment.,

You should have a Bachelor's degree in Computer Science, Information Technology, or a related field. A strong understanding of operating systems, networking basics, and Linux command-line usage is essential. Proficiency in at least one scripting language such as Python or Bash is required. Basic knowledge of cloud computing concepts, with a preference for AWS, is expected. Familiarity with DevOps principles like CI/CD, automation, and cloud infrastructure management is a plus. Awareness of version control systems like Git is necessary. It would be beneficial to have exposure to cloud platforms, preferably AWS, and infrastructure services such as EC2, S3, RDS, and Kubernetes. Understanding of Infrastructure as Code concepts and knowledge of Terraform would be advantageous. Basic knowledge of CI/CD tools like GitLab or AzureDevOps is a plus. Awareness of monitoring concepts and observability tools like New Relic and Grafana is beneficial. Basic knowledge about containerization, automation, or data/ML infra tools such as Docker, Ray, Dagster, Weights & Biases is an advantage. Exposure to scripting tasks for automation and ops workflows using Python is desired. Joining Sanas will allow you to gain real-world experience in managing cloud infrastructure, including AWS and Azure, as well as COLO datacenter. You will work on infrastructure automation using Terraform and Python, CI/CD pipeline development and management with GitLab and Spinnaker, and observability and monitoring with tools like New Relic, Grafana, and custom alerting mechanisms. You will also have the opportunity to work with cutting-edge tools in ML/AI infrastructure like Ray, Dagster, W&B, and data analytics tools such as ClickHouse and Aurora PostgreSQL. Additionally, you will learn about agile delivery models and collaborate with Engineering, Science, InfoSec, and ML teams. We offer hands-on experience with modern DevOps practices and enterprise cloud architecture, mentorship from experienced DevOps engineers, exposure to a scalable infrastructure supporting production-grade AI and ML workloads, and an opportunity to contribute to automation, reliability, and security for systems. You will participate in occasional on-call rotations to maintain system availability in a collaborative and fast-paced learning environment where your work directly supports engineering and innovation.,

Position Overview We are seeking a skilled Data Engineer with 2-4 years of experience to design, build, and maintain scalable data pipelines and infrastructure. You will work with modern data technologies to enable data-driven decision making across the organisation. Key Responsibilities Design and implement ETL/ELT pipelines using Apache Spark and orchestration tools (Airflow/Dagster). Build and optimize data models on Snowflake and cloud platforms. Collaborate with analytics teams to deliver reliable data for reporting and ML initiatives. Monitor pipeline performance, troubleshoot data quality issues, and implement testing frameworks. Contribute to data architecture decisions and work with cross-functional teams to deliver quality data solutions. Required Skills & Experience 2-4 years of experience in data engineering or related field Strong proficiency with Snowflake including data modeling, performance optimisation, and cost management Hands-on experience building data pipelines with Apache Spark (PySpark) Experience with workflow orchestration tools (Airflow, Dagster, or similar) Proficiency with dbt for data transformation, modeling, and testing Proficiency in Python and SQL for data processing and analysis Experience with cloud platforms (AWS, Azure, or GCP) and their data services Understanding of data warehouse concepts, dimensional modeling, and data lake architectures Preferred Qualifications Experience with infrastructure as code tools (Terraform, CloudFormation) Knowledge of streaming technologies (Kafka, Kinesis, Pub/Sub) Familiarity with containerisation (Docker, Kubernetes) Experience with data quality frameworks and monitoring tools Understanding of CI/CD practices for data pipelines Knowledge of data catalog and governance tools Advanced dbt features including macros, packages, and documentation Experience with table format technologies (Apache Iceberg, Apache Hudi) Technical Environment Data Warehouse: Snowflake Processing: Apache Spark, Python, SQL Orchestration: Airflow/Dagster Transformation: dbt Cloud: AWS/Azure/GCP Version Control: Git Monitoring: DataDog, Grafana, or similar

Your role: As a passionate technical development lead specializing in the GRC space, you will play a crucial role in delivering efficient configuration and customization of large GRC installations. Your responsibilities will include providing technical leadership, working on strategic programs, and collaborating with a high-performance team. You should possess a strong analytical and technical ability with a minimum of 5 years of experience in Cloud, DevOps, UI frameworks, Java, REST, and DB skills. The ideal candidate will be able to work independently, communicate effectively, and have a proven track record of working on large complex development projects. You will leverage your skills and knowledge to develop high-quality solutions that meet business needs while driving continuous integration and improvements. Additionally, you will collaborate with the Senior Tech Lead & Solution Architect to provide valuable inputs on application design. Your team: You will be part of the Compliance & Operational Risk IT team, a global team responsible for designing and implementing innovative IT solutions to track complex regulatory requirements in the financial services industry. The team is spread across various locations including the US, UK, Switzerland, and India, providing support to internal clients worldwide. Your expertise: - CI/CD pipeline creation and deployment into production (incl. GitOps practices) enabling Canary / rolling deployment, Blue/Green, Feature Flags Observability of deployments, chaos engineering - Relational DBs (SQL / PostgresSQL) - Data Flow and ETL (e.g. Airflow/Dagster similar) - JVM based languages (Java8/11+, Scala, Kotlin) - Knowledge on M7 preferred (eve About Company: Purview is a leading Digital Cloud & Data Engineering company with headquarters in Edinburgh, United Kingdom and a presence in 14 countries including India, Poland, Germany, Finland, Netherlands, Ireland, USA, UAE, Oman, Singapore, Hong Kong, Malaysia, and Australia. The company has a strong presence in the UK, Europe, and APEC regions, providing services to Captive Clients and top IT tier 1 organizations. Company Info: India Office: 3rd Floor, Sonthalia Mind Space Near Westin Hotel, Gafoor Nagar Hitechcity, Hyderabad Phone: +91 40 48549120 / +91 8790177967 UK Office: Gyleview House, 3 Redheughs Rigg, South Gyle, Edinburgh, EH12 9DQ. Phone: +44 7590230910 Email: careers@purviewservices.com Login to Apply!,

Role & responsibilities (Exp is required 6+ Years) Job Description: Enterprise Business Technology is on a mission to support and create enterprise software for our organization. We're a highly collaborative team that interlocks with corporate functions such as Finance and Product teams to deliver value with innovative technology solutions. Each day, thousands of people rely on Enlyte's technology and services to help their customers during challenging life events. We're looking for a remote Senior Data Analytics Engineer for our Corporate Analytics team. Opportunity - Technical lead for our corporate analytics practice using dbt, Dagster, Snowflake and Power BI, SQL and Python Responsibilities Build our data pipelines for our data warehouse in Python working with APIs to source data Build power bi reports and dashboards associated to this process Contribute to our strategy for new data pipelines and data engineering approaches Maintain a medallion based architecture for data analysis with Kimball Participates in daily scrum calls, follows agile SDLC Creates meaningful documentation of their work Follow organizational best practices for dbt and writes maintainable code Qualifications 5+ years of professional experience as a Data Engineer Strong dbt experience (3+ years) and knowledge of modern data stack Strong experience with Snowflake (3+ years) You have experience using Dagster and running complex pipelines (1+ year) Some Python experience, experience with git and Azure Devops Experience with data modeling in Kimball and medallion based structures

Principal Architect - Platform & Application Architect Experience 15+ years in software/data platform architecture 5+ years in architectural leadership roles Architecture & Data Platform Expertise Education Bachelors/Master’s in CS, Engineering, or related field Title: Principal Architect Location: Onsite Bangalore Experience: 15+ years in software & data platform architecture and technology strategy Role Overview We are seeking a Platform & Application Architect to lead the design and implementation of a next-generation, multi-domain data platform and its ecosystem of applications. In this strategic and hands-on role, you will define the overall architecture, select and evolve the technology stack, and establish best practices for governance, scalability, and performance. Your responsibilities will span across the full data lifecycle—ingestion, processing, storage, and analytics—while ensuring the platform is adaptable to diverse and evolving customer needs. This role requires close collaboration with product and business teams to translate strategy into actionable, high-impact platform & products. Key Responsibilities 1. Architecture & Strategy Design the end-to-end architecture for a On-prem / hybrid data platform (data lake/lakehouse, data warehouse, streaming, and analytics components). Define and document data blueprints, data domain models, and architectural standards. Lead build vs. buy evaluations for platform components and recommend best-fit tools and technologies. 2. Data Ingestion & Processing Architect batch and real-time ingestion pipelines using tools like Kafka, Apache NiFi, Flink, or Airbyte. Oversee scalable ETL/ELT processes and orchestrators (Airflow, dbt, Dagster). Support diverse data sources: IoT, operational databases, APIs, flat files, unstructured data. 3. Storage & Modeling Define strategies for data storage and partitioning (data lakes, warehouses, Delta Lake, Iceberg, or Hudi). Develop efficient data strategies for both OLAP and OLTP workloads. Guide schema evolution, data versioning, and performance tuning. 4. Governance, Security, and Compliance Establish data governance , cataloging , and lineage tracking frameworks. Implement access controls , encryption , and audit trails to ensure compliance with DPDPA, GDPR, HIPAA, etc. Promote standardization and best practices across business units. 5. Platform Engineering & DevOps Collaborate with infrastructure and DevOps teams to define CI/CD , monitoring , and DataOps pipelines. Ensure observability, reliability, and cost efficiency of the platform. Define SLAs, capacity planning, and disaster recovery plans. 6. Collaboration & Mentorship Work closely with data engineers, scientists, analysts, and product owners to align platform capabilities with business goals. Mentor teams on architecture principles, technology choices, and operational excellence. Skills & Qualifications Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field. 12+ years of experience in software engineering, including 5+ years in architectural leadership roles. Proven expertise in designing and scaling distributed systems, microservices, APIs, and event-driven architectures using Java, Python, or Node.js. Strong hands-on experience with building scalable data platforms on premise/Hybrid/cloud environments. Deep knowledge of modern data lake and warehouse technologies (e.g., Snowflake, BigQuery, Redshift) and table formats like Delta Lake or Iceberg. Familiarity with data mesh, data fabric, and lakehouse paradigms. Strong understanding of system reliability, observability, DevSecOps practices, and platform engineering principles. Demonstrated success in leading large-scale architectural initiatives across enterprise-grade or consumer-facing platforms. Excellent communication, documentation, and presentation skills, with the ability to simplify complex concepts and influence at executive levels. Certifications such as TOGAF or AWS Solutions Architect (Professional) and experience in regulated domains (e.g., finance, healthcare, aviation) are desirable.

About us: We are building a modern, scalable, fully automated on-premise data platform , designed to handle complex data workflows, including data ingestion, ETL processes, physics-based calculations and machine learning predictions. Orchestrated using Dagster , our platform integrates with multiple data sources, edge devices, and storage systems. A core principle of our architecture is self-service : granting data scientists, analysts, and engineers granular control over the entire journey of their data assets as well empowering teams to modify and extend their data pipelines with minimal friction. We're looking for a hands-on Data Engineer to help develop, maintain, and optimize this platform. Role & responsibilities: - Design, develop, and maintain robust data pipelines using Dagster for orchestration - Build and manage ETL pipelines with python and SQL - Optimize performance and reliability of the platform within on-premise infrastructure constraints - Develop solutions for processing and aggregating data on edge devices , including data filtering, compression, and secure transmission - Maintain metadata, data lineage, ensure data quality, consistency, and compliance with governance and security policies - Implement CI/CD workflows of the platform on a local Kubernetes cluster - Architect the platform with a self-service mindset , including clear abstractions, reusable components, and documentation - Develop in collaboration with data scientists, analysts, and frontend developers to understand evolving data needs - Define and maintain clear contracts/interfaces with source systems , ensuring resilience to upstream changes Preferred candidate profile: -5-7 years of experience in database-driven projects or related fields. -1-2 years of experience with data platforms, orchestration, and big data management. -Proven experience as a Data Engineer or similar role, with focus on backend data processing and infrastructure -Hands-on experience with Dagster or similar data orchestration tools (e.g., Airflow, Prefect, Luigi, Databricks) - Proficiency with SQL and Python - Strong understanding of data modeling , ETL/ELT best practices, and batch/stream processing - Familiarity with on-premises deployments and challenges (e.g., network latency, storage constraints, resource management) - Experience with version control (Git) and CI/CD practices for data workflows - Understanding of data governance , access control , and data cataloging