8 Streamprocessing Systems Jobs

As a Senior Data Engineer at SR Analytics in Pune, you will be responsible for designing, constructing, and maintaining scalable data management systems on Azure Cloud. Your role will involve developing and optimizing data pipelines and architectures using Azure Data Factory, Azure Databricks, and Azure Synapse Analytics to ensure that systems meet business requirements and industry best practices. You will integrate new Azure-based technologies and software engineering tools into existing data structures, develop processes for data mining, modeling, and production with Azure SQL Database, Cosmos DB, among others. In addition, you will be required to build and maintain custom software components and analytics applications on Azure, leveraging a variety of languages and tools such as Python, SQL, and Azure Functions. Collaboration with cross-functional teams, including data architects, modelers, and engineers, to align on long-term goals for the data platform, will be an essential part of your role. You will also optimize and recommend ways to improve data reliability, efficiency, and quality, conducting research to address complex industry and business problems using Azure-based solutions. Providing strategic input on the overall data platform architecture using Azure Data Lake Storage and Azure Blob Storage will also be expected from you. To qualify for this position, you should hold a Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field, with a minimum of 4 years of experience as a Data Engineer, Software Developer, or in a similar role. Strong hands-on experience with Azure Cloud technologies like Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Data Lake is mandatory. Proficiency in SQL, Python, Java, or other scripting languages, along with experience in relational databases and NoSQL databases on Azure, is required. Strong analytical and problem-solving skills, familiarity with DevOps practices, including CI/CD pipelines in Azure DevOps, and excellent communication skills to explain technical concepts to non-technical stakeholders are also necessary. Having experience with machine learning on Azure, knowledge of Power BI for data visualization, or possessing certifications in Azure Data Engineer or Azure Solutions Architect will be considered an added advantage. This opportunity is for individuals who are passionate about Azure Cloud technologies and are ready to take on the challenge in a fast-paced environment. If you are excited about this role and meet the requirements, we encourage you to apply now to join our team at SR Analytics.,

You should have a total experience of 9+ years and be located in Thane. It is essential to have a minimum of 6 years of experience in designing and architecting large-scale data solutions as a Software Architect. You should be proficient in designing, architecting, and creating specifications for solutions on any of the following IOT Stacks such as Azure IOT, AWS Greengrass / AWS IOT Core, or any other IOT Stack. Additionally, you must have experience and understanding of unit test frameworks like NUnit, XUnit, and API Testing, along with Cloud Native Microservices development and CI/CD pipelines. Hands-on experience with Dot Net Core is required, as well as exposure to and conceptual knowledge of Docker, MQTT / AMQP. Demonstrated skill in using coding standards and actively participating in code and design reviews is also necessary. You should be able to technically communicate and articulate Solution Design and Architecture, with a good working knowledge of Agile software development methodology. Nice-to-have skills include experience working with ReactJS, Angular, or other NodeJS frontend frameworks, along with working knowledge of Python. Experience in managing and working with teams based in multiple geographies is preferred. Prior experience working with messaging systems, stream-processing systems, event-hubs, IoT-hubs, stateless services & applications is an advantage. Understanding of security principles & best practices with knowledge of encryption & encoding technologies, securing cloud services & environments, creating secure network designs, and using common security tools & technologies (i.e., SSO, oAuth2, JWT, Identity Management, etc.) is beneficial. Experience with issue tracking tools (JIRA, Azure DevOps, etc.) and understanding of AI capabilities on the edge would be a plus.,

You are a seasoned Software Architect with a minimum of 10-18 years of experience, based in Thane West. You will be working 100% from the office with no hybrid or work from home option. Your role will require you to have experience in working with ReactJS, Angular, or other NodeJS frontend frameworks, as well as a working knowledge of Python. It would be beneficial if you have previously managed and collaborated with teams across different geographies. Additionally, prior exposure to messaging systems, stream-processing systems, event-hubs, IoT-hubs, stateless services & applications, and security principles including encryption & encoding technologies, securing cloud services & environments, creating secure network designs, and using common security tools & technologies such as SSO, oAuth2, JWT, Identity Management, etc., is desired. Familiarity with issue tracking tools like JIRA, Azure DevOps, and understanding AI capabilities on the edge would be advantageous. In terms of technical skills, you should have a minimum of 6 years of experience in designing and architecting large-scale data solutions. You should be proficient in designing, architecting, and creating specifications for solutions using any of the following IOT Stacks (Azure IOT, AWS Greengrass / AWS IOT Core or any other IOT Stack). Experience with unit test frameworks like NUnit, XUnit, Api Testing, Cloud Native Microservices development, CI/CD pipelines, Dot Net Core, Docker, MQTT / AMQP, and demonstrated skill in using coding standards and actively participating in code and design reviews are essential. Effective communication skills are crucial as you will be required to articulate Solution Design and Architecture effectively. A good working knowledge of Agile software development methodology is also expected. Your educational background should include a B.Tech/B.E. in Any Specialization. Please note that this role is based in Thane West and requires 100% in-office presence with no remote working options available.,

A career in the Advisory Acceleration Centre at PwC is an opportunity to leverage PwC's global delivery capabilities and provide premium, cost-effective, and high-quality services to support client engagements. As an Azure Senior Developer at PwC - AC, you will collaborate with the Offshore Manager and Onsite Business Analyst to grasp the project requirements and take charge of the complete implementation of Cloud data engineering solutions. Your role will involve utilizing your expertise in Azure cloud services such as Storage services (Blob, ADLS, PaaS SQL), Azure Data Factory, and Azure Synapse. You should excel in planning, organization, and have the ability to lead as a cloud developer in an agile team, delivering automated cloud solutions. With 4-8 years of hands-on experience, you must be proficient in Azure cloud computing, including big data technologies. Your deep understanding of traditional and modern data architecture and processing concepts will be critical, covering relational databases, data warehousing, big data, NoSQL, and business analytics. Your experience with Azure ADLS, Data Bricks, Data Flows, HDInsight, and Azure Analysis services will be essential, along with building stream-processing systems using solutions like Storm or Spark-Streaming. Designing and implementing scalable ETL/ELT pipelines using Databricks and Apache Spark, optimizing data workflows, and understanding big data use-cases and design patterns are key requirements. Your role will also involve architecture, design, implementation, and support of complex application architectures, as well as hands-on experience in implementing Big Data solutions using Microsoft Data Platform and Azure Data Services. Knowledge of Azure SQL DB, Azure Synapse Analytics, Azure HD Insight, Azure Data Lake Storage, Azure Data Lake Analytics, Azure Machine Learning, Stream Analytics, Azure Data Factory, Azure CosmosDB, and Power BI is essential. Exposure to Open-Source technologies like Apache Spark, Hadoop, NoSQL, Kafka, Solr/Elastic Search, as well as expertise in quality processes, design strategies leveraging Azure and Databricks, and Application DevOps tools will be advantageous. Desired skills include experience in stream-processing systems, Big Data ML toolkits, Python, and AWS Architecture certification. Having worked in Offshore/Onsite Engagements, experience in AWS services like STEP & Lambda, good project management skills, and knowledge in Cloud technologies like AWS, GCP, Informatica-Cloud, Oracle-Cloud, Cloud DW - Snowflake & DBT are also beneficial. The ideal candidate should have a professional background in BE/B.Tech/MCA/M.Sc/M.E/M.Tech/MBA, possess good analytical and problem-solving skills, and excel in communication and presentation. In conclusion, as an Azure Senior Developer at PwC - AC, you will play a pivotal role in delivering high-quality Cloud data engineering solutions, leveraging your expertise in Azure cloud services, big data technologies, and modern data architecture concepts to support client engagements effectively.,

As a Senior AWS Data Engineer Cloud Data Platform at Teamware Solutions, a division of Quantum Leap Consulting Pvt. Ltd, located in Bangalore, you will be responsible for end-to-end implementation of Cloud data engineering solutions like Enterprise Data lake and Data hub in AWS. Working onsite in an office environment for 5 days a week, you will collaborate with the Offshore Manager and Onsite Business Analyst to understand the requirements and deliver scalable, distributed, cloud-based enterprise data solutions. You should have a strong background in AWS cloud technology, with 4-8 years of hands-on experience. Proficiency in architecting and delivering highly scalable solutions is a must, along with expertise in Cloud data engineering solutions, Lambda or Kappa Architectures, Data Management concepts, and Data Modelling. You should be proficient in AWS services such as EMR, Glue, S3, Redshift, and DynamoDB, as well as have experience in Big Data frameworks like Hadoop and Spark. Additionally, you must have hands-on experience with AWS compute and storage services, AWS Streaming Services, troubleshooting and performance tuning in Spark framework, and knowledge of Application DevOps tools like Git and CI/CD Frameworks. Familiarity with AWS CloudWatch, Cloud Trail, Account Config, Config Rules, security, key management, data migration processes, and analytical skills is required. Good communication and presentation skills are essential for this role. Desired skills include experience in building stream-processing systems, Big Data ML toolkits, Python, Offshore/Onsite Engagements, flow tools like Airflow, Nifi or Luigi, and AWS services like STEP & Lambda. A professional background in BE/B.Tech/MCA/M.Sc/M.E/M.Tech/MBA is preferred, and an AWS certified Data Engineer certification is recommended. If you are interested in this position and meet the qualifications mentioned above, please send your resume to netra.s@twsol.com.,

The role of Software Architect requires a minimum of 10-18 years of experience in designing and architecting large-scale data solutions. As a Software Architect, you should have the ability to design, architect, and create specifications for solutions on various IOT Stacks such as Azure IOT, AWS Greengrass / AWS IOT Core, or any other IOT Stack. Additionally, experience and understanding of unit test frameworks like NUnit, XUnit, and Api Testing are essential. It is also important to have expertise in Cloud Native Microservices development and CI/CD pipelines, along with hands-on experience with Dot Net Core and exposure to Docker, MQTT, and AMQP. Demonstrated skill in using coding standards and active participation in code and design reviews are expected from the Software Architect. Furthermore, the ability to technically communicate and articulate Solution Design and Architecture is crucial. Nice to have skills for this role include experience working with ReactJS, Angular, or other NodeJS frontend frameworks, working knowledge of Python, managing teams based in multiple geographies, and experience with messaging systems, stream-processing systems, event-hubs, IoT-hubs, stateless services & applications. Understanding of security principles & best practices, AI capabilities on the edge, issue tracking tools like JIRA and Azure DevOps, and Agile software development methodology are also desirable. The Software Architect position is located in Thane West and requires 100 percent work from the office with no hybrid or work from home options. The ideal candidate should have a B.Tech/B.E. degree in Any Specialization at the undergraduate level.,

Join us as an Engineering & Design Lead at Barclays and spearhead the evolution of the digital landscape, driving innovation and excellence. You will harness cutting-edge technology to revolutionize our digital offerings, ensuring unparalleled customer experiences. Your role will involve a deep understanding of software engineering using modern development tools and languages. Hands-on experience in Cloud technologies, preferably AWS, and knowledge of enterprise database systems like Oracle will be essential. Experience in building Multi-Tenant platforms, High-Performing systems, and understanding DevSecOps concepts will be critical. You should possess strong programming skills in Java and Python, proficiency in object-oriented design principles, and familiarity with frameworks such as DropWizard, Spring, and Hibernate. Experience with Testing frameworks, web development frameworks, and building stream-processing systems using Kubernetes will be beneficial. Additionally, knowledge of distributed storage systems, streaming platform systems, caching solutions, build tools, CI/CD pipelines, and Unix/Linux will be required. Key skills like stakeholder management, analytical/problem-solving mindset, interpersonal communication, and collaboration with global teams will also be highly valued. Formal qualifications in digital technology or methodology will be advantageous. Your primary purpose will be to design, develop, and improve software using various engineering methodologies to provide business, platform, and technology capabilities for customers and colleagues. Key accountabilities will include developing high-quality software solutions, collaborating with product managers and engineers, promoting code quality, staying informed about technology trends, adhering to secure coding practices, and implementing effective unit testing. As a Vice President, you will advise key stakeholders, manage and mitigate risks, provide leadership, collaborate across functions, and create solutions based on sophisticated analytical thinking. Building and maintaining trusting relationships with internal and external stakeholders to achieve key business objectives will also be crucial. All colleagues are expected to demonstrate Barclays Values of Respect, Integrity, Service, Excellence, and Stewardship, as well as the Barclays Mindset of Empower, Challenge, and Drive in their behavior.,

You should possess a Bachelor's degree in Computer Science, Engineering, or a related field along with at least 8 years of work experience in Data First systems. Additionally, you should have a minimum of 4 years of experience working on Data Lake/Data Platform projects specifically on AWS/Azure. It is crucial to have extensive knowledge and hands-on experience with Data warehousing tools such as Snowflake, BigQuery, or RedShift. Proficiency in SQL for managing and querying data is a must-have skill for this role. You are expected to have experience with relational databases like Azure SQL, AWS RDS, as well as an understanding of NoSQL databases like MongoDB for handling various data formats and structures. Familiarity with orchestration tools like Airflow and DBT would be advantageous. Experience in building stream-processing systems using solutions such as Kafka or Azure Event Hub is desirable. Your responsibilities will include designing and implementing ETL/ELT processes using tools like Azure Data Factory to ingest and transform data into the data lake. You should also have expertise in data migration and processing with AWS (S3, Glue, Lambda, Athena, RDS Aurora) or Azure (ADF, ADLS, Azure Synapse, Databricks). Data cleansing and enrichment skills are crucial to ensure data quality for downstream processing and analytics. Furthermore, you must be capable of managing schema evolution and metadata for the data lake, with experience in tools like Azure Purview for data discovery and cataloging. Proficiency in creating and managing APIs for data access, preferably with experience in JDBC/ODBC, is required. Knowledge of data governance practices, data privacy laws like GDPR, and implementing security measures in the data lake are essential aspects of this role. Strong programming skills in languages like Python, Scala, or SQL are necessary for data engineering tasks. Additionally, experience with automation and orchestration tools, familiarity with CI/CD practices, and the ability to optimize data storage and retrieval for analytical queries are key requirements. Collaboration with the Principal Data Architect and other team members to align data solutions with architectural and business goals is crucial. As a lead, you will be responsible for critical system design changes, software projects, and ensuring timely project deliverables. Collaboration with stakeholders to translate business needs into efficient data infrastructure systems is a key aspect of this role. Your ability to review design proposals, conduct code review sessions, and promote best practices is essential. Experience in an Agile model, delivering quality deliverables on time, and translating complex requirements into technical solutions are also part of your responsibilities.,