AWS Data Engineer - Con/AM Sampoorna Consultants Pvt. Ltd

4.0 - 9.0 years

8 - 18 Lacs

bengaluru

Work from Office

Job Requirements Mandatory Skills Bachelors degree in computer science, Data Science, engineering, mathematics, information systems, or a related technical discipline 7+ years of relevant experience in data engineering roles Detailed knowledge of data warehouse technical architectures, data modelling, infrastructure components, ETL/ ELT and reporting/analytic tools and environments, data structures and hands-on SQL coding Proficient in at least one or more programming languages: Java, Python, Ruby, Scala Experienced with AWS services such as Redshift, S3, EC2, Lambda, Athena, EMR, AWS Glue, Datapipeline. Exposure to data visualization and reporting with tools such as Amazon QuickSight, Metabase, Tableau, or similar software Experience building metrics deck and dashboards for KPIs including the underlying data models. Understand how to design, implement, and maintain a platform providing secured access to large datasets Primary Roles and Responsibilities An AWS Data Engineer is responsible for designing, building, and maintaining the data infrastructure for an organization using AWS cloud services. This includes creating data pipelines, integrating data from various sources, and implementing data security and privacy measures. The AWS Data Engineer will also be responsible for monitoring and troubleshooting data flows and optimizing data storage and processing for performance and cost efficiency. Preferred Skills Masters degree in computer science, Data Science, engineering, mathematics, information systems, or a related technical discipline 7+ years of work experience with ETL, Data Modelling, and Data Architecture. Experience or familiarity with newer analytics tools such as AWS Lake Formation, Sagemaker, DynamoDB, Lambda, ElasticSearch. Experience with Data streaming service e.g Kinesis Kafka Ability to develop experimental and analytic plans for data modeling processes, use of strong baselines, ability to accurately determine cause and effect relations Proven track record partnering with business owners to understand requirements and developing analysis to solve their business problems Proven analytical and quantitative ability and a passion for enabling customers to use data and metrics to back up assumptions, develop business cases, and complete root cause analysis

Posted 5 days ago

Apply

azure data engineer with fabric ASV Consulting Services

6.0 - 10.0 years

8 - 12 Lacs

Pune, Gurugram, Bengaluru

Work from Office

Contractual Hiring manager :- My profile :- linkedin.com/in/yashsharma1608 Payroll of :- https://www.nyxtech.in/ 1. AZURE DATA ENGINEER WITH FABRIC The Role : Lead Data Engineer PAYROLL Client - Brillio About Role: Experience 6 to 8yrs Location- Bangalore , Hyderabad , Pune , Chennai , Gurgaon (Hyderabad is preferred) Notice- 15 days / 30 days. Budget -15 LPA AZURE FABRIC EXP MANDATE Skills : Azure Onelake, datapipeline , Apache Spark , ETL , Datafactory , Azure Fabric , SQL , Python/Scala. Key Responsibilities: Data Pipeline Development: Lead the design, development, and deployment of data pipelines using Azure OneLake, Azure Data Factory, and Apache Spark, ensuring efficient, scalable, and secure data movement across systems. ETL Architecture: Architect and implement ETL (Extract, Transform, Load) workflows, optimizing the process for data ingestion, transformation, and storage in the cloud. Data Integration: Build and manage data integration solutions that connect multiple data sources (structured and unstructured) into a cohesive data ecosystem. Use SQL, Python, Scala, and R to manipulate and process large datasets. Azure OneLake Expertise: Leverage Azure OneLake and Azure Synapse Analytics to design and implement scalable data storage and analytics solutions that support big data processing and analysis. Collaboration with Teams: Work closely with Data Scientists, Data Analysts, and BI Engineers to ensure that the data infrastructure supports analytical needs and is optimized for performance and accuracy. Performance Optimization: Monitor, troubleshoot, and optimize data pipeline performance to ensure high availability, fast processing, and minimal downtime. Data Governance & Security: Implement best practices for data governance, data security, and compliance within the Azure ecosystem, ensuring data privacy and protection. Leadership & Mentorship: Lead and mentor a team of data engineers, promoting a collaborative and high-performance team culture. Oversee code reviews, design decisions, and the implementation of new technologies. Automation & Monitoring: Automate data engineering workflows, job scheduling, and monitoring to ensure smooth operations. Use tools like Azure DevOps, Airflow, and other relevant platforms for automation and orchestration. Documentation & Best Practices: Document data pipeline architecture, data models, and ETL processes, and contribute to the establishment of engineering best practices, standards, and guidelines. C Innovation: Stay current with industry trends and emerging technologies in data engineering, cloud computing, and big data analytics, driving innovation within the team.C

Posted 2 months ago

Apply

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.