Lead Data Architect

5 - 7 years

0 Lacs

Posted:4 days ago| Platform: Foundit logo

Apply

Work Mode

On-site

Job Type

Full Time

Job Description

The Role Context:


This is an exciting opportunity to join a dynamic and growing organization, working at the forefront of technology trends and developments in social impact sector. Wadhwani Center for Government Digital Transformation (WGDT) works with the government ministries and state departments in India with a mission of Enabling digital transformation to enhance the impact of government policy, initiatives and programs.

Data Engineer

Key Responsibilities:

Data Architecture Design

Data Integration:

Data Modeling:

Data Pipeline Development/ ETL (Extract, Transform, Load):

Data Quality and Governance:

Data lakes or Warehousing:

Data Security

Database Management:

Data Migration:

Performance Optimization

Collaboration:

Documentation and Reporting:

Continuous Learning:

Desired Skills/ Competencies

  • Education:

    A Bachelor's or Master's degree in Computer Science, Software Engineering, Data Science, or equivalent with at least 5 years of experience.
  • Database Management:

    Strong expertise in working with databases, such as SQL databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).
  • Big Data Technologies:

    Familiarity with big data technologies, such as Apache Hadoop, Spark, and related ecosystem components, for processing and analyzing large-scale datasets.
  • ETL Tools:

    Experience with ETL tools (e.g., Apache NiFi, Talend, Apache Airflow, Talend Open Studio, Pentaho, Infosphere) for designing and orchestrating data workflows.
  • Data Modeling and Warehousing:

    Knowledge of data modeling techniques and experience with data warehousing solutions (e.g., Amazon Redshift, Google BigQuery, Snowflake).
  • Data Governance and Security:

    Understanding of data governance principles and best practices for ensuring data quality and security.
  • Cloud Computing:

    Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and their data services for scalable and cost-effective data storage and processing.
  • Streaming Data Processing:

    Familiarity with real-time data processing frameworks (e.g., Apache Kafka, Apache Flink) for handling streaming data.
  • KPIs:

  • Data Pipeline Efficiency:

    Measure the efficiency of data pipelines in terms of data processing time, throughput, and resource utilization. KPIs could include average time to process data, data ingestion rates, and pipeline latency.
  • Data Quality Metrics:

    Track data quality metrics such as completeness, accuracy, consistency, and timeliness of data. KPIs could include data error rates, missing values, data duplication rates, and data validation failures.
  • System Uptime and Availability:

    Monitor the uptime and availability of data infrastructure, including databases, data warehouses, and data processing systems. KPIs could include system uptime percentage, mean time between failures (MTBF), and mean time to repair (MTTR).
  • Data Storage Efficiency:

    Measure the efficiency of data storage systems in terms of storage utilization, data compression rates, and data retention policies. KPIs could include storage utilization rates, data compression ratios, and data storage costs per unit.
  • Data Security and Compliance:

    Track adherence to data security policies and regulatory compliance requirements such as DPDP, GDPR, HIPAA, or PCI DSS. KPIs could include security incident rates, data access permissions, and compliance audit findings.
  • Data Processing Performance:

    Monitor the performance of data processing tasks such as ETL (Extract, Transform, Load) processes, data transformations, and data aggregations. KPIs could include data processing time, CPU usage, and memory consumption.
  • Scalability and Performance Tuning:

    Measure the scalability and performance of data systems under varying workloads and data volumes. KPIs could include scalability benchmarks, system response times under load, and performance improvements achieved through tuning.
  • Resource Utilization and Cost Optimization:

    Track resource utilization and costs associated with data infrastructure, including compute resources, storage, and network bandwidth. KPIs could include cost per data unit processed, cost per query, and cost savings achieved through optimization.
  • Incident Response and Resolution:

    Monitor the response time and resolution time for data-related incidents and issues. KPIs could include incident response time, time to diagnose and resolve issues, and customer satisfaction ratings for support services.
  • Documentation and Knowledge Sharing

    : Measure the quality and completeness of documentation for data infrastructure, data pipelines, and data processes. KPIs could include documentation coverage, documentation update frequency, and knowledge sharing activities such as internal training sessions or knowledge base contributions.
  • Years of experience of the current role holder

    New Position

    Ideal years of experience

    3 5 years

    Career progression for this role

    CTO WGDT (Head of Incubation Centre)

    *******************************************************************************

    Wadhwani Corporate Profile: (Click on this link)

    Our Culture:

    WF is a global not-for-profit, and works like a start-up, in a fast-moving, dynamic pace where change is the only constant and flexibility is the key to success.

    Three mantras that we practice across job roles, levels, functions, programs and initiatives, are Quality, Speed, Scale, in that order.

    We are an ambitious and inclusive organization, where everyone is encouraged to contribute and ideate. We are intensely and insanely focused on driving excellence in everything we do.

    We want individuals with the drive for excellence, and passion to do whatever it takes to deliver world class outcomes to our beneficiaries. We set our own standards often more rigorous than what our beneficiaries demand, and we want individuals who love it this way.

    We have a creative and highly energetic environment one in which we look to each other to innovate new solutions not only for our beneficiaries but for ourselves too. Open to collaborate with a borderless mentality, often going beyond the hierarchy and siloed definitions of functional KRAs, are the individuals who will thrive in our environment.

    This is a workplace where expertise is shared with colleagues around the globe. Individuals uncomfortable with change, constant innovation, and short learning cycles and those looking for stability and orderly working days may not find WF to be the right place for them.

    Finally, we want individuals who want to do greater good for the society leveraging their area of expertise, skills and experience.

    The foundation is an equal opportunity firm with no bias towards gender, race, colour, ethnicity, country, language, age and any other dimension that comes in the way of progress.

    Join us and be a part of us!

    Mock Interview

    Practice Video Interview with JobPe AI

    Start Job-Specific Interview
    cta

    Start Your Job Search Today

    Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

    Job Application AI Bot

    Job Application AI Bot

    Apply to 20+ Portals in one click

    Download Now

    Download the Mobile App

    Instantly access job listings, apply easily, and track applications.

    coding practice

    Enhance Your Skills

    Practice coding challenges to boost your skills

    Start Practicing Now
    Wadhwani Foundation logo
    Wadhwani Foundation

    Non-Profit / Economic Development

    N/A

    RecommendedJobs for You

    noida, navi mumbai, bengaluru

    noida, hyderabad, bengaluru

    pune, maharashtra, india

    hyderabad, telangana, india