Jobs
Interviews
ThreatXIntel logo

Freelance Opportunity: Big Data Engineer (Cloud & Data Pipeline Focus)

5 years

0 Lacs

India

Posted:4 weeks ago| Platform: Linkedin logo

Apply

Skills Required

data pipeline security mobile testing assessment devsecops support engineering software drive strategies design hive mapreduce etl architecture aws interface technology development sql unix apache sqoop iam logging

Work Mode

On-site

Job Type

Part Time

Job Description

Company Description

ThreatXIntel is a startup cyber security company that specializes in protecting businesses and organizations from cyber threats. We offer customized, affordable solutions in cloud security, web and mobile security testing, cloud security assessment, and DevSecOps. Our proactive approach to security ensures that our clients' digital assets are safeguarded.


Role Description

Big Data Engineer

Big Data frameworks

Key Responsibilities

  • Drive

    process improvement

    initiatives and propose

    cost-saving

    strategies across data engineering practices.
  • Act as an

    evangelist

    for the data engineering function by promoting Big Data best practices and standards.
  • Design, develop, and monitor robust

    data pipelines

    using Hive, Impala, MapReduce, and related frameworks.
  • Provide production support and troubleshoot issues in

    streaming data jobs

    , batch jobs, and ETL processes.
  • Contribute to

    cloud architecture strategy

    , especially with AWS-based data tools.
  • Collaborate with software engineers, analysts, and data scientists to understand data needs and deliver integrated solutions.
  • Interface independently with stakeholders across technology and business units to gather requirements and present solutions.

Required Skills & Experience

  • 5+ years

    of hands-on experience in

    Big Data application design, development, and implementation

  • 3+ years

    of

    cloud experience

    , preferably on

    AWS

  • 5+ years

    of experience in

    SQL

    , with strong skills in

    Hive/Impala

    and

    performance tuning

  • Solid understanding of

    Data Warehousing concepts

  • Experience working with multiple databases including

    Oracle

    ,

    MongoDB

    ,

    SQL Server

    , and

    PostgreSQL

  • Proficiency in

    Unix Shell scripting

    and

    SQL scripting

  • Experience with

    job schedulers

    such as

    Control-M

  • Hands-on experience with

    Apache Kafka

    or

    AWS Kinesis

    for stream processing
  • Production support experience for

    Big Data pipelines

    ,

    Oozie workflows

    ,

    MapReduce

    , and

    Sqoop jobs

Preferred AWS Services Experience

  • AWS EMR

  • AWS Glue

  • Amazon S3

  • AWS Athena

  • Amazon DynamoDB

  • IAM (Identity and Access Management)

  • AWS Lambda

  • CloudWatch

    for logging and monitoring


More Jobs at ThreatXIntel

Freelance KQL Developer

India

Experience: Not specified

|

Salary: Not disclosed

Senior SAP ABAP Developer – Payroll Domain

India

Experience: Not specified

|

Salary: Not disclosed

Freelance Big Data Developer (Scala, Spark, IntelliJ)

India

Experience: Not specified

|

Salary: Not disclosed

Freelance SIEM Specialist Splunk SPL Cisco AMP

India

Experience: Not specified

|

Salary: Not disclosed

Freelance Scala & Spark Developer – Debugging, Unit Testing, and Code Enhancement

India

Experience: Not specified

|

Salary: Not disclosed

Mock Interview

Practice Video Interview with JobPe AI

Start Job-Specific Interview
cta

Start Your Job Search Today

Browse through a variety of job opportunities tailored to your skills and preferences. Filter by location, experience, salary, and more to find your perfect fit.

Job Application AI Bot

Job Application AI Bot

Apply to 20+ Portals in one click

Download Now

Download the Mobile App

Instantly access job listings, apply easily, and track applications.

Google Play App Store
coding practice

Enhance Your Skills

Practice coding challenges to boost your skills

Start Practicing Now
ThreatXIntel logo
ThreatXIntel

RecommendedJobs for You

ThreatXIntel logo

Freelance Opportunity: Big Data Engineer (Cloud & Data Pipeline Focus)

ThreatXIntel

India

ThreatXIntel logo

Freelance Opportunity: Big Data Engineer (Cloud & Data Pipeline Focus)

ThreatXIntel

India