Senior Big Data Engineer

Job Title: Senior Big Data Engineer

Experience: 6+ Years

Job Summary

We are seeking a Senior Big Data Engineer with 6+ years of experience to architect, build, and optimize large-scale data platforms. The ideal candidate will lead complex data initiatives, ensure high performance and reliability, and mentor junior engineers.

Key Responsibilities

Architect and implement scalable big data solutions

Design and maintain batch & real-time data pipelines

Lead development using Apache Spark (PySpark/Scala)

Work extensively with Hadoop ecosystem (HDFS, Hive, HBase, YARN)

Build streaming pipelines using Kafka / Flink / Spark Streaming

Optimize data storage, compute, and query performance

Ensure data quality, governance, and security standards

Collaborate with stakeholders to translate business needs into data solutions

Guide and mentor junior and mid-level engineers

Perform code reviews and enforce best practices

Troubleshoot production issues and improve system reliability

Required Skills & Qualifications

Bachelor’s or Master’s degree in Computer Science / Engineering

6+ years of hands-on experience in Big Data & Data Engineering

Expert-level knowledge of Apache Spark

Strong experience with Hadoop ecosystem

Advanced proficiency in Python / Scala / Java

Strong SQL skills and experience with large datasets

Experience with NoSQL databases (HBase, Cassandra, MongoDB)

Strong Linux, shell scripting, and performance tuning skills

Preferred / Good to Have

Cloud experience: AWS / Azure / GCP

Data orchestration tools: Airflow / Oozie

Data lake & lakehouse architectures

Experience with Delta Lake / Iceberg / Hudi

CI/CD, DevOps, and infrastructure automation

Exposure to data governance & compliance