Senior Executive - Core Data Engineering

Strong proficiency in Python, Scala, and SQL for large-scale data processing, transformation, and analytics.

Hands-on experience with distributed data processing like (Apache Spark/ Flink & Kafka) and streaming frameworks for batch and near real-time pipelines.

Solid understanding of ETL / ELT concepts, data transformation practices and analytics engineering (e.g., dbt)

Strong knowledge of data lake and data warehouse architecture including medallion (Bronze / Silver / Gold) models

Experience working with modern table formats, metadata, and catalog-driven storage (e.g., Parquet, Iceberg)

Hands-on experience with distributed and object storage systems (e.g., HDFS, S3 or equivalent)

Experience integrating relational and NoSQL databases and handling semi-structured data (Oracle, PostgreSQL, MongoDB, JSON)

Proficiency in data orchestration and workflow management (Airflow, Dagster, Mage AI or equivalent)

Experience with containerization and deployment using Docker and Kubernetes

Proficiency in shell scripting for automation and operational support

Design, develop, and maintain scalable batch and near real-time data pipelines to support analytics, reporting and advanced use cases.

Define and implement data architecture standards including layered data models for raw, refined, and curated datasets.

Build and manage end-to-end data ingestion, transformation, and delivery workflows across multiple data sources.

Ensure data consistency, quality, reliability, and performance across all data pipelines.

Design and maintain analytics-ready and ML-ready datasets, including structured and semi-structured data.

Establish and maintain data lineage, traceability, and documentation from source to consumption.

Implement effective data orchestration, dependency management, and workflow scheduling.

Design and maintain dimensional data models to support BI, analytics, and reporting needs.

Collaborate with BI, Analytics, AI/ML, and Assurance teams to align data outputs with business requirements.

Ensure adherence to data governance, security, and access control standards.