Data Engineer

Skills

About this role

Data Engineer to build and scale the data infrastructure that powers AI-driven products and research initiatives. In this role, you will develop distributed data pipelines, manage large-scale datasets across cloud environments, and design reliable data systems that support data processing, experimentation, and model development at scale.

Data Engineering, Python, SQL, Spark, AWS

Responsibilities

• Design, build, and maintain scalable data pipelines to ingest, process, and transform large-scale datasets from multiple sources.
• Develop and optimize distributed data processing workflows using Spark and cloud-native technologies.
• Build and maintain data storage solutions across SQL and NoSQL systems, ensuring scalability, performance, and reliability.
• Design and implement data architectures on AWS to support high-volume data ingestion, processing, and distribution.
• Write efficient Python and SQL code to extract, transform, validate, and analyze large datasets.
• Ensure data quality, integrity, monitoring, and operational reliability across data pipelines and storage layers.
• Collaborate with AI researchers, data scientists, and engineering teams to support data-intensive applications and experimentation.
• Implement automation, orchestration, and monitoring workflows to support scalable and efficient data operations.

Qualifications

• Strong proficiency in Python, SQL, and distributed data processing frameworks such as Apache Spark.
• Hands-on experience with AWS data services and cloud-native data architectures.
• Experience working with both SQL and NoSQL databases.
• Experience managing and processing large-scale datasets in distributed environments.
• Strong understanding of data partitioning, performance optimization, and scalable data architectures