Job Description

Must Have Technical/Functional Skills

• 10+ years of experience in Data Engineering and Data Architecture.
• 5+ years of hands-on experience with Databricks, PySpark, and Apache Spark.
• Strong expertise in - Databricks Lakehouse Architecture, Delta Lake, Unity Catalog, Delta Live Tables (DLT), Databricks Workflows, Medallion Architecture
• Experience designing enterprise-scale analytics and data platforms.
• Expert-level proficiency in PySpark and Spark SQL.
• Advanced SQL development and query optimization skills.
• Strong understanding of batch and streaming architectures.
• Experience with data warehousing, ETL/ELT, and large-scale data processing.
• Expertise in dimensional, relational, and analytical data modeling.
• Experience implementing data governance, data quality, and metadata management solutions.
• Knowledge of CI/CD practices, Git, DevOps, and Infrastructure as Code (Terraform preferred).
• Experience establishing architectural standards and engineering best practices.

Roles & Responsibilities

• Architect and design scalable, secure, and high-performance data platforms using Databricks and AWS.
• Lead solution design, data modeling, and enterprise data architecture initiatives.
• Design and optimize batch and real-time data pipelines using Databricks, PySpark, and Delta Lake.
• Partner with business and technology stakeholders to define requirements and deliver strategic data solutions.
• Establish standards for data governance, security, quality, performance, and operational excellence.
• Provide technical leadership, mentoring, and architectural guidance to engineering teams.
• Drive cloud modernization, platform optimization, and data transformation initiatives.

Job Tags

Similar Jobs