Senior databricks specialist / data architect

Job Role: Senior Databricks Specialist / Data Architect Role Overview We are looking for a Senior Databricks specialist with extensive experience designing,implementing,and optimizing enterprise-scale data platforms using Microsoft Azure Databricks. The candidate will be responsible for the architecture, development, optimization,governance, integration, and performance of the organization's Databricks data platformand its integration with Dynamics 365 CRM and other enterprise systems. Key Responsibilities Design and implement enterprise-scale data solutions using Azure Databricks. Define and maintain Databricks data architecture and platform standards. Design data ingestion and integration pipelines from multiple source systems. Develop scalable ETL/ELT pipelines using Databricks and Apache Spark. Design and implement Medallion Architecture (Bronze, Silver, Gold) whereappropriate. Work with Delta Lake and optimize Delta tables and data pipelines. Design data models and transformation processes for analytical and operationalreporting. Integrate Databricks with Microsoft Dynamics 365 / Dataverse and otherenterprise applications. Develop and optimize batch and near-real-time data processing pipelines. Implement data quality, validation, monitoring, lineage, and governanceprocesses. Optimize Spark jobs, clusters, SQL queries, Delta tables, and data pipelines forperformance and cost. Implement appropriate data security and access controls. Work with Unity Catalog for data governance, access control, cataloging, andlineage. Support Power BI and other analytical/reporting platforms through optimized datamodels and data services. Troubleshoot complex data engineering and performance issues. Establish development, testing, and production standards for Databricks. Support CI/CD and Dev Ops practices for Databricks solutions. Conduct technical reviews and mentor data engineers. Ensure data platform solutions meet security, availability, scalability, and disaster recovery requirements. Mandatory Technical Requirements Minimum 8+ years of experience in data engineering/data architecture. Minimum 4+ years of strong hands-on experience with Azure Databricks. Strong hands-on experience with Apache Spark and Py Spark. Strong experience with Delta Lake. Strong experience designing enterprise data platforms and data pipelines. Strong knowledge of Databricks Medallion Architecture. Strong experience with Unity Catalog, data governance, access control, and datalineage. Strong SQL skills and experience with large-scale data processing. Strong experience with Azure data services. Experience with Azure Data Factory / Azure Data Lake Storage Gen2. Experience integrating Databricks with APIs, enterprise applications, CRMsystems, and other data sources. Experience with data modeling and dimensional modeling. Strong understanding of data quality and data governance. Strong understanding of performance optimization and Databricks costoptimization. Experience with CI/CD, Git, Azure Dev Ops, and deployment automation. Experience working with large-volume enterprise datasets. Ability to troubleshoot complex production data issues. Preferred Certifications Databricks Certified Data Engineer Professional Databricks Certified Data Engineer Associate Microsoft Certified: Azure Data Engineer Associate Other relevant Databricks/Azure certifications #J-18808-Ljbffr