Senior Solution Architect - Databricks

Avensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success we have evolved to become one of the top trusted providers in Singapore and service a client base across banking and financial services, insurance, information technology, healthcare, retail, and supply chain. We are currently looking to hire Senior Solution Architect - Databricks. This is an exciting opportunity to expand your skill set, achieve job satisfaction and work-life balance. More details as below. Job Role: Job Description Essential Technical Skills • Databricks Platform: Proven hands-on experience with Databricks platform including workspace administration, cluster management, and workflow orchestration • Unity Catalog: Demonstrated experience implementing Unity Catalog for unified governance, including metastore configuration, catalog/schema design, and access control policies • Data Modelling: Strong expertise in designing and implementing data models, including dimensional modeling, data vault, and lakehouse architectures • Delta Lake: Deep understanding of Delta Lake features including ACID transactions, time travel, and optimization techniques • Apache Spark: Proficiency in Spark SQL, DataFrames, and performance tuning • Programming: Strong skills in Python, SQL, and/or Scala for data processing • Cloud Platforms: Experience with Azure, AWS, or GCP cloud services Additional Technical Skills • Data pipeline development and ETL/ELT processes • Data governance frameworks and metadata management • Performance optimization and troubleshooting • CI/CD practices for data platforms • Data quality and validation frameworks • Understanding of data security and compliance requirements Professional Experience • Minimum 8-10 years in data architecture, engineering, or analytics roles • At least 3-5 years of hands-on experience with Databricks platform • Databricks Certified Associate Developer for Apache Spark (mandatory) • Proven track record implementing Unity Catalog in enterprise environments • Experience designing and implementing complex data models for large-scale systems • Background working with Databricks Professional Services or similar partnerships preferred • Experience across multiple industry sectors (Public Service, Financial Services, Healthcare, etc.) is advantageous Certifications (Preferred) - mandatory to have at least one certification • Data Engineer Professional • Cloud platform certifications (Azure Data Engineer, AWS Data Analytics, or GCP Data Engineer) • Relevant data management or analytics certifications Soft Skills • Excellent communication and presentation abilities • Strong analytical and problem-solving capabilities • Ability to work collaboratively with Partners and cross-functional teams • Client-focused mindset with relationship-building skills • Self-motivated with ability to manage multiple priorities • Experience working in agile environments Priority for candidates in Singapore End-to-End Data Architecture • Collaborate with Databricks PS and customer • on comprehensive end-to-end data architecture on Databricks platform • Design and implement data ingestion strategies from various source systems • Architect scalable storage solutions leveraging Delta Lake and lakehouse architecture • Develop efficient data processing frameworks using Apache Spark and Databricks workflows • Create data consumption layers for analytics, reporting, and machine learning use cases • Ensure seamless data flow across the entire data lifecycle Governance, Security & Compliance • Collaborate with Databricks PS on implementing robust governance frameworks • Design and enforce security policies and access controls within Databricks environments • Ensure compliance with regulatory requirements and industry standards • Implement data lineage, audit trails, and monitoring capabilities • Optimize performance through best practices in cluster configuration and query optimization • Establish data quality standards and validation processes Data Modelling & Design • Define business-aligned data models that support organizational objectives • Design Delta table structures optimized for performance and scalability • Create dimensional and normalized data models based on business requirements • Implement medallion architecture (Bronze, Silver, Gold layers) for…