Data Engineer (Public Healthcare)

6-month contract, renewable Government project Hybrid work arrangement We are seeking one Data Engineer to establish and maintain the data infrastructure required to integrate, process, govern and use these data reliably. The data engineer will support the development of scalable data pipelines, standardised data models, secure data environments and high-quality datasets for product analytics, programme evaluation, research and AI development. This capability is necessary to enable evidence-informed decision-making, measure health and operational outcomes, support responsible AI deployment, and facilitate the future scaling and integration of mental health innovations across the healthcare ecosystem. 1. Data Architecture and Strategy Design and maintain scalable data architectures to support digital mental health platforms, analytics, research, and AI-enabled use cases. Translate programme, product, research, and operational requirements into data architecture, data flow, storage, processing, and integration requirements. Develop target-state and transitional data architecture plans aligned with system roadmaps, security requirements, and anticipated data volumes. Recommend appropriate data engineering approaches, technologies, and design patterns based on performance, cost, maintainability, interoperability, and security considerations. Ensure data architecture supports future scaling, cross-system integration, advanced analytics, machine learning, and responsible data reuse. 2. Data Pipeline Development and Integration Design, develop, test, deploy, and maintain batch and real-time data pipelines across relevant source systems. Extract, transform, and load data from digital platforms, applications, clinical systems, surveys, research tools, third-party services, and other approved data sources. Integrate structured, semi-structured, and unstructured data, including user interaction data, assessment results, system logs, conversational data, operational data, and AI-generated outputs. Develop and maintain application programming interfaces, connectors, data ingestion services, and data exchange mechanisms. Ensure pipelines are reliable, modular, reusable, scalable, and capable of handling changes in source data structures. Implement appropriate error handling, retry logic, reconciliation processes, and failure notifications. 3. Data Modelling and Storage Design and maintain logical and physical data models, schemas, data marts, and analytical datasets. Develop standardised data structures and common definitions across programmes, products, and use cases. Establish appropriate relationships between user, session, assessment, intervention, engagement, referral, escalation, provider, and outcome data. Optimise data storage and query performance for operational reporting, research analysis, product analytics, and machine learning workloads. Implement data partitioning, indexing, retention, archival, and deletion mechanisms where required. Maintain clear separation between raw, processed, curated, and consumption-ready data layers. 4. Data Quality, Validation and Observability Define and implement automated data quality checks covering completeness, accuracy, validity, consistency, uniqueness, timeliness, and referential integrity. Establish data validation rules and acceptance thresholds in consultation with product, research, analytics, clinical, and operational teams. Develop monitoring dashboards and alerts for pipeline failures, delayed data, schema changes, anomalous values, missing records, and data drift. Investigate and resolve data quality issues, including root-cause analysis and corrective action. Maintain data quality logs, issue registers, reconciliation reports, and resolution records. Support validation of key metrics to ensure consistency between source systems, analytical datasets, and reporting outputs. 5. Data Governance, Privacy and Security Implement data engineering controls in accordance with applicable data protection, cybersecurity, healthcare, research, and organisational requirements. Apply appropriate access controls, encryption, masking, pseudonymisation, anonymisation, tokenisation, and segregation of sensitive data. Ensure personal, health-related, research, and conversational data are handled according to approved purposes and access permissions. Maintain data lineage, data provenance, processing records, and traceability across data pipelines and systems. Support implementation of dat…