M20 - Data Engineer

Responsibilities Design and develop scalable backend services integrating with external vendor systems via RESTful APIs Implement robust data ingestion services including pagination and incremental/time-window retrieval Handle API error handling, retries, and idempotent processing to ensure reliable integrations Build and maintain data pipelines for processing large volumes of transactional and event data Perform data transformation, validation, and enrichment for downstream systems Support batch and near real-time data processing workflows Develop and deploy applications using Microsoft Azure services such as Azure Functions, App Services, Storage, Key Vault, APIM, and VNET Design scalable, resilient, and highly available cloud-native solutions Implement observability practices including logging, monitoring, and alerting Ensure data accuracy, consistency, and reconciliation across multiple data sources Support auditability and traceability of data flows for regulatory requirements Collaborate with external vendors on API contracts and integration designs Deliver high-quality, maintainable, and reusable software solutions Contribute to engineering standards, best practices, and continuous improvement initiatives Support CI/CD pipelines, automated testing, and continuous delivery practices Requirements Core Technical Skills Strong proficiency in Python (preferred) or backend languages such as Java or Node.js Strong SQL proficiency including PostgreSQL, MySQL, or T-SQL Solid experience building backend services and API integrations Experience working with RESTful APIs and JSON payloads Knowledge of secure integration patterns including API keys, TLS, certificates, and PGP Good understanding of: Pagination, filtering, and incremental data retrieval Idempotency, retries, and distributed system failure handling Experience with data visualisation tools such as Power BI, including connectivity and configuration Data Engineering & Processing Experience building batch or …