Big Data Administrator

Summary of Position He/She will be a technical L2 resource for all Big Data services and will provide support for all production support activities within the Big Data team in Singapore. Will work with L3/Service Manager to gain control over the scope of technical activities, develop best practices and gain knowledge over all aspects of support. Job Description As a L2 resource of his /her team, he /she: Takes up technical tasks and also manages delegation for technical issues within the team, animates the team to encourage collaboration and sharing of best practices, supports new technologies and leverages them to provide consistency of service across streams, proposes service improvements for all Big Data services supported throughout the organization, documents, reviews, maintains and shares relevant technical information within the team provides technical knowledge, supports services both proactively and reactively to maintain the availability and reliability of system infrastructure in accordance to the SLA, Actively engages during any high severity issue and drives for issue resolution. reviews technology changes to identify potential risks, As an experienced professional in Big Data Services, he/she: supports his/her team during diagnosis when technical issues rise in his/her scope of expertise, is aware of the global IT structure so that he/she anticipates interrelationships within the organization, engages with technical peer, Development team, Service managers, Architect and project teams on technology roadmap and projects, facilitates transformation projects and suggest future directions for new areas of improvement and change, guarantees the production readiness and license to operate of new projects and solutions is available and able to drive technically, any complex or high severity incidents that occur within the scope of their role technically coach and develop partner resources to improve quality and productivity, Candidate profile Mandatory track record Administer and manage Redis clusters for low-latency caching and real-time transaction processing. Manage MongoDB clusters (replication, sharding) for scalable transaction and semi-structured data storage. Working knowledge of Hadoop ecosystem (Hadoop, Hive, Pig, Oozie, Hbase, Flume, sqoop) using both automated tool sets as well as manual processes. Support and maintain Hadoop (HDP) clusters for batch processing, analytics, and regulatory reporting. Perform cluster lifecycle management: provisioning, scaling, patching, and decommissioning nodes. Ensure 24x7 availability and resilience of production systems supporting payment flows. Manage and optimize Apache Kafka for high-throughput, real-time payment event streaming. Ensure data consistency and fault tolerance across streaming pipelines. Support Apache NiFi for ingestion pipelines from upstream payment systems and external partners. Work with AWS EMR for scalable processing of transaction data and reconciliation workloads. Administer HDFS, ensuring optimal replication, storage utilization, and fault tolerance. Monitor and tune MapReduce and YARN workloads to handle large-scale transaction data efficiently. Ensure proper configuration and validation of jobs handling payment clearing, settlement, and reporting. Manage OpenSearch / Elasticsearch clusters for transaction search, audit trails, and operational dashboards. Optimize indexing and query performance for near real-time analytics and monitoring. Implement Kerberos-based authentication and secure access controls across the Hadoop ecosystem. Manage user provisioning (Linux + Hadoop stack) ensuring least-privilege access. Ensure compliance with banking regulations, audit requirements, and data governance policies. Monitor cluster security, encryption, and network connectivity. Conduct capacity planning aligned with transaction growth and peak payment volumes. Optimize systems for low latency and high throughput, critical for digital payments. Identify bottlenecks and implement performance tuning strategies across platforms. Ensure high availability through failover mechanisms, DR strategies, and proactive monitoring. Develop and maintain runbooks, SOPs, and architecture documentation. Define and enforce best practices for cluster operations, deployments, and data pipelines. Contribute to continuous improvement initiatives and knowledge sharing. Excellent communication, interpersonal and logical skills Customer service oriented and a strong tea…