Messaging Engineering- Messaging Infrastructure & Real-Time Data Exchange

Role: Messaging Engineering- Messaging Infrastructure & Real-Time Data Exchange JD: Skills: Middleware, Apache Kafka, Java or Python etc Core Responsibilities: Design, implement, and maintain the messaging infrastructure that underpins real-time data exchange between factory automation systems, the MES, and enterprise IT platforms. Deploy and manage distributed message brokers (such as Apache Kafka clusters or MQTT-based systems) to enable streaming communication from shop-floor equipment and sensors to cloud or data center applications. Define and govern message topics/channels and data schemas to ensure consistent structure for all events (e.g. equipment status updates, test results, alarm signals) flowing through the system. Optimize the messaging setup for low latency and high reliability, so that critical events propagate instantly and without loss or duplication. Implement monitoring and alerting for the messaging pipeline – tracking metrics like throughput, lag, and error rates – and quickly troubleshoot any issues in message flow health. Collaborate with other developers to integrate their applications with the messaging layer, and assist in tuning producer/consumer performance to meet factory throughput demands. Must Have Skills: Strong expertise in message-oriented middleware and streaming platforms. Hands-on experience deploying or managing Apache Kafka (preferred) or similar technologies (e.g. RabbitMQ, IBM MQ, Apache Pulsar) in a production environment– including topic design, partitioning, replication, and broker configuration. Proficiency in designing real-time data pipelines with pub/sub patterns, ensuring high throughput and fault tolerance (e.g. using Kafka Connect or MQTT clients to interface with devices and applications). Solid programming skills (Java/Scala or Python) for developing producers and consumers, and optimizing them for performance. Familiarity with messaging system internals (broker clustering, retention policies, consumer groups, offset management) to fine-tune the system for optimal delivery guarantees. Good to Have Skills: Experience with messaging in industrial/IoT contexts, such as using MQTT or AMQP protocols to collect data from PLCs or IoT sensors. Knowledge of data serialization formats and schema management, for example, using Avro or Protocol Buffers and a Schema Registry to enforce consistent message structure across services. Proficiency in monitoring tools and practices for streaming systems (e.g. Kafka monitoring via Prometheus/Grafana, or cloud services monitoring) and experience setting up alerts/dashboard for proactive incident response. Familiarity with other integration patterns (like enterprise service bus architectures or event-driven microservices) and understanding of how messaging integrates with the overall factory IT landscape (MES, databases, analytics) would be beneficial.