Post Category → IoT

Experiences with Kafka and exactly-once processing in IoT apps

Some context on message brokers and delivery guarantees (If you have fair amount of experiences with message processing and delivery guarantees please skip to the next part of this post.) Message delivery guarantee is one of the canonical requirements for message brokers and they are very relevant for all types of brokers: the ones based on queue semantics and the ones… Continue reading →

TAPO for Airports – A Streaming usecase

Airports, especially the busy ones face an interesting challenge when it comes to serving the commuters, they need a smoother way to handle passengers in queues without long frustrating waits and thereby elevate the overall experience. No one likes to wait/stand in long queues. But airports, unfortunately, have lots of queues one for check-in, baggage… Continue reading →

Apache Flink CEP and ATM Fraud usecase- Part 2

On the 1st part of this multi-part series on Apache Flink CEP library, I briefly covered the case for a dedicated CEP framework among the toolsets of open-source stream processing frameworks. Quick recap on the use case For a customer, an ATM Withdrawal Txn >= 10,000 made more than ‘3 times’ in a location >  50 mile radius… Continue reading →

“Exactly-once” with a Kafka-Storm Integration

Update 4, Nov 2016: When I first wrote this post it was outright mockery and contempt. But the Google Data flow paper (The Unified google framework for Batch (FlumeJava) and Stream processing (MillWheel)) and the Google MillWheel paper clearly explains that this is exactly the same approach google team has taken to solve the duplicate events problem…. Continue reading →

CoAP can put IoT to REST

HTTP is ok for Web, but for WoT (Web-Of-Things) it cannot be HTTP but WoT still would need a Req/Res paradigm. The answer is UDP based light weight CoAP. CoAP (Constrained Application Protocol) is a specialized wire protocol which is built on top of UDP to provide HTTP/REST like Req/Res paradigm for constrained devices. Basing on… Continue reading →

What wikipedia can’t tell you about Apache storm and Apache spark streaming

I am seeing a lot of questions around Spark streaming and Storm in Quora. When to choose what and what are their performances, reliability and support like. There are a lot of comparisons as usual available in the web , if you google around you could find. But instead comparing them side by side I thought of talking… Continue reading →

What you didn’t know about Real-time notification systems

I have been intrigued by Event Notification systems for a long time now, In fact this started from my programming days in legacy environments like iSeries. So I started working on a toy project which evolved into a solid project. I thought I will muse about that recent project the RealTimeNotification. But before going into the details of the… Continue reading →

RTOS Roundup for IoT

I have been musing about my views on IoT gateways, protocols and how to handle message inflow from different forms of  “things” in IoT. For a change I wanted to focus on the ingredients of a “thing”. “Things” may or may not be constrained or constrained for different things depending on the context. either case the key ingredient is… Continue reading →