IoT-PaaS started reaching the masses, I have already captured the list of both freely and commercially available IoT platforms in my previous post here.  But I wanted to concentrate on the elements of a successful IoT platform in a post. But again  it will be a too much for a single post to cover.Let me break it down into 2 parts. This being the Part 1, I wanted to talk about the heart of the IoT platform which is the “messaging broker”.

But I would like to clear something here and I most certainly don’t want to risk misinforming people. IoT should be viewed as suit of applications and implementation heavily depends on the needs of the business.

Flavour 1: A set of sensors or constrained “things”, Let me give an example – a heat sensor with a coin cell battery with no access or capacity to do IP (Wi-Fi/GSM) connected together in WPAN all reporting to a intermediate IoT gateway via non-IP protocols like ZIGBEE or MQTT-SN or Compressed-IP protocols like Thread/6LoWPAN

Flavour 2: A set of “things” or sensors not necessarily constrained as in the case of Flavour 1, Let me give an example – A GPS sensor with GSM SIM connected to a OBDII port of a car reporting location data via IP like MQTT or TCP or UDP to a Cloud application over WAN. Here there is no constraint in terms of power as it is derived front the vehicles ECU.

Both Flavour 1 and Flavour 2 are IoT and M2M respectively, just that the implementation is different and they are solving 2 different business problems but same IoT platform can back these 2 flavours. In both cases IoT platforms simply connect “things”, take the reported data and feed it to the downstream systems for deeper processing. So “things” over WAN need to connect to this platform and exchange messages. So essentially this platform has provide a firehose and a broker to help devices post messages and optionally get a ackonwledgement.

Key Ingredients of a IoT Platform

• Multi IoT IPfied protocol support ( protocol like  STOMP, MQTT, CoAP, 6LoWPAN etc..)
• A durable firehose with support for guaranteed delivery  semantics (Atleast once, Atmost once, Exactly once)
• Distributed Pub/Sub.
• Clustering and Federation support.
• Minimum Scale of messages delivered is “in the multiples” 10k/s
• Mechanism to acknowledge

Hearing the firehose ingredient  most people would go – Apache Kafka !!

While you could build something on your own to support all this, of late if you float this requirements to some experts dealing with massive distributed pub/sub solutions Apache Kafka would fly out of their mouth right away !. That isn’t wrong infact thats one of the possible solutions, comes with a great scale of about 100k/s.

The Trade-off

Lets zoom into “build vs leverage open source” debate. If we have to build this using Kafka or any other equivalent “producer centric” pub/sub broker all the support for IoT protocols has to be some how hooked into this mix.  Like I said in the case of “producer centric” brokers like Kafka the actual constrained device itself isn’t connected to the broker. So sending an ACK back to the actual sensors/devices needs an intermediate layer with all the “guaranteed delivery” dynamics built into it.

I am not saying this is hard or debating against this way of doing it. Its just not trivial and needs work.

RabbitMQ: The Polyglot Broker

A lot of people might know RabbitMQ as a Message oriented or queue centric middleware. In a way we can say Rabbit is one of the modest brokers out there. RabbitMQ has a  very unique feature. The end points talking to RabbitMQ can talk the protocol they are comfortable with, it will translate the protocol internally to suit end point. This is a very powerful feature. RabbitMQ comes with MQTT plugins and  STOMP plugins apart from the AMQP and HTTP support. Here is an article with some experiments on this feature.

Here is another good Quora debate RabbitMQ vs Apache Kafka

RabbitMQ has also set a new benchmark hitting 1 million messages/s, read.  So my recommendation is if supporting multiple constrained devices is the centre of all requirements go for a Polyglot broker like RabbitMQ (I am not promoting RabbitMQ ;-)). if centre of all requirements is pure distributed pub/sub with a layer that needs to act as a firehose shielding and routing messages to variety of subscribers go for a solution like kafka.

Key Take-away ?

If you were to build a IoT-PaaS yourself,  IoT-PaaS needs a durable firehose like Kafka and a polyglot broker like RabbitMQ. Emphasis is on the word “like” and not recommending these products, so whatever variant of these capabilities works for you.That said we should touch upon of the other elements of an IoT platforms. That will include the persistence layer, high speed routing and adapting with existing backend solutions. I will cover that in part 2. Do share your thoughts below.