Skip to content

Architectural Overview

Jump to bottom Edit New page
davisford edited this page Aug 14, 2012 · 6 revisions

Introduction

The Internet of Things is a collection of technologies that make it possible to connect things like sensors and actuators to the Internet, thereby allowing the physical world to be accessed through software -- Wikipedia Article on Internet of Things

The goal of this page is to give a systematic overview of the code in this repo and how it relates to our Internet of Things IoT solution. Our vision is to provide low cost, simple hardware and a simple software platform to enable all kinds of IoT use cases for tech-savvy and non-tech-savvy users.

A good overview of what we are trying to build here can be seen in our (sadly unsuccessful) Kickstarter Project

If you are interested in helping contribute to this project - drop me a note or head over to the discussion group

The code in this repo provides the server as well as a client webapp for interacting with the server and interacting with devices. Initially, we aim to provide support for the Daisy WiFi devices, but plan to extend it and make it generic enough for other hardware devices.

Any general IoT type system typically involves the following:

  • Small wired or wireless devices (e.g. Arduino) that monitor the environment around them and have a variety of sensors and/or actuators on them
  • A server that provides communication with those devices either passively by receiving updates from them about their status, or bi-directionally allowing commands to be passed to the device for actions
  • Optionally, you may have additional clients involved in the network such as a web client or mobile app client that can communicate in the system by monitoring device status and or issuing commands that are proxied through the server to the device.

Simplified view

Daisy WiFi

A small, low-power WiFi-capable device that may run on battery or AC. There are variations on the hardware but it can be configured for a variety of sensors (e.g. temperature, moisture, motion detection, etc.) and it may have an additional micro-controller for programmability (e.g. attach an Arduino or Raspberry Pi). Communication with the server is done over the HTTP protocol. Most communication is unidirectional with the device posting its state to the server to be recorded at specific intervals or via interrupt, but we also have the ability to send commands back to the device proxied through the server.

The Daisy WiFi incorporates Roving Networks RN-171 WiFly chip PDF User Manual. The RN chip was chosen based on its feature set and cost -- we did look at plenty of alternatives. We generally configure it to use the built-in HTTP client networking for communicating with the server. The device can be configured to periodically wake up on a specified interval and post its state, or it can be triggered to post its state via interrupt.

WiFly HTTP Unidirectional

If the Daisy WiFi has an integrated microcontroller (e.g. Atmel AVR), then we have complete control on what is sent to the server in terms of an HTTP request. If we don't have a microcontroller -- and we're just using the built-in Wi-Fly HTTP networking support, we have little control for how the RN chip posts its state to the server. It is sent via HTTP querystring -- a sample request looks like the following:

GET /wifly-data?DATA=051208BECF29CF29001F09F30AF52EC33DD7&id=Garden&mac=00:06:66:72:10:ec&bss=e0:46:9a:5b:22:ee&rtc=42ab&bat=2621&io=510&wake=2&seq=409&cnt=2&rssi=bc HTTP/1.0\nHost: demo.daisyworks.com\n\n

Unidirection HTTP request / response

The sensor data is encoded in the DATA param. For details on how it is decoded, you can read the source.

Note Roving Networks has an SDK for the chip, but it is not free, and it is a significant investment. If we could procure the funds, we would like to buy the SDK so we have further control over the logic on the RN chip itself including the network code. This is, of course, one of the reasons why we chose to provide integration with another microcontroller -- such as an Arduino / Atmel AVR. Open firmware allows us to have more control over the device -- the down-side is that adding that additional hardware increases our power requirements, and makes the device no longer feasible for long battery-operated use cases (limiting it to wall-power scenarios).

In a typical scenario, the device would create a new TCP connection to the server, post its sensor data via HTTP, and the server would terminate the connection. This provides unidirectional communication, but we would also like to support bi-directional communication so we can tell the device to do things.

WiFly HTTP Bi-directional

There are different ways to solve that. The RN chip does have some capacity to run a TCP / UDP / HTTP server, but having the device run a server itself poses all kinds network security challenges. Ports have to be opened in firewalls, which represent security risks, etc. It is not an optimal approach.

Instead, we cheat a little. When the device connects to send its sensor data, we have the server hold the connection open. This allows us to stream commands back through the TCP connection. When the device receives character data, it looks for the special sequence $$$ -- this puts the device in Command Mode. Once in command mode, we can issue a number of commands to the device (refer to the RN-171 User Manual) such as re-configuring the device or telling the device to drive a signal on a GPIO pin.

The server keeps a cache of device connections. We devise an RPC-style API and a user interface that allow a user to issue commands back to the device through a standard webapp. Commands received through the RPC API are passed to Redis pub/sub -- any node.js server that is holding a connection for the target device will then relay the command direct to the device, and relay the response back through the same path all the way to the user interface.

The Daisy will continue to post its sensor data on a specified interval or via an interrupt on the same TCP connection unhindered -- even while in command mode.

WiFly Bidirectional communication

  1. Daisy Server registers with the Redis server to subscribe to a channel for Daisy WiFi commands
  2. Some time later, Daisy WiFi posts its sensor data to Daisy Server via HTTP either on its specified interval or because it awoke via interrupt
  3. Daisy Server receives sensor data and processes it but does not close the TCP connection
  4. Some time later, Client (user interface / webapp) wants to send a command DO SOMETHING to the Daisy WiFi -- they use the UI to issue the command
  5. The command is received by the Webapp Server, and it is published to a redis pub/sub channel
  6. Redis server relays the command to Daisy Server because it is subscribed to the channel
  7. Daisy Server receives the command, finds the device in its cache of open TCP connections and issues the command to the Daisy WiFi
  8. Daisy WiFi receives the command, processes it, and sends its response (the response is passed all the way back to the client through the same path)
  9. The connection is left open, and some time later Daisy WiFi posts its sensor data again either on a specified interval or because it has awoken via interrupt.

This allows us to do a number of interesting things; it allows user the ability to plan an active role in the IoT system as opposed to just being a passive monitor of device state. If we have a microcontroller attached to the DaisyWifi we can devise our own special commands the trigger the micro to take any action we wish (e.g. drive a servo motor, toggle a relay switch, etc.)

Servers

There are two types of servers in this system:

  • Daisy WiFi Server - provides an HTTP server for receiving sensor data from the devices - in addition it relays commands for the device from the appropriate Redis pub/sub channel
  • Webapp Server - provides the backend for the user interface

Communication between the two servers is done through Redis pub/sub. This facilitates horizontal scalability - we can spin up more instances of each server type as needed. Client/server communication for each server type is not stateless:

  • The Daisy device connects to a single server and leaves the channel open)
  • A user of the webapp will establish a websocket connection via socket.io The communication across servers via pub/sub allows stateful user/sessions to communicate with stateful device/sessions in a decoupled manner. If more resources are needed, it would likely be more Webapp Servers -- since the Daisy WiFi Server processing is fairly light and communication is never likely to come in large bursts inherent in Internet webapps.

Load balancing could be achieved by installing a single endpoint and some type of scheduling algorithm with node-http-proxy

Sample Deployment

Daisy WiFi Server

This is deployed as a separate server instance that listens only for messages from the device. When messages are received, it stores a snapshot of the sensor data in a time-series MongoDB collection. As described above -- it also subscribes to a Redis channel for receiving commands for a particular device when a user wishes to contact a device and issue a command for it to execute.

Webapp Server

Tech Choices

Some rationale...

node.js

We originally built a prototype out in the Java Play Framework. This is the server that is running in the Kickstarter Videos and it is receiving messages from devices, sending commands to devices, and doing things like calling a user's phone via Twilio.

I decided to rebuild the server using node.js. I wanted to learn node.js, and it seemed to be a good fit technically:

  • No CPU-bound operations
  • Mostly a lot of I/O
  • Same language client/server - code re-use, reduce developer context switching, etc.

Turtles all the way down

MongoDB

We also needed persistence -- mainly for three areas:

  1. Sensor Data -- this mainly fits the pattern of time-series data. It will likely represent large volumes of data, isn't likely mission critical if you lose some, is not relational in nature, and we may want to truncate it regularly, and mine it for statistical patterns
  2. User Account / Profile Data -- standard profile stuff
  3. Server Side Rules -- allow a user the ability to derive rules which automate the server taking action when devices reach a certain state -- also not really relational in nature

The characteristics of the data seem to fit nosql, so that is the path I pursued. There is no shortage of nosql choices for node.js. I investigated two: MongoDB and Riak. I'm not going to write a large essay on why I went with MongoDB. I like Riak -- a lot. I built a number of prototypes with it, read through the Riak Handbook, and I experimented quite a bit with it. In the end, the main showstopper was that the maintainer for the node-riak driver had stopped developing it, and no one was really stepping up to the plate to take over. There were some forks and all, and writing or evolving a node/riak driver might be a fun project, but my focus was building this project.

While mongo has its critics, it worked well out of the box for me, and having read the criticism, I still felt comfortable with it. I also went with mongoose which provides a lot of nice functionality on top of the native mongo driver.

SocketStream

After I watched this video I was sold on SocketStream. It seemed to be a perfect match. I'm a fan of single page apps -- put another way: SOFEA / SOUI. Basically, presentation logic is all in the client, and the server merely provides services that the client makes use of for data. I've built a number of these types of apps in other technologies (e.g. Silverlight / Flex / jQuery-UI).

With the introduction of WebSockets, this allows for server push notifications to the client, allowing a near real-time experience. SocketStream provides the glue that ties all this together in a lightweight, flexible manner. There are a number of alternatives to SocketStream, and I have not tried them all, but once I started using it, I didn't want to use anything else. I've since built a couple other SocketStream apps, and it is a joy to work with.

The documentation is always a work in progress, but I make use of nearly all of this

Client

Client-Side Tech Rationale

Twitter Bootstrap

I'm not a skilled designer - it helps a great deal in making layouts bearable with the grid, and provides sensible typography and default styling. You often hear complaints about Bootstrap apps all looking the same - I wouldn't mind tweaking the theme a bit after the functionality were finished, but I aim to finish the functionality first. I also like the built in responsiveness. If I'm somewhat sensible about how the layout is done, the app can be accessed on smaller phone / tablet screens and work reasonably well without too much headache.

Backbone.js

I think something needs to reign in and give structure to the client-side micro-architecture. I've used backbone.js before on other projects, and was happy with it. There are a lot of alternatives today, but I just stuck with what I know.

DataTables

A few places in the app, a table seems to be what is called for:

  • View that shows all my devices
  • View that shows all my rules

I researched various JS table libraries, tried this one, and found it to work quite well.

jsPlumb

I wanted to build a visual rule editor (see below) that had the look/feel of Yahoo Pipes. It is a stretch goal, but I would like to make a user interface that can be consumed by people who may not be all that technically savvy -- I want to allow them to define a rule for a device with the least amount of frustration possible. For example, I want my Dad to be able to define a simple rule that says, "If the temperature in my home drops below 55F, then call my phone".

I looked at a number of JavaScript drawing libraries. I tried some prototypes with a few alternatives, and settled on jsPlumb. It works really well, and I have no complaints. The only catch is that it requires one of:

  • jQuery-UI
  • MooTools
  • YUI3

Each of which is a fairly heavy dependency. I'm familiar with jQuery-UI, and I'm already using jQuery, so that seemed like the path of least resistance. jQuery-UI is one of those client side dependencies that I would like to load on-demand -- only if you load the rule editor will you need to pull it in.

Flot

I wanted some charts -- experimented with Flot...looked at a few others. d3.js looks awesome, but also looks like it has a pretty steep learning curve -- plus it seems a little like overkill for basic visualization. nvd3 looks like a nice alternative to flot, and I'd consider using it.

Core Functionality Provided By User Interface

As a user of this system, I want it to provide the following base functionality:

  1. Basic user profile / login -- OAuth support is nice-to-have (this is mostly done, and OAuth support for Google accounts is implemented. I investigated OAuth support for Twitter / Facebook, but each posed a technical challenge.)
  2. Register / Own a device so I can tie an individual device to my user account. Ideally I'd like to be able to share devices with other users as well, so they could also visualize my data, and even create their own rules against my devices. (this is partly done, the hooks are there to allow anyone to register a device with a one-to-many relationship, although there is currently no sense of roles or property rights -- e.g. Bob can visualize my data, but he cannot write a rule against it)
  3. Ability to have a place for my hardware to send data to, store the data, and allow me to
    • Visualize It - lots of different fun ways to do this with d3.js -- (This is partially done. Right now I am using Flot charts to show basic line series data.) screen shot of flot data
    • Query It - scroll back / forward, query for specific statistical events or sensor aberrations -- (This is partially done. Querying the collections in Mongoose is fairly trivial; right now I just grab the last 25 or so sensor data, and as the device posts fresh data, it is relayed to the client's UI real-time and updated. Additional, more complex query mechanisms can be built into the UI, and server RPC)
  • Ability to create automation rules that are stored on the server which monitor incoming sensor data for my devices. The basic vision for this was built out in a user interface prototype I did for the Kickstarter. This involves a fairly complex UI piece, and a corresponding server-side rules engine that monitors fresh data posts for rules that match some criteria. If the criteria are a match, the server will then take the defined action. (this is not finished, but I am in the process of trying to port the user interface prototype over to the new Twitter Bootstrap environment. Work has not begun yet on the server-side rules engine) The goal is to make the rules engine modular so new actions can be written and plugged in, for example: * Email me when some event occurs * SMS me when some event occurs * Call my phone when some event occurs using API like Twilio * IM me when some event occurs (e.g. GTalk, Skype, Jabber) * Send a command to one or more Daisies (e.g. pulse a signal on GPIO pin) -- see the editor demo, click on Actions, and drag a Signal widget on the canvas for a demo
Add a custom footer
Add a custom sidebar
Clone this wiki locally