Skip to main content

Data on Golain

info

All data flowing to/from a device is transmitted using Protocol Buffers with Golain.

Overview​

One of the key constraints we had in place while developing our platform was to ensure that all systems and services (internal and client-side) that need to interact with any device data must be able to use a extremely well-defined schema. This is to ensure that we can provide a consistent experience across all our services and SDKs.

This constraint led us to choose a universally accepted and loved schema language, and we chose Protocol Buffers for this purpose.

What are Protocol Buffers?​

Protocol Buffers are a language-neutral, platform-neutral, extensible way of serializing structured data for use in communications protocols, data storage, and more. They are designed to be fast, efficient, and easy to use.

Instead of having to deal with string operations with JSON or XML (famously ineffcient on embedded devices), you can define your data schema in a .proto file and then use the protoc compiler to generate code to read and write your structured data to and from a variety of data streams and using a variety of languages.

This is what a typical .proto file looks like:

syntax = "proto3";

message Data { // a message is a collection of name-value pairs
    int32 temperature = 1;  // the number is the `tag` for this field
    int32 humidity = 2;     // tags must be unique (numbers cannot be repeated within a message)
    int32 pressure = 3;     // tags can be any number between 1 and 2^29 - 1
    double latitude = 4;    // tags 1-15 require one less byte to encode
    double longitude = 5;   // be sure to use tags 1-15 for frequently populated fields
    float battery = 6;
}

message DeviceData { // a message can contain other messages
    string device_id = 1; 
    Data data = 2; // this is a nested message
}

### What does that mean for me? 
Your data is now strongly typed, and you can use the generated code to read and write your data to and from a variety of data streams and using a variety of languages.
No more missed JSON keys, or typos in your XML tags - protocol buffers give you classes and methods to read and write your data, and you can use them in any language you want.
Here is what a typical data flow looks like for a device:

```mermaid
sequenceDiagram
    participant Environment
    participant Device
    participant Golain
    Environment->>Device: Sensor Data
    Device->>Device: Serialize data using Protobuf
    Device->>Golain: Encoded data
    Golain->>Golain: Decode data using Protobuf

I already have my data in JSON/XML, how do I use it with Golain?​

Here is a super handy tool that will help you convert your JSON/XML to Protocol Buffers: JSON to Protobuf Definition

Are APIs also represented using Protocol Buffers?​

No, APIs follow a RESTful design and are represented using JSON. We use OpenAPI to define our APIs.
The APIs expose device data in JSON format for easy consumption by client applications.
Protocol Buffers are used primarily for data flowing to and from devices.

What about the Golain SDKs?​

The SDKs all include examples and documentation to help you get started with and use protocol buffers with your devices and mobile applications.
See more in the SDKs section.