Create Our First Microservice
JavaScript Edition
Node.js is a JavaScript runtime built on Google Chrome's V8 JavaScript engine. In this step we're going to build a simple microservice using Node.js and the ioFog SDK.
This tutorial assumes a working knowledge of Node.js and modern JavaScript.
Use Case
To keep things fairly simple, we'll want to have our microservice do something interesting, but not particularly complex. To get a feel for the primary functions of the SDK, we want a microservice that:
- Uses dynamic configuration variables at runtime
- Takes input from another microservice
- Does some processing on it
- Outputs new data from the results
Let's build a microservice that computes a real-time moving average from the input, which will send the result to any other microservices that might be listening. We'll also set up a dynamic configuration of the rolling window size. We can later change the configuration without needing to restart anything.
If we're in a hurry, we can skip ahead to the end.
Project Setup
Since we're going to be writing a new microservice, we'll need to create a project directory. Let's create it inside our tutorial's previous working directory that we should already be in.
Let's name our project "moving-average" and create our service's "main" entry point.
mkdir moving-average
cd moving-average
touch index.js
We then run npm init to set up our default Node.js package.json, providing the answers to all its questions and setting "main": "index.js"
npm init
Now we need to install the ioFog SDK for Node.js, which is published to NPM as @datasance/iofog-nodejs-sdk:
npm install --save @datasance/iofog-nodejs-sdk
The package.json file should look something like this now:
{
"name": "moving-average",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC",
"dependencies": {
"@datasance/iofog-nodejs-sdk": "3.0.1"
}
}
Using ioFog SDK
Before we start writing the code for our microservice, lets take a look at the SDK's APIs.
The ioFog Node.js SDK has a number of APIs, but in this tutorial we're most interested in these ones:
iofog.init()iofog.getConfig()iofog.wsControlConnection()iofog.wsMessageConnection()iofog.ioMessage()iofog.wsSendMessage()
First, include the ioFog SDK in our index.js.
const iofog = require('@datasance/iofog-nodejs-sdk');
iofog.init()
The callback we provide should be treated for mostly as a pseudo entry point of our microservice. We have to make sure we don't call any SDK APIs before this function has been called!
We now have to register a callback for ioFog once the ioFog SDK has finished initializing. It accepts a number of arguments, but we'll most likely want to pass these defaults. Here we register main() function as the init callback.
iofog.init('iofog', 54321, null, main);
});
For the curious, the first argument is the host name of the Agent's Local API, the second is the port number of the SDK, and the third can be the container's ID, though it is not required.
iofog.getConfig()
Asynchronously fetch the microservice's current configuration (config). When it starts, the configuration of the microservice is only read once.
function updateConfig() {
iofog.getConfig({
onNewConfig: newConfig => {
config = newConfig;
},
onBadRequest: err => console.error('updateConfig failed: ', err),
onError: err => console.error('updateConfig failed: ', err)
});
}
iofog.wsControlConnection()
Note that when a configuration of a microservice changes, the Controller will send a message to the involved microservice.
Therefore we have to connect the ioFog control channel via WebSocket, which is used to receive notifications from the Controller that our microservice's config has changed.
Because a config can be any arbitrary JSON, including very large files, the change notifications themselves do not actually include the config. So if we would like to update our local cache of the config, we have to follow up a change notification with a call to iofog.getConfig().
iofog.wsControlConnection({
onNewConfigSignal: () => updateConfig(),
onError: err => console.error('Error with Control Connection: ', err)
});
iofog.wsMessageConnection()
Next, we have to connect to the ioFog message channel via WebSocket. This is where we'll receive any messages routed to this microservice from another.
Under the hood, communication is brokered by our Router and messages are routed according to that microservice's route settings on the Controller.
iofog.wsMessageConnection(onMessageConnectionOpen, {
onMessages: messages => {
// Do something with messages...
},
onMessageReceipt: (messageId, timestamp) => {
console.log('message receipt: ', {
messageId,
timestamp
});
},
onError: err => console.error('Message WebSocket error: ', err)
});
iofog.ioMessage() / iofog.wsSendMessage()
Now that we can read control signals and message, we also need to send messages out with the actual moving average. We create and send ioMessages in JSON, which is the Node.js serialization format used for intercommunication between microservices.
When our code wants to publish a message to any other microservice, these are what we'll be sending.
There are a number of optional fields, but the most common are: contentdata, infotype, and infoformat. The contentdata field is the actual data payload we want to send, which needs to be provided as a string.
const output = iofog.ioMessage({
contentdata: Buffer.from(JSON.stringify(result)).toString(),
infotype: 'application/json',
infoformat: 'text/utf-8'
});
iofog.wsSendMessage(output);
ioMessages can contain a number of options and fields not described here. To learn more, check out the Local API Reference.
Putting The Moving Average Together
We're ready to start writing some code! First, lets open (or create) the index.js file we set as our package.json "main". This is where we'll place all of our code.
Before we begin, let's review our goals for our moving average microservice:
- Window size should be configurable
- Take input from another microservice
- Compute a moving average on that input
- Output new data from the results
We'll want to have our microservice expect a custom config with a maxWindowSize field telling us what the max size of our rolling window should be.
To compute our real-time moving average, we can first create some utilities to compute an average from any array of numbers:
const sum = values => values.reduce((a, b) => a + b, 0);
const average = values => sum(values) / (values.length || 1);
average([2, 6, 14, 53, 87]); // returns 32.4
In order to do a rolling window, we'll store incoming values in an array up until the point where a max window size is reached. After which we'll discard the oldest value, computing a new average each time.
function getMovingAverage(arr, newValue) {
// Evict the oldest values once we've reached our max window size.
while (arr.length >= config.maxWindowSize) {
// <------- config
arr.shift();
}
arr.push(newValue);
return average(arr);
}
The Sensors microservice produces objects that look like this:
{
"time": 1540855847710,
"speed": 41.71445712,
"acceleration": "0.52431",
"rpm": "2078.3"
}
What we'll do is produce an average for speed, acceleration, and rpm.
onMessages: messages => {
if (messages) {
for (const msg of messages) {
const input = JSON.parse(msg.contentdata.toString());
// Produce moving averages for all the sensor values
const result = {
isAverage: true,
time: input.time, // same time as
speed: getMovingAverage(prevSpeeds, parseFloat(input.speed)),
acceleration: getMovingAverage(
prevAccelerations,
parseFloat(input.acceleration)
),
rpm: getMovingAverage(prevRpms, parseFloat(input.rpm))
};
const output = iofog.ioMessage({
contentdata: Buffer.from(JSON.stringify(result)).toString(),
infotype: 'application/json',
infoformat: 'text/utf-8'
});
iofog.wsSendMessage(output);
}
}
};
The Final Moving Average Code
We now have everything we need to complete our microservice!
const iofog = require('@datasance/iofog-nodejs-sdk');
// Used as our in-memory cache of our configuration
// that will be provided by the Controller
let config = {
maxWindowSize: 150 // Default value in case no config is provided
};
function updateConfig() {
iofog.getConfig({
onNewConfig: newConfig => {
config = newConfig;
},
onBadRequest: err => console.error('updateConfig failed: ', err),
onError: err => console.error('updateConfig failed: ', err)
});
}
const sum = values => values.reduce((a, b) => a + b, 0);
const average = values => sum(values) / (values.length || 1);
function getMovingAverage(arr, newValue) {
// Evict the oldest values once we've reached our max window size.
while (arr.length >= config.maxWindowSize) {
// <------- config
arr.shift();
}
arr.push(newValue);
return average(arr);
}
// This is basically our "entry point", provided to iofog.init() below
function main() {
updateConfig();
iofog.wsControlConnection({
onNewConfigSignal: () => updateConfig(),
onError: err => console.error('Error with Control Connection: ', err)
});
const onMessageConnectionOpen = () => {
console.log('Listening for incoming messages');
};
// Cache for our previous values received so we can compute our average
const prevSpeeds = [];
const prevAccelerations = [];
const prevRpms = [];
iofog.wsMessageConnection(onMessageConnectionOpen, {
onMessages: messages => {
if (messages) {
for (const msg of messages) {
const input = JSON.parse(msg.contentdata.toString());
// Produce moving averages for all the sensor values
const result = {
isAverage: true,
time: input.time, // same time as
speed: getMovingAverage(prevSpeeds, parseFloat(input.speed)),
acceleration: getMovingAverage(
prevAccelerations,
parseFloat(input.acceleration)
),
rpm: getMovingAverage(prevRpms, parseFloat(input.rpm))
};
const output = iofog.ioMessage({
contentdata: Buffer.from(JSON.stringify(result)).toString(),
infotype: 'application/json',
infoformat: 'text/utf-8'
});
iofog.wsSendMessage(output);
}
}
},
onMessageReceipt: (messageId, timestamp) => {
console.log('message receipt: ', {
messageId,
timestamp
});
},
onError: err => console.error('Message WebSocket error: ', err)
});
}
iofog.init('iofog', 54321, null, main);
Create Dockerfile
We now to need to package up our code as a Docker image, so that we can deploy it in the next step. Docker images are created from instructions written in a Dockerfile.
Like all build scripts, Dockerfiles can become a bit complex for advanced applications, but fortunately, ours is fairly simple:
echo "FROM node:14
WORKDIR /moving-average
COPY ./package.json .
RUN npm install --only=production
COPY index.js .
CMD node ." > Dockerfile
In case you are not familiar with Dockerfile, here is a quick run down of the building steps:
- Use a public base image
node:14to start with - Set the image current working directory to
/moving-average - Copy
package.jsonfrom the current local folder into/moving-averageinside the image - Run
npm install --only=production, which will install all requirednode_modulesfor production - Copy
index.jsfrom the current local folder into/moving-averageinside the image - When a container is run using this image, start the container by running the following command
node .
Build Our Docker Image
With our Dockerfile setup, we can go ahead and build our image:
docker build --tag iofog-tutorial/moving-average:v1 .
Sending build context to Docker daemon 7.8MB
Step 1/6 : FROM node:8
8: Pulling from library/node
092586df9206: Pull complete
ef599477fae0: Pull complete
4530c6472b5d: Pull complete
d34d61487075: Pull complete
87fc2710b63f: Pull complete
e83c771c5387: Pull complete
544e37709f92: Pull complete
3aaf6653b5f3: Pull complete
1fed50f6e111: Pull complete
Digest: sha256:c00557b8634c74012eda82ac95f1813c9ea8c152a82f53ad71c5c9611f6f8c3c
Status: Downloaded newer image for node:8
---> 7a9afc16a57f
Step 2/6 : WORKDIR /moving-average
---> Running in 640e44403abe
Removing intermediate container 640e44403abe
---> 9b9c5c8036d5
Step 3/6 : COPY ./package.json .
---> ebe7bf4fd2cf
Step 4/6 : RUN npm install --only=production
---> Running in d2cd4b22f27e
npm WARN [email protected] No description
npm WARN [email protected] No repository field.
audited 176 packages in 1.331s
found 9 vulnerabilities (6 moderate, 3 high)
run `npm audit fix` to fix them, or `npm audit` for details
Removing intermediate container d2cd4b22f27e
---> 7818facf5c9c
Step 5/6 : COPY index.js .
---> da965f5084b9
Step 6/6 : CMD node .
---> Running in 91a726deaea4
Removing intermediate container 91a726deaea4
---> a1b78cc399d1
Successfully built a1b78cc399d1
Successfully tagged iofog-tutorial/moving-average:v1
We'll wait a few minutes while it downloads a default Node.js environment we're using as a base.
Let's double check the images were successfully created. The image name and tag are important in the next step of the tutorial, where we are going to deploy the moving average service.
docker image ls --filter 'reference=*/moving-average'
REPOSITORY TAG IMAGE ID CREATED SIZE
iofog-tutorial/moving-average v1 5bf0943c4cd2 2 minutes ago 904MB
Deploy Our Microservice
We now want to see this code in action, so let's go ahead and learn how to deploy this microservice to our ioFog tutorial environment.
Continue To Next Step: Deploy Our Microservice.
If you ran into an issue, have a question, or just want to get plugged into the community, head over to our Discussion Forum. We'd love to have you!