Of Crickets and Continuous Integration

A story of lending a hand, exploring technology, and scope creep.

Presented by Zackary Lowery
on January 24th, 2019 at Leading EDJE and on April 26th, 2019 at StirTrek.

Available online at https://presentations.xcjs.com/

Press your space bar or swipe to navigate down and then forward. ยป

Disclaimer

What follows may not be definitive of best practices for IoT development.

It was a lot of fun, though.

Topics

  • IoT with the Raspberry Pi Zero W
  • Node.js
    • Interactive CLI Application Development
    • Dependency Injection
    • Unit Testing
    • Hardware Integration
    • Cross-platform Development
  • GitLab
    • GitLab CI with the GitLab Runners
Raspberry Pi Zero W

The Story

Anthony Anthony
Disclaimer: Any and all conversations in this presentation may be paraphrased.

Hey man, I owe my cousin a favor and need some help.

What's the favor?

So, my cousin owns this cricket farm in Montana...

Cricket? Like the insect?

Yeah! Can you upload temperature data to AWS from a Raspberry Pi?

Probably? I'll see what I can do.

Thanks! Let me know what I owe you.

Oh, I will.

The Adventure Begins

Technical Requirements

DHT-22 Temperature / Humidity Sensor MQ2 Gas Sensor AWS SDK
DHT-22 Sensor MQ2 Sensor AWS Logo
Node.js Logo
Python Logo
Node.js Logo
Python Logo
Node.js Logo
Python Logo

...and more.

Libraries

DHT-22 Sensor

DHT-22 Sensor

node-dht22

Native

MQ2 Sensor MQ2 Sensor

MQ-2 Sensor/ADS1115

node-ads1x15

Native

AWS Logo

AWS SDK

aws-sdk

JavaScript

Supported Hardware

Raspberry Pi Zero W

R-Pi Zero W

  • BCM2835 SoC
  • ARMv6
  • No hardware floating point operations.
Raspberry Pi 3

R-Pi 3

  • BCM2835 SoC
  • ARMv8
  • armhf (hardware float)
x86 Architecture x86_64 Architecture

x86/x84_64 (somewhat)

  • Mock hardware sensors injected

Hardware Utilization and Services

BCM2835 Support

napa - A helper for installing stuff without a package.json with npm.

package.json


						{
							...
							"scripts": {
								"install": "napa",
								...
							},
							...
							"napa": {
								"bcm2835": "http://www.airspayce.com/mikem/
									bcm2835/bcm2835-1.55.tar.gz",
								...
							}
						}
					

Reading the DHT-22

TemperatureSensorReader.js


						var deferred = new Promise(() => (resolve, reject) {
							temperatureSensor.read(22, 22, // sensor model, pin
								(err, temperature, humidity) => {
									if (!err) {
										resolve({
									        temperature,
									        humidity
										});
									} else { reject(err); }
							});
						});
					

Mocking the DHT-22

MockTemperatureSensor.js


						function read(dhtModel, gpio, callback) {
							// ...

							var temperature = getRandomInt(15, 60);
							var humidity = getRandomInt(0, 100);

							// ...

							setTimeout(() => {
								callback(err, temperature, humidity);
							}, 2000); // Simulate maximum 2s readout interval.
						}
					

Reading the MQ-2

GasSensorReader.js


						var deferred = new Promise(() => (resolve, reject) {
							gasSensor.readADCSingleEnded
								(channel, progGainAmp, sampleRate,
									(error, data) => {
										if (!error) { resolve(data); }
										else { reject(error); }
									}
								);
						});
					

Mocking the MQ-2

MockGasSensor.js


						function readADCSingleEnded(
							channel, progGainAmp, sampleRate, callback) {
							// ...

							var gasLevel = getRandomInt(0, 65000);
							callback(null, gasLevel);

							// ...
						}
					

Persistence

DynamoStorage.js


						var docClient = new aws.DynamoDB.DocumentClient({
							apiVersion: '2012-08-10'
						});

						var deferred = new Promise(() => (resolve, reject) {

							docClient.put(params, () => (error, response) {
								if (!error) { resolve(response); }
								else { reject(error); }
							});

						});
					

The Requirements Have Been Altered - Pray I Do Not Alter Them Further

The prototype uploaded data, but I had to use the desktop environment to connect to my WiFi. Is there a better way to do that?
I'll see what I can do.

Requirements

  • Provide WiFi configuration through this application (now dubbed "Elise")
  • Constrain users to Elise
  • Improve responsiveness compared to the full desktop environment

Accepting User Input

https://www.npmjs.com/package/inquirer

Elise's Main Menu

MainView.js


						const choices = [
							'Manage WiFi Connection',
							'About Elise'
						];

						const question = {
							name: 'main',
							message: 'Select an option below using the arrow keys ' +
								'on your keyboard and then press Enter: ',
							type: 'list',
							choices: choices
						};

						return inquirer.prompt([question]);
					

WiFi Configuration

node-wifi Elise's WiFi Menu

WifiController.js


					var deferred = new Promise(() => (resolve, reject) {
					  wifiService.scan(() => (err, networks) {
					    if (!err) {
						  view.showMainMenu(networks).then
					        (promptWifiPassword, showMainMenu).then(
						      connectToWifi, checkIfBack).then(
						        () => (connectedNetwork) {
					              resolve(connectedNetwork);
								}, () => (err) {
					              reject(err);
					            });
					    } else {
						  reject(err);
					}});});
					

The Deal is Altered Further

I got the WiFi working, but I don't know what to do with all these files, and they take forever to copy.
Can't you make like an .exe or something?
Maybe? I'm not sure.

Nexe

Create a single executable out of your node.js apps

https://github.com/nexe/nexe

Animaed Nexe Example

package.json


						// ...
						"scripts": {
							// ...
							"build": "npx nexe --build index.js -o bin/elise",
							// ...
						}
					

						$ npm run build
					

Nexe Benefits

  • Single-file deployment
  • Includes embedding native modules (*.node files)
  • Bundles known working version of the Node runtime with your application
  • Runtime can be upgraded (or reverted) anytime during a standard deployment
GitLab Logo

GitLab and Continuous Integration

GitLab

GitLab is a single application for the entire software development lifecycle. From project planning and source code management to CI/CD, monitoring and security.

Gitlab Runner

GitLab Runner Logo

CI Pipelines

Auto DevOps

or

.gitlab-ci.yml

Auto DevOps

Pre-configured pipeline centered around scalable web applictions.

.gitlab-ci.yml

In-repository, per-branch configuration file for custom CI pipelines.

GitLab CI Structure

  • Pipeline
    • Stages
      • Jobs

Elise's .gitlab-ci.yml

Stages


						stages:
						- Setup
						- Analyze
						- Test
						- Build
						- Stage
					

Cache Configuration


						cache:
						  untracked: true
						  key: "$CI_COMMIT_REF_NAME"
						  paths:
						    - node_modules/
					

Setup


						Setup:
							stage: Setup
							script:
								- npm install
								- cd ./node_modules/bcm2835
								- ./configure
								- make
								- sudo make check
								- sudo make install
								- cd ../../
								- npm rebuild
					

Analysis


						Analysis:
							stage: Analyze
							script:
								- npm run lint
								- npm audit
					

Test


						Test:
							stage: Test
							script:
								- npm test
					
package.json

						// ...
						"scripts": {
							// ...
							"test": "npx nyc --reporter=html --reporter=text mocha
								--recursive",
							// ...
						},
						// ...
					

Build


						Build:
							stage: Build
							script:
								- npm run build
					
package.json

						// ...
						"scripts": {
							// ...
							"build": "npx nexe --build index.js -o bin/elise",
							// ...
						},
						// ...
					

Stage


						Virtual Staging:
							stage: Stage
							before_script:
								- vagrant plugin install vagrant-vbguest
								- vagrant box update
							script:
								- vagrant up
							after_script:
								- vagrant destroy --force
					

Tags

ARMv6/ARMv8

  • Setup
  • Analyze
  • Test
  • Build

x86_64

  • Virtual Staging

GitLab Runners

ARMv6 / RUNNERPIZERO
 

ARMv8 / RUNNERPI
 

x86_64 / USERV
(Ubuntu SERVer)

More Tags

Runners and their associated tags.

Final Pipeline

Final Pipeline Screen Shot

Next Steps

Docker Logo Docker / DockerHub
(Free distribution network!)
PiBakery Screen Shot PiBakery

Questions and Comments

Return to the rest of the presentations.