chore(exec/temp):Fix CSV fields in stdout output

[EVA-2020-02.git] / doc / setup / README.org

#+TITLE:Ninfacyzga-1 Main Setup
#+AUTHOR:Steven Baltakatei Sandoval
#+EMAIL:baltakatei@gmail.com
* Main Setup
** About
This document created by [[http://baltakatei.com][Steven Baltakatei Sandoval]] on
~2020-10-07T18:39Z~ under a [[http://creativecommons.org/licenses/by-sa/4.0/][CC BY-SA 4.0]] license and last updated on
~2020-10-17T21:22Z~.

This document contains information regarding setup of the
ninfacyzga-01 hardware common to all operation modes. This includes:

- Raspberry OS installation
- WiFi configuration
- Remote SSH login configuration

** Scope
This document describes hardware and software installation steps
common to the various environmental sensing functions of
ninfacyzga-01.

** Narrative
The Raspberry Pi Zero W is the platform in which environment data is
gathered, packaged, and stored for further forwarding to a remote
repository. The Raspberry OS 10 operating system is used. The device
may be equipped with a UPS module in order to allow it to function as
a mobile device for short periods of time. The system may use
executables such as ~bklog~ to append segments of observed compressed
(~gzip~) encrypted (~age~) data to a ~tar~ archive to local disk. This
document describes hardware and software configuration procedures
generally required by all environment sensing operations.

** Description
*** Hardware
**** Raspberry Pi Zero W
See the [[https://www.raspberrypi.org/pi-zero-w/][OEM]] webpage for this product.
**** PiZ UpTime 2.0
See the [[https://alchemy-power.com/piz-uptime-2-0/][OEM]] webpage for this product.

*** Software
~bklog~ : A bash script that saves its stdin stream to a tar file. The
file may be compressed by ~gzip~ and encrypted by ~age~. It is an
executable file contained within this repository at ~exec/bklog~. It
should be copied to ~$HOME/.local/bin~.

~bkgpslog~ : A legacy bash script similar to ~bklog~ but narrower in
scope in that it only records output from ~gpspipe~.

~gzip~ : A simple command line app that compresses stdin into a
smaller stdout stream.

~age~ : A simple command line app that encrypts stdin against public
keys specified in its options. Produces encrypted stdout. Is an
executable file contained within this repository at ~exec/age~. It
should be copied to ~$HOME/.local/bin~.

*** Output
**** Encryption Method
Files produced by the bklog script are encrypted against a set of
public keys using [[https://github.com/FiloSottile/age][~age~]], a simple command line encryption tool
selected over ~gpg~ because of ~age~'s deliberate lack of
configurability.

The public keys are bech32 strings supplied as options to bkgpslog
when called. The secret key should *NOT* be stored in Ninfacyzga-01.

If a key pair was generated using ~age-keygen~, then it is an [[https://en.wikipedia.org/wiki/Curve25519][~X25519~]]
key pair. See the [[https://age-encryption.org/v1][~age~ Version 1 specification]].

An ~ssh-rsa~ or ~ssh-ed25519~ SSH public key string may be used instead of
the bech32 public key string produced by ~age-keygen~ for convenience.

Help information for ~age~ is available by running ~$ age --help~.
***** Encryption Commands
****** Encryption through ~age~
In order to illustrate how ~bklog~ encrypts files, below is an example
command illustrating how ~age~ may be used to encrypt a file.

#+BEGIN_EXAMPLE
$ echo "asdf" | age -r \
age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \
> "$HOME/secret_file"
#+END_EXAMPLE

The resulting ~secret-file~ is a binary blob with a plaintext header
indicating how the blob was encrypted (which version of age was used,
which public key was used).

****** Encryption through ~bklog~
~bklog~ may instructed to encrypt files via the ~-e~ and ~-r [pubkey
string]~ options. An example is shown below:

#+BEGIN_EXAMPLE
$ gpspipe -r | bklog -e \
-r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \
-r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \
-r age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 \
-o "$HOME/Location"
#+END_EXAMPLE

~bklog~ may be instructed via the ~-e~ and ~-R~ options to watch a
directory in order to locate public key strings in its files. ~bklog~
reads the first line of each file and interprets it as a public key
string.

In this example, the strings beginning with ~age1...~ are
bech32-formatted public key strings. Please see the [[*Key Generation][Key Generation]]
section for an explanation.

Since ~age~ also accepts ~ssh~ public key strings, these may also be
used if they are of the following form (no comment).

: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABA…AACAQDLnJbPs7CjwPT+OxXd

***** Decryption Commands
Files may be decrypted using a command similar to:

#+BEGIN_EXAMPLE
cat location.gpx.age | age -d -i key.txt > location.gpx
#+END_EXAMPLE

The version of ~age~ used to perform the encryption 

** Operating Procedures
*** Initial Startup
**** Physical Setup
The device should be supplied with 5V power and an SD card with the
latest Raspberry Pi OS image installed. As of 2020-10-07, this will be
version 10 (e.g. Raspbian Buster 10).

No additional hardware (ex: GPS module, UPS module, thermocouples) is
required to perform actions described in this document

**** Software Setup
***** Install Operating System
Install Raspberry Pi OS onto an SD card image. See the Raspberry Pi
Foundation [[https://www.raspberrypi.org/documentation/installation/installing-images/README.md][installation instructions]].

Note: "Raspberry Pi OS" is the name used by the Raspberry Pi
Foundation to refer to their operating system images to be installed
on Raspberry Pi hardware. The change was made in order to facilitate
education of beginners not familiar with the wordplay between
"Raspberry" and "Debian". See [[https://www.raspberrypi.org/forums/viewtopic.php?f=66&t=275380&sid=1a468f226394ccddf4654a3d3d90cb7d#p1668466][this]] forum post made on 2020-05-28 by
plugwash.

***** Configure Wireless
Configure WiFi in order to permit file transfer and remote
administration. For a Raspberry Pi W, the WiFi settings may be
programmed via a specific text file in the `boot` partition of a
freshly installed image of Raspberry OS. Raspberry Pi Foundation
instructions [[https://www.raspberrypi.org/documentation/configuration/wireless/headless.md][here]].

In summary, create a ~wpa_supplicant.conf~ file containing the
following text:
#+BEGIN_EXAMPLE
ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1
country=US

network={
 ssid="<Name of your wireless LAN>"
 psk="<Password for your wireless LAN>"
}
#+END_EXAMPLE

Replace ~<Name of your wireless LAN>~ with your WiFi network's SSID.

Replace ~<Password for your wireless LAN>~ with your WiFi network's
passphrase.
***** Enable Remote SSH Login
Configure SSH to permit remote administration via the command line
interface. Raspberry Pi Foundation instructions [[https://www.raspberrypi.org/documentation/remote-access/ssh/README.md][here]].

In summary, remote SSH access may be enabled upon initial startup of a
freshly installed image of Raspberry Pi OS by making sure an empty
file named ~ssh~ is present on the ~boot~ partition.

***** Login via SSH
Assuming your router supports finding your Raspberry Pi via its
default hostname of `raspberypi`, log into the pi via Wi-Fi using the
following command:

: $ ssh pi@raspberrypi

Otherwise, you may have to identify the raspbery pi's IP address via
your network router's administration console and login via a command
resembling this:

: $ ssh pi@192.168.x.x

If you had previously set up a different raspberry pi that also used
the same hostname ~raspberrypi~ or the same IP address (ex:
~192.168.123.123~), you may have to inform your computer that this is a
different device. You may do so using these commands:

: $ ssh-keygen -f ~/.ssh/known_hosts -R "raspberrypi"
: $ ssh-keygen -R 192.168.123.123

***** Add SSH public key
If the use has an SSH public key, it may be added as a line in
~~/.ssh/authorized_keys~.

Add the ~~/.ssh~ directory if it doesn't already exist.

: $ mkdir ~/.ssh

Follow [[https://superuser.com/a/925859/][these]] directions to set permissions.

: $ chmod 700 ~/.ssh
: $ chmod 644 ~/.ssh/authorized_keys

***** Change default passphrase
The default username is ~pi~ and the default passphrase is
~raspberry~. Change them to something unique.

: $ passwd

***** Update software
Update software with distribution repository.

: $ sudo apt update
: $ sudo apt upgrade -y
: $ sudo apt dist-upgrade -y

***** Change time zone
The time zone should be set to "UTC" for simplicity.

: $ sudo raspi-config

Navigate to ~4 Localisation Options~, ~I2 Change Time Zone~, ~None of the above~, ~UTC~.

***** Update hostname
A unique hostname is required to uniquely identify the device on the
network.

Start up the Raspberry Pi Software Configuration Tool by running:
: $ sudo raspi-config

- Select `2 Network Options`
- Select `N1 Hostname`

This document recommends a hostname beginning with the prefix:
: ninfacyzga-1-

An example hostname would be ~ninfacyzga-1-2~.

***** Install software
****** ~unattended-upgrades~
Make sure to install the ~unattended-upgrades~ package to make sure
the latest security patches for packages are installed. See [[https://linux-audit.com/using-unattended-upgrades-on-debian-and-ubuntu/][this page]]
for a description of how ~unattended-upgrades~ works.

: $ sudo apt install unattended-upgrades

The configuration file is located at:
~/etc/apt/apt.conf.d/50unattended-upgrades~ ([[https://linux-audit.com/using-unattended-upgrades-on-debian-and-ubuntu/][ref]]). Make sure that the
following lines are present and not commented out.

#+BEGIN_EXAMPLE
Unattended-Upgrade::Automatic-Reboot "true";
#+END_EXAMPLE

****** ~syncthing~
Install ~syncthing~ for log file transfer capability.

: $ sudo apt install syncthing

Enable automatic startup. (See [[https://docs.syncthing.net/users/autostart.html][ref]]).

: $ sudo systemctl enable syncthing@pi.service
: $ sudo systemctl start syncthing@pi.service

The WebUI of the local instance of syncthing (port 8384) can be
accessed by running the following command from a separate machine:

: $ ssh -L 127.0.0.1:8390:127.0.0.1:8384 pi@ninfacyzga-1-x

Then, the separate machine should navigate to ~localhost:8390~ in a
web browser in order to change the ninfacyzga-1 device's
configuration. The separate machine's Syncthing configuration options
are accessible via its own web browser via ~localhost:8384~.

****** ~git~
~git~ facilitates downloading files from this repository to the
device. It may be installed via:

: $ sudo apt install git

****** ninfacyzga-01 git repository
Create the directory ~/git-OC/~ . Within this directory, run the
following commands to clone the ~ninfacyzga-01~ git repository:
: $ git clone https://zdv2.bktei.com/gitweb/ninfacyzga-01.git
: $ cd ninfacyzga-01

Check out the ~develop~ branch (if the latest changes are desired over
those of the ~master~ branch).
: $ git checkout --track origin/develop

****** ~age~
~age~ is required for encrypting data at rest.

Place ~age~ binary (the one compiled for ARM CPU architecture for
Linux) in ~$HOME/.local/bin~. A copy of binary may be found within the
~exec~ directory.

: $ mkdir ~/.local/bin
: $ cp exec/age ~/.local/bin/

***** Disable Swap File
Since standard Raspberry OS 10 install involves copying unencrypted
file system image to SD card which is mounted by the Raspberry Pi,
system memory may be written to disk in the form of a Swap file as
described [[https://ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/][here]]. In order to reduce the chance that location log data
is ever written to disk, swap file functionality must be
disabled[fn:ideaheap_20130731_disableswap].

Raspbian 10 uses dphys-swapfile to manage a swap file. It may be
disabled persistently[fn:rpf_20190702_disableswappersist] by running
the following command:

: sudo systemctl disable dphys-swapfile.service

To view the status of the swap file in Raspbian 10, run ~free -m~:

#+BEGIN_EXAMPLE
pi@ninfacyzga-01:~$ free -m
          total    used    free  shared  buff/cache   available
Mem:        432      86      36      21         309         268
Swap:        99       0      99
#+END_EXAMPLE

After disabling the swap file and rebooting:

#+BEGIN_EXAMPLE
pi@ninfacyzga-01:~$ free -m
          total    used    free  shared  buff/cache   available
Mem:        432      89     214       3         128         289
Swap:         0       0       0
#+END_EXAMPLE

[fn:ideaheap_20130731_disableswap] Explanation:
https://ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/

[fn:rpf_20190702_disableswappersist] Persistant disabling of swap in
Raspbian 10 Buster:
https://www.raspberrypi.org/forums/viewtopic.php?p=1490692&sid=5c596a124b7805d6b10dab8d3d7caf16#p1490692

***** Disable Bluetooth
In order to reduce power consumed by bluetooth transmissions,
bluetooth functionality should be disabled (see [[https://di-marco.net/blog/it/2020-04-18-tips-disabling_bluetooth_on_raspberry_pi/][link]]).

Modify the ~/boot/config.txt~ file (the Pi's equivalent to BIOS
settings; see [[https://www.raspberrypi.org/documentation/configuration/config-txt/][link]]) to make sure the following lines are added:

#+BEGIN_EXAMPLE
# Disable Bluetooth
dtoverlay=disable-bt
#+END_EXAMPLE

The ~hciuart~ service is associated with bluetooth functionality via
UART which may conflict with location and time data provided via
~/dev/ttyAMA0~. It should be disabled like so:

#+BEGIN_EXAMPLE
$ sudo systemctl disable hciuart
#+END_EXAMPLE

***** Disable login console via serial port
Some ~ninfacyzga~ functions (location and time) require data transfer
via ~/dev/ttyAMA0~. In order to prevent serial login programs from
interfering with such functions, it is necessary to disable them.

Run the following commands to disable login via ~ttyAMA0~:

#+BEGIN_EXAMPLE
$ sudo systemctl stop serial-getty@ttyAMA0.service
$ sudo systemctl disable serial-getty@ttyAMA0.service
$ sudo systemctl disable hciuart
#+END_EXAMPLE

Modify ~/boot/cmdline.txt~ to remove the console:

#+BEGIN_EXAMPLE
$ sudo nano /boot/cmdline.txt
#+END_EXAMPLE

Remove ~console=serial0,115200~

***** Log Transfer Configuration
Log files may be shared to other machines via ~syncthing~. See [[https://docs.syncthing.net/][this]]
manual for how to set up a shared folder and add Ninfacyzga-01 as a
device. Syncthing's directory synchronization capability allows a
remote machine to delete files from Ninfacyzga-01 by deleting from the
shared folder that they both share.

When log files are removed from Ninfacyzga-01 is not within the scope
of this document.
***** Key Generation
An ~age~ encryption key may be generated like so:
#+BEGIN_EXAMPLE
$ umask          # Gets current umask
0022             # Note: This is the default umask for Raspbian 10
$ umask 066      # So key.txt will have no perms except for owner (you)
$ umask          # Confirm umask set to 066
0066
$ age-keygen > key.txt
Public key: age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5
$ ls -al key.txt
-rw------- 1 baltakatei baltakatei 184 Jun 29 18:28 key.txt
$ umask 0022     # Return umask to default value
$ umask          
0022
#+END_EXAMPLE

The resulting public/private keypair data looks like:
#+BEGIN_EXAMPLE
$ cat key.txt
# created: 2020-06-29T18:01:56Z
# public key: age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5
AGE-SECRET-KEY-1NEUU5U2XGZGL9UYWNPU5DL99TGJJHFSN4F2E2WCCSDJJ6L5ZMLESNTVTU0
#+END_EXAMPLE

The file ~key.txt~ is not password-protected by default and should be
secured like an SSH public key should. The ~$ umask 066~ command run
before the ~$ age-keygen > key.txt~ command ensures ~key.txt~ will not
be readable, writeable, or executable to anyone except the owner
(you).

*** Normal Startup
*** Normal Operation
*** Normal Shutdown
*** Unscheduled Shutdown
*** End of Life Disposal
See [[file:../setup/README.org][Main Setup]] procedures.

LiPo batteries used by the PiZ Uptime 2.0 module should be disposed of
properly with their potential ignitability in mind, especially if they
are not fully discharged.

Consult your local municipality for its "E-Waste Disposal" (or
equivalent) policy. Metals used in the Raspberry Pi and related
components may be recycled.

Take extra precuation if lead solder was used in assembling the
electronics. Consumer electronics in early 21st century should use
lead-free solder.