chore(exec/temp):Slow temp log period to 10s

[EVA-2020-02.git] / doc / location / README.org

* Location Logging
#+TITLE: Ninfacyzga-01 Manual
#+AUTHOR: Steven Baltakatei Sandoval
#+EMAIL: baltakatei@gmail.com
** About
This document was created by Steven Baltakatei Sandoval on
~2020-06-29T12:14Z~ under a [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons BY-SA 4.0 license]]. It was
updated by Steven Baltakatei Sandoval on ~2020-10-18T00:13Z~

** Narrative
Ninfacyzga-01 records (logs) its position in time and space using a
[[https://en.wikipedia.org/wiki/Satellite_navigation_device][GPS receiver]]. The NMEA location data produced by the receiver is
converted into the more commonly used GPS data storage formats of GPX
and KML. All three types of data are then compressed and encrypted
against a set of public keys. The encrypted data is then written to
disk. Data produced by the receiver is segmented into 60-second chunks
before being processed and written to disk.
** Description
*** Hardware
**** Raspberry Pi Zero W
See the [[https://www.raspberrypi.org/pi-zero-w/][OEM]] webpage for this product.
**** PiZ UpTime 2.0
See the [[https://alchemy-power.com/piz-uptime-2-0/][OEM]] webpage for this product.
*** Software
~bklog~ : A bash script that saves its stdin stream to a tar file. The
file may be compressed by ~gzip~ and encrypted by ~age~. It is an
executable file contained within this repository at ~exec/bklog~. It
should be copied to ~$HOME/.local/bin~.

[[https://tracker.debian.org/pkg/gpsd][~gpsd~]] : A background daemon app capable of interfacing with the
[[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][Ozzmaker BerryGPS-IMU]]'s GPS submodule. Installed and initialized by
~apt~. Should be installed along with the ~gpsd-clients~ package.

~gpspipe~ : A command line app that polls ~gpsd~ and produces a stream
stdout consisting of GPS data lines in NMEA format. Installed via
~apt~.

~gpsbabel~ : A command line app that converts GPS data from one format
into another. ~bklog~ may be used to convert NMEA data into GPX and
KML. Installed via ~apt~.

~gzip~ : A simple command line app that compresses stdin into a
smaller stdout stream.

~age~ : A simple command line app that encrypts stdin against public
keys specified in its options. Produces encrypted stdout. Is an
executable file contained within this repository at ~exec/age~. It
should be copied to ~$HOME/.local/bin~.

**** Narrative
~bklog~ may be used to log location data by receiving stdout produced
by ~gpspipe~. ~bklog~ contains options that allow file writes to be
performed at adjustable time intervals (default: 300 seconds),
compressed (with ~gzip~), and encrypted (with ~age~). Files are
written in the form of appendages to a ~tar~ archive saved to a
specified location. The NMEA data produced by ~gpspipe~ may be
processed via a ~gpsbabel~ command string specified as an option to
~bklog~, assuming ~gpsbabel~ is installed.

*** Output

Several output file formats have been tested with ~bklog~.

**** File Formats
***** NMEA
~NMEA~ is an acronym for National marine Electronics Association. The
NMEA format described in this document follows the NMEA 0183
standard. It is a newline-delimited streaming text format that encodes
global positioning system (GPS) data such as WGS84 location, time and
date information, satellite count, accuracy, and other
information. Each line is an "NMEA sentence". Descriptions of various
NMEA sentences can be found on [[http://aprs.gids.nl/nmea/][this]] webpage.

See the [[https://en.wikipedia.org/wiki/NMEA_0183][Wikipedia page for NMEA 0183]] for this. This file format is the
default output of the ~gpspipe -r~ command.
***** GPX
See the [[https://en.wikipedia.org/wiki/GPS_Exchange_Format][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. An NMEA file
may be converted to this format using ~gpsbabel~.
***** KML
See the [[https://en.wikipedia.org/wiki/Keyhole_Markup_Language][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. An NMEA file
may be converted to this format using ~gpsbabel~.

**** Encryption Method
See [[file:../setup/README.org][Main Setup]] procedures.

** Operating Procedures
*** Initial Startup
See OEM (Ozzmaker) [[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][quickstart guide for the BerryGPS-IMU]].
**** Physical Setup

BerryGPS-IMU must be electrically connected to the correct pins on the
GPIO header of a Raspberry Pi Zero W.

*Optional*: stack together with PiZ Uptime 2.0 module. No GPIO pins
conflict so a simple stacking and soldering with long header pins is
possible.

**** Software Setup
***** Install Executables
Follow the [[file:../setup/README.org][Main Setup]] procedures to obtain required files from this
repository.

Install ~gpsd~, ~gpsd-clients~, and ~gpsbabel~.

: $ sudo apt install gpsd gpsd-clients gpsbabel

***** Setup Serial for BerryGPS
The Ozzmaker BerryGPS-IMU unit requires that the serial console be
disabled and the serial port enabled. (see [[https://ozzmaker.com/berrygps-setup-guide-raspberry-pi/][ref]]).

: $ sudo raspi-config

Navigate to ~5 Interfacing Options~, then ~P6 Serial~.

When prompted "Would you like a login shell to be accessible over
serial?", answer ~No~.

When prompted "Would you like the serial port hardware to be
enabled?", answer ~Yes~.

***** Setup ~gpsd~
~gpsd~ needs to know which serial port to look at for NMEA location
data generated by the GPS unit. This can be done by modifying the
~gpsd~ configuration file at ~/etc/default/gpsd~.

: sudo nano /etc/default/gpsd

Make sure the following lines are present:

#+BEGIN_EXAMPLE
START_DAEMON="true"
USBAUTO="false"
DEVICES="/dev/ttyAMA0"
GPSD_OPTIONS="-n"
#+END_EXAMPLE

NOTE: The ~DEVICES=~ line must be adjusted if the the ~ninfacyzga~
device is configured to also track time. (See [[file:../time/README.org::*Setup%20~gpsd~][ref]]).

NOTE: The ~-n~ option causes ~gpsd~ to begin polling GPS devices
without waiting for a client to connect. This is important if ~gpsd~
provides data to other applications (ex: ~gpspipe~, ~chrony~).

Enable ~gpsd~ to run automatically via:

: $ sudo systemctl enable gpsd

Restart ~gpsd~ via:

: $ sudo systemctl restart gpsd

Run ~gpsmon~ to verify that location data is being recorded.

: $ gpsmon

***** Automatic Start Configuration
It is recommended to create a daily ~cron~ job that executes a bash
script that starts ~bklog~ jobs to record location along with other
types of data recording. An example of such a script is below:

#+BEGIN_EXAMPLE
#!/bin/bash

# Log location
gpspipe -r | /bin/bash "$HOME/.local/bin/bklog" -c -e -z "UTC" -t "/dev/shm" \
          -r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \
          -r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \
          -R "$HOME/.config/bklog/recipients" -w ".nmea" -b "300" -B "day" \
	  -o "$HOME/Sync/Evanescent_Location" -l "location" \
          -p "gpsbabel -i nmea -f - -o gpx -F - " ".gpx" \
          -p "gpsbabel -i nmea -f - -o kml -F - " ".kml" &
# Log pressure
python ~/.local/bin/bmp388.py | /bin/bash "$HOME/git-OC/ninfacyzga-01/exec/bklog" \
          -c -e -z "UTC" -t "/dev/shm" \
          -r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \
          -r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \
          -R "$HOME/.config/bklog/recipients" -w ".txt" -b "300" -B "day" \
	  -o "$HOME/Sync/Evanescent_Location" -l "pressure" &
#+END_EXAMPLE

This script, if it were saved at
~"$HOME/.local/bin/cron/dailylog.sh"~ would then be added as a
line in the ~crontab~ file as shown below:

#+BEGIN_EXAMPLE
$ crontab -e
0 0 * * * /bin/bash ~/.local/bin/cron/dailylog.sh
@reboot /bin/bash ~/.local/bin/cron/dailylog.sh
#+END_EXAMPLE

In the example script, the options are:

: -c : tells bklog to compress output
: -e : tells bklog log to encrypt output
: -r : tells bklog to interpret the next argument as a pubkey string
: -R : tells bklog to interpret the next argument as a directory
:        where public keys may be found (first line of each file is
:        read).
: -o : tells bklog to write output files to the directory represented
: -t : tells bklog to interpret the next argument as a directory
:        for storing temporary files
:        by the next argument
: -p : tells bklog a command string through which output is piped
:        before being compressed and encrypted. Also expected is a
:        file extension to be appended before the compression and
:        encryption file extensions.
: -w : tells bklog to save the unprocessed stdin with a specified
:        file extension (instead of the default '.stdin').
: -b : tells bklog how long each buffer round (time between file
:        writes) lasts in seconds.
: -B : specifies the time-to-live for the bklog script. A valid value may
:        one of the time elements such as "day" or "hour".
: -l : specfies a custom string to be used in output file names to
:        help differentiate tar files produced via bklog from different
:        sources of data.
: -z : specifies a time zone to be used to determine the script time-to-live.
:        By default, bklog uses whatever time is specified by the TZ
:        environment variable.

***** Log Transfer Configuration
See [[file:../setup/README.org][Main Setup]] procedures.

Log files may be shared to other machines via ~syncthing~. See [[https://docs.syncthing.net/][this]]
manual for how to set up a shared folder and add Ninfacyzga-01 as a
device. Syncthing's directory synchronization capability allows a
remote machine to delete files from Ninfacyzga-01 by deleting from the
shared folder that they both share.

When log files are removed from Ninfacyzga-01 is not within the scope
of this document.

***** Key Generation
See [[file:../setup/README.org][Main Setup]] procedures.

*** Normal Startup
Turn on Ninfacyzga-01 by supplying 5VDC power to the Raspberry Pi. No
further interaction should be required.
*** Normal Operation
No interaction beyond continually supplying approximately 100mA of
5VDC power and occasionally removing log files to conserve disk space
is required.
**** Log Transfer
Log files may be transferred by use of ~syncthing~ shared folders.
**** Automatic Updates
The ~automatic-upgrades~ package, if installed, should automatically
install security patches to packages installed via ~apt~.
*** Normal Shutdown
The system may be shutdown via SSH by running:

: $ sudo shutdown -r 0

*** Unscheduled Shutdown
Ninfacyzga-01 as described and setup should tolerate unscheduled power
loss. Log files being written every 60 seconds means, at most, 60
seconds worth of location data may be lost.
*** End of Life Disposal
See [[file:../setup/README.org][Main Setup]] procedures.