X-Git-Url: https://zdv2.bktei.com/gitweb/EVA-2020-02.git/blobdiff_plain/dac703dda941e1fd4ad0f6c0934a1f937fe5d45f..78507dc2c597052cdb5eefcf365a9170842bc05c:/doc/location/README.org diff --git a/doc/location/README.org b/doc/location/README.org index 397968f..d45fdae 100644 --- a/doc/location/README.org +++ b/doc/location/README.org @@ -1,10 +1,12 @@ * Location Logging -This document was created by Steven Baltakatei Sandoval on -~2020-06-29T12:14Z~ under a [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons BY-SA 4.0 license]]. It -was updated by Steven Baltakatei Sandoval on ~2020-06-30T19:44Z~ #+TITLE: Ninfacyzga-01 Manual #+AUTHOR: Steven Baltakatei Sandoval #+EMAIL: baltakatei@gmail.com +** About +This document was created by Steven Baltakatei Sandoval on +~2020-06-29T12:14Z~ under a [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons BY-SA 4.0 license]]. It was +updated by Steven Baltakatei Sandoval on ~2020-10-18T00:13Z~ + ** Narrative Ninfacyzga-01 records (logs) its position in time and space using a [[https://en.wikipedia.org/wiki/Satellite_navigation_device][GPS receiver]]. The NMEA location data produced by the receiver is @@ -20,21 +22,21 @@ See the [[https://www.raspberrypi.org/pi-zero-w/][OEM]] webpage for this product **** PiZ UpTime 2.0 See the [[https://alchemy-power.com/piz-uptime-2-0/][OEM]] webpage for this product. *** Software -~bkgpslog~ : The bash script that performs the location data -collection and processing. Is an executable file contained within this -repository at ~exec/bkgpslog~. It should be copied to -~$HOME/.local/bin~. +~bklog~ : A bash script that saves its stdin stream to a tar file. The +file may be compressed by ~gzip~ and encrypted by ~age~. It is an +executable file contained within this repository at ~exec/bklog~. It +should be copied to ~$HOME/.local/bin~. -~gpsd~ : A background daemon app capable of interfacing with the -Ozzmaker BerryGPS-IMU's GPS submodule. Installed and initialized by -~apt~. +[[https://tracker.debian.org/pkg/gpsd][~gpsd~]] : A background daemon app capable of interfacing with the +[[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][Ozzmaker BerryGPS-IMU]]'s GPS submodule. Installed and initialized by +~apt~. Should be installed along with the ~gpsd-clients~ package. ~gpspipe~ : A command line app that polls ~gpsd~ and produces a stream stdout consisting of GPS data lines in NMEA format. Installed via ~apt~. ~gpsbabel~ : A command line app that converts GPS data from one format -into another. ~bkgpslog~ uses it to convert NMEA data into GPX and +into another. ~bklog~ may be used to convert NMEA data into GPX and KML. Installed via ~apt~. ~gzip~ : A simple command line app that compresses stdin into a @@ -46,61 +48,44 @@ executable file contained within this repository at ~exec/age~. It should be copied to ~$HOME/.local/bin~. **** Narrative -~bkgpslog~ populates a 60-second buffer with NMEA data from ~gpsd~ via -~gpspipe~. This buffer is used by ~gpsbabel~ to produce GPX and KML -versions of the buffer. All 3 buffers are then comprssed with ~gzip~, -encrypted with ~age~, and then written to disk. +~bklog~ may be used to log location data by receiving stdout produced +by ~gpspipe~. ~bklog~ contains options that allow file writes to be +performed at adjustable time intervals (default: 300 seconds), +compressed (with ~gzip~), and encrypted (with ~age~). Files are +written in the form of appendages to a ~tar~ archive saved to a +specified location. The NMEA data produced by ~gpspipe~ may be +processed via a ~gpsbabel~ command string specified as an option to +~bklog~, assuming ~gpsbabel~ is installed. *** Output + +Several output file formats have been tested with ~bklog~. + **** File Formats ***** NMEA -See the [[https://en.wikipedia.org/wiki/NMEA_0183][Wikipedia page]] for this. +~NMEA~ is an acronym for National marine Electronics Association. The +NMEA format described in this document follows the NMEA 0183 +standard. It is a newline-delimited streaming text format that encodes +global positioning system (GPS) data such as WGS84 location, time and +date information, satellite count, accuracy, and other +information. Each line is an "NMEA sentence". Descriptions of various +NMEA sentences can be found on [[http://aprs.gids.nl/nmea/][this]] webpage. + +See the [[https://en.wikipedia.org/wiki/NMEA_0183][Wikipedia page for NMEA 0183]] for this. This file format is the +default output of the ~gpspipe -r~ command. ***** GPX -See the [[https://en.wikipedia.org/wiki/GPS_Exchange_Format][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. +See the [[https://en.wikipedia.org/wiki/GPS_Exchange_Format][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. An NMEA file +may be converted to this format using ~gpsbabel~. ***** KML -See the [[https://en.wikipedia.org/wiki/Keyhole_Markup_Language][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. -**** Encryption Method -Files produced by the bkgpslog script are encrypted against a set of -public keys using [[https://github.com/FiloSottile/age][~age~]], a simple command line encryption tool -selected over ~gpg~ because of ~age~'s deliberate lack of -configurability. - -The public keys are bech32 strings supplied as options to bkgpslog -when called. The secret key should *NOT* be stored in Ninfacyzga-01. - -If a key pair was generated using ~age-keygen~, then it is an [[https://en.wikipedia.org/wiki/Curve25519][~X25519~]] -key pair. See the [[https://age-encryption.org/v1][~age~ Version 1 specification]]. - -An ~ssh-rsa~ or ~ssh-ed25519~ SSH public key string may be used instead of -the bech32 public key string produced by ~age-keygen~ for convenience. +See the [[https://en.wikipedia.org/wiki/Keyhole_Markup_Language][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. An NMEA file +may be converted to this format using ~gpsbabel~. -Help information for ~age~ is available by running ~$ age --help~. -***** Encryption Commands -Files may be encrypted to several recipients using a command similar to: -#+BEGIN_EXAMPLE -timeout "60s" gpspipe -r | gpsbabel -i nmea -f - -o gpx -F | age \ --r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ --r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \ --r age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 \ -> location.gpx.age -#+END_EXAMPLE - -In this example, the strings beginning with ~age1...~ are -bech32-formatted public key strings. - - -***** Decryption Commands -Files may be decrypted using a command similar to: - -#+BEGIN_EXAMPLE -cat location.gpx.age | age -d -i key.txt > location.gpx -#+END_EXAMPLE +**** Encryption Method +See [[file:../setup/README.org][Main Setup]] procedures. -The version of ~age~ used to perform the encryption ** Operating Procedures *** Initial Startup See OEM (Ozzmaker) [[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][quickstart guide for the BerryGPS-IMU]]. - **** Physical Setup BerryGPS-IMU must be electrically connected to the correct pins on the @@ -112,103 +97,129 @@ possible. **** Software Setup ***** Install Executables +Follow the [[file:../setup/README.org][Main Setup]] procedures to obtain required files from this +repository. -Install Raspbian 10 Buster onto an SD card image. See the Raspberry Pi -Foundation [[https://www.raspberrypi.org/documentation/installation/installing-images/README.md][installation instructions]]. Configure WiFi to permit log -file transfer. Configure SSH to permit remote administration via the -command line interface. +Install ~gpsd~, ~gpsd-clients~, and ~gpsbabel~. -Make sure to install the ~unattended-upgrades~ package to make sure -the latest security patches for packages are installed. See [[https://linux-audit.com/using-unattended-upgrades-on-debian-and-ubuntu/][this page]] -for a description of how ~unattended-upgrades~ works. +: $ sudo apt install gpsd gpsd-clients gpsbabel -Install ~gpsd~, ~gpspipe~, ~git~, and this repository for location -logging capability. +***** Setup Serial for BerryGPS +The Ozzmaker BerryGPS-IMU unit requires that the serial console be +disabled and the serial port enabled. (see [[https://ozzmaker.com/berrygps-setup-guide-raspberry-pi/][ref]]). -Install ~syncthing~ for log file transfer capability. +: $ sudo raspi-config -Place ~age~ binary (the one compiled for ARM CPU architecture for -Linux) in ~$HOME/.local/bin~. +Navigate to ~5 Interfacing Options~, then ~P6 Serial~. -***** Disable Swap File -Since standard Raspbian 10 (Buster) install involves copying -unencrypted file system image to SD card which is mounted by the -Raspberry Pi, system memory may be written to disk in the form of a -Swap file as described [[https://ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/][here]]. In order to reduce the chance that -location log data is ever written to disk, swap file functionality -must be disabled[fn:ideaheap_20130731_disableswap]. +When prompted "Would you like a login shell to be accessible over +serial?", answer ~No~. -Raspbian 10 uses dphys-swapfile to manage a swap file. It may be -disabled persistently[fn:rpf_20190702_disableswappersist] by running -the following command: +When prompted "Would you like the serial port hardware to be +enabled?", answer ~Yes~. -: sudo systemctl disable dphys-swapfile.service +***** Setup ~gpsd~ +~gpsd~ needs to know which serial port to look at for NMEA location +data generated by the GPS unit. This can be done by modifying the +~gpsd~ configuration file at ~/etc/default/gpsd~. -To view the status of the swap file in Raspbian 10, run ~free -m~: +: sudo nano /etc/default/gpsd -#+BEGIN_EXAMPLE -pi@ninfacyzga-01:~$ free -m - total used free shared buff/cache available -Mem: 432 86 36 21 309 268 -Swap: 99 0 99 -#+END_EXAMPLE - -After disabling the swap file and rebooting: +Make sure the following lines are present: #+BEGIN_EXAMPLE -pi@ninfacyzga-01:~$ free -m - total used free shared buff/cache available -Mem: 432 89 214 3 128 289 -Swap: 0 0 0 +START_DAEMON="true" +USBAUTO="false" +DEVICES="/dev/ttyAMA0" +GPSD_OPTIONS="-n" #+END_EXAMPLE -[fn:ideaheap_20130731_disableswap] Explanation: -https://ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/ +NOTE: The ~DEVICES=~ line must be adjusted if the the ~ninfacyzga~ +device is configured to also track time. (See [[file:../time/README.org::*Setup%20~gpsd~][ref]]). -[fn:rpf_20190702_disableswappersist] Persistant disabling of swap in -Raspbian 10 Buster: -https://www.raspberrypi.org/forums/viewtopic.php?p=1490692&sid=5c596a124b7805d6b10dab8d3d7caf16#p1490692 +NOTE: The ~-n~ option causes ~gpsd~ to begin polling GPS devices +without waiting for a client to connect. This is important if ~gpsd~ +provides data to other applications (ex: ~gpspipe~, ~chrony~). -***** Automatic Start Configuration +Enable ~gpsd~ to run automatically via: -Edit the user cron job list with ~$ crontab -e~ to add the following -lines: +: $ sudo systemctl enable gpsd -#+BEGIN_EXAMPLE -0 * * * * /bin/bash ~/bkgpslog --output ~/dir +Restart ~gpsd~ via: -@reboot /bin/bash ~/bkgpslog --output ~/dir -#+END_EXAMPLE +: $ sudo systemctl restart gpsd -The first line will run ~bkgpslog~ at the start of every hour and save -output files to the ~dir~ directory in your home folder. +Run ~gpsmon~ to verify that location data is being recorded. -The second line will run ~bkgpslog~ when the system starts up. +: $ gpsmon -~/bin/bash~ tells ~cron~ to run ~bkgpslog~ with Bash. - -If encryption and compression are required, then the appropriate -options must be added. The lines that must be added via ~$ crontab -e~ -may resemble: +***** Automatic Start Configuration +It is recommended to create a daily ~cron~ job that executes a bash +script that starts ~bklog~ jobs to record location along with other +types of data recording. An example of such a script is below: #+BEGIN_EXAMPLE -0 * * * * /bin/bash ~/bkgpslog -c -e -r age1z2...qkv6p -o ~/dir - -@reboot /bin/bash ~/bkgpslog -c -e -r age1z2...qkv6p -o ~/dir +#!/bin/bash + +# Log location +gpspipe -r | /bin/bash "$HOME/.local/bin/bklog" -c -e -z "UTC" -t "/dev/shm" \ + -r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ + -r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \ + -R "$HOME/.config/bklog/recipients" -w ".nmea" -b "300" -B "day" \ + -o "$HOME/Sync/Evanescent_Location" -l "location" \ + -p "gpsbabel -i nmea -f - -o gpx -F - " ".gpx" \ + -p "gpsbabel -i nmea -f - -o kml -F - " ".kml" & +# Log pressure +python ~/.local/bin/bmp388.py | /bin/bash "$HOME/git-OC/ninfacyzga-01/exec/bklog" \ + -c -e -z "UTC" -t "/dev/shm" \ + -r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ + -r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \ + -R "$HOME/.config/bklog/recipients" -w ".txt" -b "300" -B "day" \ + -o "$HOME/Sync/Evanescent_Location" -l "pressure" & #+END_EXAMPLE -The ~age1z2...qkv6p~ is an ~age~ public key string. Please see the -[[*Key Generation][Key Generation]] section for an explanation. +This script, if it were saved at +~"$HOME/.local/bin/cron/dailylog.sh"~ would then be added as a +line in the ~crontab~ file as shown below: -The options are: +#+BEGIN_EXAMPLE +$ crontab -e +0 0 * * * /bin/bash ~/.local/bin/cron/dailylog.sh +@reboot /bin/bash ~/.local/bin/cron/dailylog.sh +#+END_EXAMPLE -: -c : tells bkgpslog to compress output -: -e : tells bkgpslog log to encrypt output -: -r : tells bkgpslog to interpret the next argument as a pubkey string -: -o : tells bkgpslog to write output files to the directory represented +In the example script, the options are: + +: -c : tells bklog to compress output +: -e : tells bklog log to encrypt output +: -r : tells bklog to interpret the next argument as a pubkey string +: -R : tells bklog to interpret the next argument as a directory +: where public keys may be found (first line of each file is +: read). +: -o : tells bklog to write output files to the directory represented +: -t : tells bklog to interpret the next argument as a directory +: for storing temporary files : by the next argument +: -p : tells bklog a command string through which output is piped +: before being compressed and encrypted. Also expected is a +: file extension to be appended before the compression and +: encryption file extensions. +: -w : tells bklog to save the unprocessed stdin with a specified +: file extension (instead of the default '.stdin'). +: -b : tells bklog how long each buffer round (time between file +: writes) lasts in seconds. +: -B : specifies the time-to-live for the bklog script. A valid value may +: one of the time elements such as "day" or "hour". +: -l : specfies a custom string to be used in output file names to +: help differentiate tar files produced via bklog from different +: sources of data. +: -z : specifies a time zone to be used to determine the script time-to-live. +: By default, bklog uses whatever time is specified by the TZ +: environment variable. ***** Log Transfer Configuration +See [[file:../setup/README.org][Main Setup]] procedures. + Log files may be shared to other machines via ~syncthing~. See [[https://docs.syncthing.net/][this]] manual for how to set up a shared folder and add Ninfacyzga-01 as a device. Syncthing's directory synchronization capability allows a @@ -219,35 +230,7 @@ When log files are removed from Ninfacyzga-01 is not within the scope of this document. ***** Key Generation -An ~age~ encryption key may be generated like so: -#+BEGIN_EXAMPLE -$ umask # Gets current umask -0022 # Note: This is the default umask for Raspbian 10 -$ umask 066 # So key.txt will have no perms except for owner (you) -$ umask # Confirm umask set to 066 -0066 -$ age-keygen > key.txt -Public key: age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 -$ ls -al key.txt --rw------- 1 baltakatei baltakatei 184 Jun 29 18:28 key.txt -$ umask 0022 # Return umask to default value -$ umask -0022 -#+END_EXAMPLE - -The resulting public/private keypair data looks like: -#+BEGIN_EXAMPLE -$ cat key.txt -# created: 2020-06-29T18:01:56Z -# public key: age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 -AGE-SECRET-KEY-1NEUU5U2XGZGL9UYWNPU5DL99TGJJHFSN4F2E2WCCSDJJ6L5ZMLESNTVTU0 -#+END_EXAMPLE - -The file ~key.txt~ is not password-protected by default and should be -secured like an SSH public key should. The ~$ umask 066~ command run -before the ~$ age-keygen > key.txt~ command ensures ~key.txt~ will not -be readable, writeable, or executable to anyone except the owner -(you). +See [[file:../setup/README.org][Main Setup]] procedures. *** Normal Startup Turn on Ninfacyzga-01 by supplying 5VDC power to the Raspberry Pi. No @@ -271,15 +254,4 @@ Ninfacyzga-01 as described and setup should tolerate unscheduled power loss. Log files being written every 60 seconds means, at most, 60 seconds worth of location data may be lost. *** End of Life Disposal -LiPo batteries used by the PiZ Uptime 2.0 module should be disposed of -properly with their potential ignitability in mind, especially if they -are not fully discharged. - -Consult your local municipality for its "E-Waste Disposal" (or -equivalent) policy. Metals used in the Raspberry Pi and related -components may be recycled. - -Take extra precuation if lead solder was used in assembling the -electronics. Consumer electronics in early 21st century should use -lead-free solder. - +See [[file:../setup/README.org][Main Setup]] procedures.