From aedd19f6b8743c5fbb923e552a961d0440ff8ff5 Mon Sep 17 00:00:00 2001 From: Steven Baltakatei Sandoval Date: Sun, 12 Jul 2020 21:05:23 +0000 Subject: [PATCH] doc(bklog):Update location readme with bklog details --- doc/location/README.org | 157 ++++++++++++++++++++++++++++------------ 1 file changed, 112 insertions(+), 45 deletions(-) diff --git a/doc/location/README.org b/doc/location/README.org index 397968f..1ffdccd 100644 --- a/doc/location/README.org +++ b/doc/location/README.org @@ -1,7 +1,7 @@ * Location Logging This document was created by Steven Baltakatei Sandoval on ~2020-06-29T12:14Z~ under a [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons BY-SA 4.0 license]]. It -was updated by Steven Baltakatei Sandoval on ~2020-06-30T19:44Z~ +was updated by Steven Baltakatei Sandoval on ~2020-07-12T21:04Z~ #+TITLE: Ninfacyzga-01 Manual #+AUTHOR: Steven Baltakatei Sandoval #+EMAIL: baltakatei@gmail.com @@ -20,10 +20,13 @@ See the [[https://www.raspberrypi.org/pi-zero-w/][OEM]] webpage for this product **** PiZ UpTime 2.0 See the [[https://alchemy-power.com/piz-uptime-2-0/][OEM]] webpage for this product. *** Software -~bkgpslog~ : The bash script that performs the location data -collection and processing. Is an executable file contained within this -repository at ~exec/bkgpslog~. It should be copied to -~$HOME/.local/bin~. +~bklog~ : A bash script that saves its stdin stream to a tar file. The +file may be compressed by ~gzip~ and encrypted by ~age~. It is an +executable file contained within this repository at ~exec/bklog~. It +should be copied to ~$HOME/.local/bin~. + +~bkgpslog~ : A legacy bash script similar to ~bklog~ but narrower in +scope in that it only records output from ~gpspipe~. ~gpsd~ : A background daemon app capable of interfacing with the Ozzmaker BerryGPS-IMU's GPS submodule. Installed and initialized by @@ -34,7 +37,7 @@ stdout consisting of GPS data lines in NMEA format. Installed via ~apt~. ~gpsbabel~ : A command line app that converts GPS data from one format -into another. ~bkgpslog~ uses it to convert NMEA data into GPX and +into another. ~bklog~ may be used to convert NMEA data into GPX and KML. Installed via ~apt~. ~gzip~ : A simple command line app that compresses stdin into a @@ -46,19 +49,30 @@ executable file contained within this repository at ~exec/age~. It should be copied to ~$HOME/.local/bin~. **** Narrative -~bkgpslog~ populates a 60-second buffer with NMEA data from ~gpsd~ via -~gpspipe~. This buffer is used by ~gpsbabel~ to produce GPX and KML -versions of the buffer. All 3 buffers are then comprssed with ~gzip~, -encrypted with ~age~, and then written to disk. +~bklog~ may be used to log location data by receiving stdout produced +by ~gpspipe~. ~bklog~ contains options that allow file writes to be +performed at adjustable time intervals (default: 300 seconds), +compressed (with ~gzip~), and encrypted (with ~age~). Files are +written in the form of appendages to a ~tar~ archive saved to a +specified location. The NMEA data produced by ~gpspipe~ may be +processed via a ~gpsbabel~ command string specified as an option to +~bklog~, assuming ~gpsbabel~ is installed. *** Output + +Several output file formats have been tested with ~bklog~. + **** File Formats ***** NMEA -See the [[https://en.wikipedia.org/wiki/NMEA_0183][Wikipedia page]] for this. +See the [[https://en.wikipedia.org/wiki/NMEA_0183][Wikipedia page]] for this. This file format is the default +output of the ~gpspipe -r~ command. ***** GPX -See the [[https://en.wikipedia.org/wiki/GPS_Exchange_Format][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. +See the [[https://en.wikipedia.org/wiki/GPS_Exchange_Format][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. An NMEA file +may be converted to this format using ~gpsbabel~. ***** KML -See the [[https://en.wikipedia.org/wiki/Keyhole_Markup_Language][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. +See the [[https://en.wikipedia.org/wiki/Keyhole_Markup_Language][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. An NMEA file +may be converted to this format using ~gpsbabel~. + **** Encryption Method Files produced by the bkgpslog script are encrypted against a set of public keys using [[https://github.com/FiloSottile/age][~age~]], a simple command line encryption tool @@ -76,18 +90,45 @@ the bech32 public key string produced by ~age-keygen~ for convenience. Help information for ~age~ is available by running ~$ age --help~. ***** Encryption Commands -Files may be encrypted to several recipients using a command similar to: +****** Encryption through ~age~ +In order to illustrate how ~bklog~ encrypts files, below is an example +command illustrating how ~age~ may be used to encrypt a file. + +#+BEGIN_EXAMPLE +$ echo "asdf" | age -r \ +age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ +> "$HOME/secret_file" +#+END_EXAMPLE + +The resulting ~secret-file~ is a binary blob with a plaintext header +indicating how the blob was encrypted (which version of age was used, +which public key was used). + +****** Encryption through ~bklog~ +~bklog~ may instructed to encrypt files via the ~-e~ and ~-r [pubkey +string]~ options. An example is shown below: + #+BEGIN_EXAMPLE -timeout "60s" gpspipe -r | gpsbabel -i nmea -f - -o gpx -F | age \ +$ gpspipe -r | bklog -e \ -r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ -r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \ -r age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 \ -> location.gpx.age +-o "$HOME/Location" #+END_EXAMPLE +~bklog~ may be instructed via the ~-e~ and ~-R~ options to watch a +directory in order to locate public key strings in its files. ~bklog~ +reads the first line of each file and interprets it as a public key +string. + In this example, the strings beginning with ~age1...~ are -bech32-formatted public key strings. +bech32-formatted public key strings. Please see the [[*Key Generation][Key Generation]] +section for an explanation. + +Since ~age~ also accepts ~ssh~ public key strings, these may also be +used if they are of the following form (no comment). +: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABA…AACAQDLnJbPs7CjwPT+OxXd ***** Decryption Commands Files may be decrypted using a command similar to: @@ -97,6 +138,7 @@ cat location.gpx.age | age -d -i key.txt > location.gpx #+END_EXAMPLE The version of ~age~ used to perform the encryption + ** Operating Procedures *** Initial Startup See OEM (Ozzmaker) [[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][quickstart guide for the BerryGPS-IMU]]. @@ -171,42 +213,68 @@ https://www.raspberrypi.org/forums/viewtopic.php?p=1490692&sid=5c596a124b7805d6b ***** Automatic Start Configuration -Edit the user cron job list with ~$ crontab -e~ to add the following -lines: +It is recommended to create a daily ~cron~ job that executes a bash +script that starts ~bklog~ jobs to record location along with other +types of data recording. An example of such a script is below: #+BEGIN_EXAMPLE -0 * * * * /bin/bash ~/bkgpslog --output ~/dir - -@reboot /bin/bash ~/bkgpslog --output ~/dir +#!/bin/bash + +# Log location +gpspipe -r | /bin/bash "$HOME/.local/bin/bklog" -c -e -z "UTC" -t "/dev/shm" \ + -r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ + -r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \ + -R "$HOME/.config/bklog/recipients" -w ".nmea" -b "300" -B "day" \ + -o "$HOME/Sync/Evanescent_Location" -l "location" \ + -p "gpsbabel -i nmea -f - -o gpx -F - " ".gpx" \ + -p "gpsbabel -i nmea -f - -o kml -F - " ".kml" & +# Log pressure +python ~/.local/bin/bmp388.py | /bin/bash "$HOME/git-OC/ninfacyzga-01/exec/bklog" \ + -c -e -z "UTC" -t "/dev/shm" \ + -r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ + -r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \ + -R "$HOME/.config/bklog/recipients" -w ".txt" -b "300" -B "day" \ + -o "$HOME/Sync/Evanescent_Location" -l "pressure" & #+END_EXAMPLE -The first line will run ~bkgpslog~ at the start of every hour and save -output files to the ~dir~ directory in your home folder. - -The second line will run ~bkgpslog~ when the system starts up. - -~/bin/bash~ tells ~cron~ to run ~bkgpslog~ with Bash. - -If encryption and compression are required, then the appropriate -options must be added. The lines that must be added via ~$ crontab -e~ -may resemble: +This script, if it were saved at +~"$HOME/.local/bin/cron/dailylog.sh"~ would then be added as a +line in the ~crontab~ file as shown below: #+BEGIN_EXAMPLE -0 * * * * /bin/bash ~/bkgpslog -c -e -r age1z2...qkv6p -o ~/dir - -@reboot /bin/bash ~/bkgpslog -c -e -r age1z2...qkv6p -o ~/dir +$ crontab -e +0 0 * * * /bin/bash ~/.local/bin/cron/dailylog.sh +@reboot /bin/bash ~/.local/bin/cron/dailylog.sh #+END_EXAMPLE -The ~age1z2...qkv6p~ is an ~age~ public key string. Please see the -[[*Key Generation][Key Generation]] section for an explanation. - -The options are: - -: -c : tells bkgpslog to compress output -: -e : tells bkgpslog log to encrypt output -: -r : tells bkgpslog to interpret the next argument as a pubkey string -: -o : tells bkgpslog to write output files to the directory represented +In the example script, the options are: + +: -c : tells bklog to compress output +: -e : tells bklog log to encrypt output +: -r : tells bklog to interpret the next argument as a pubkey string +: -R : tells bklog to interpret the next argument as a directory +: where public keys may be found (first line of each file is +: read). +: -o : tells bklog to write output files to the directory represented +: -t : tells bklog to interpret the next argument as a directory +: for storing temporary files : by the next argument +: -p : tells bklog a command string through which output is piped +: before being compressed and encrypted. Also expected is a +: file extension to be appended before the compression and +: encryption file extensions. +: -w : tells bklog to save the unprocessed stdin with a specified +: file extension (instead of the default '.stdin'). +: -b : tells bklog how long each buffer round (time between file +: writes) lasts in seconds. +: -B : specifies the time-to-live for the bklog script. A valid value may +: one of the time elements such as "day" or "hour". +: -l : specfies a custom string to be used in output file names to +: help differentiate tar files produced via bklog from different +: sources of data. +: -z : specifies a time zone to be used to determine the script time-to-live. +: By default, bklog uses whatever time is specified by the TZ +: environment variable. ***** Log Transfer Configuration Log files may be shared to other machines via ~syncthing~. See [[https://docs.syncthing.net/][this]] @@ -282,4 +350,3 @@ components may be recycled. Take extra precuation if lead solder was used in assembling the electronics. Consumer electronics in early 21st century should use lead-free solder. - -- 2.30.2