-* bkgpslog task list
-** DONE Add job control for short buffer length
- CLOSED: [2020-07-02 Thu 16:04]
-2020-07-02T14:56Z; bktei> File write operations were bundled into a
-magicWriteBuffer function that is called then detached from the script
-shell (job control), but the detached job is not tracked by the main
-script. A problem may arise if two instances of magicWriteBuffer
-attempt to write to the same tar simultaneously. Two instances of
-magicWriteBuffer may exist if the buffer length is low (ex: 1 second);
-the default buffer length of 60 seconds should reduce the probability
-of a collision but it should be possible for the main script to track
-the process ID of a magicWriteBuffer() as soon as it detaches and then
-checking (via ~$!~ as described [[https://bashitout.com/2013/05/18/Ampersands-on-the-command-line.html][here]]) that the process is still alive.
-2020-07-02T15:23Z; bktei> I found that the Bash ~wait~ built-in can be
-used to delay processing until a specified job completes. The ~wait~
-command will pause script execution until all backgrounded processes
-complete.
-2020-07-02T16:03Z; bktei> Added ~wait~.
-** DONE Rewrite tar initialization function
- CLOSED: [2020-07-02 Thu 17:23]
-2020-07-02T17:23Z; bktei> Simplify tar initialization function so
-VERSION file is used to test appendability of tar as well as to mark
-when a new session is started.
-** DONE Consolidate tar checking/creation into function
- CLOSED: [2020-07-02 Thu 18:33]
-2020-07-02T18:33Z; bktei> Simplify how the output tar file's existence
-is checked and its status as a valid tar file is validated. This was
-done using a new function ~checkMakeTar~.
-** DONE Add VERSION if output tar deleted between writes
-
- CLOSED: [2020-07-02 Thu 20:22]
-2020-07-02T20:21Z; bktei> Added bkgpslog-specified function
-magicWriteVersion() to be called whenever a new time-stamped ~VERSION~
-file needs to be generated and appended to the output tar file
-~PATHOUT_TAR~.
-** DONE Rewrite buffer loop to reduce lag between gpspipe runs
-
- CLOSED: [2020-07-03 Fri 20:57]
-2020-07-03T17:10Z; bktei> As is, there is still a 5-6 second lag
-between when ~gpspipe~ times out at the end of a buffer round and when
-~gpspipe~ is called by the subsequent buffer round. I believe this can
-be reduced by moving variable manipulations inside the
-asynchronously-executed magicWriteBuffer() function. Ideally, the
-while loop should look like:
-
-#+BEGIN_EXAMPLE
-while( $SECONDS < $SCRIPT_TTL); do
- gpspipe-r > "$DIR_TMP"/buffer.nmea
- writeBuffer &
-done
-#+END_EXAMPLE
-2020-07-03T20:56Z; bktei> I simplified it futher to something like
-this:
-#+BEGIN_EXAMPLE
-while( $SECONDS < $SCRIPT_TTL); do
- writeBuffer &
- sleep $SCRIPT_TTL
-done
-#+END_EXAMPLE
-
-Raspberry Pi Zero W shows approximately 71ms of drift per buffer round
-with 10s buffer.
-** TODO Feature: Recipient watch folder
-2020-07-03T21:28Z; bktei> This feature would be to scan the contents
-of a specified directory at the start of every buffer round in order
-to determine encryption (age) recipients. This would allow a device to
-dynamically encrypt location data in response to automated changes
-made by other tools. For example, if such a directory were
-synchronized via Syncthing and changes to such a directory were
-managed by a trusted remote server, then that server could respond to
-human requests to secure location data.
-
-Two specific privacy subfeatures come to mind:
-
-1. Parallel encryption: Given a set of ~n~ public keys, encrypt data
- with a single ~age~ command with options causing all ~n~ pubkeys to
- be recipients. In order to decrypt the data, any individual private
- key could be used. No coordination between key owners would be
- required to decrypt.
-
-2. Sequential encryption: Given a set of ~n~ public keys, encrypt data
- with ~n~ sequential ~age~ commands all piped in series with each
- ~age~ command utilizing only one of the ~n~ public keys. In order
- to decrypt the data, all ~n~ private keys would be required to
- decrypt the data. Since coordination is required, it is less
- convenient than parallel encryption.
-
-In either case, a directory would be useful for holding configuration
-files specifying how to execute which or combination of which features
-at the start of every buffer round.
-
-I don't yet know how to program the rules, although I think it'd be
-easier to simply add an option providing ~bkgpslog~ with a directory
-to watch. When examining the directory, check for a file with the
-appropriate file extension (ex: .pubkey) and then read the first line
-into the script's pubKey array.
-
-** TODO Feature: Simplify option to reduce output size
-
-~gpsbabel~ [[https://www.gpsbabel.org/htmldoc-development/filter_simplify.html][features]] a ~simplify~ option to trim data points from GPS
-data. There are several methods for prioritizing which points to keep
-and which to trim, although the following seems useful given some
-sample data I've recorded in a test run of ninfacyzga-01:
-
-#+BEGIN_EXAMPLE
-gpsbabel -i nmea -f all.nmea -x simplify,error=10,relative -o gpx \
--F all-simp-rel-10.gpx
-#+END_EXAMPLE
-
-An error level of "10" with the "relative" option seems to retain all
-desireable features for GPS data while reducing the number of points
-along straightaways. File size is reduced by a factor of
-about 11. Noise from local stay-in-place drift isn't removed; a
-relative error of about 1000 is required to remove stay-in-place drift
-noise but this also trims all but 100m-size features of the recorded
-path. A relative error of 1000 reduces file size by a factor of
-about 450.
-
-#+BEGIN_EXAMPLE
- 67M relerror-0.001.kml
- 66M relerror-0.01.kml
- 58M relerror-0.1.kml
- 21M relerror-1.kml
-5.8M relerror-10.kml
-797K relerror-100.kml
-152K relerror-1000.kml
-#+END_EXAMPLE
-
-* bkgpslog narrative
-** Initialize environment
-*** Init variables
-**** Save timeStart (YYYYmmddTHHMMSS±zz)
-*** Define Functions
-**** Define Debugging functions
-**** Define Argument Processing function
-**** Define Main function
-** Run Main Function
-*** Process Arguments
-*** Set output encryption and compression option strings
-*** Check that critical apps and dirs are available, displag missing ones.
-*** Set lifespans of script and buffer
-*** Init temp working dir ~DIR_TMP~
-Make temporary dir in tmpfs dir: ~/dev/shm/$(nonce)..bkgpslog/~ (~DIR_TMP~)
-*** Initialize ~tar~ archive
-**** Write ~bkgpslog~ version to ~$DIR_TMP/VERSION~
-**** Create empty ~tar~ archive in ~DIR_OUT~ at ~PATHOUT_TAR~
-
-Set output file name to:
-: PATHOUT_TAR="$DIR_OUT/YYYYmmdd..hostname_location.gz.age.tar"
-Usage: ~iso8601Period $timeStart $timeEnd~
-
-**** Append ~VERSION~ file to ~PATHOUT_TAR~
-
-Append ~$DIR_TMP/VERSION~ to ~PATHOUT_TAR~ via ~tar --append~
-
-*** Read/Write Loop (Record gps data until script lifespan ends)
-**** Determine output file paths
-**** Define GPS conversion commands
-**** Fill Bash variable buffer from ~gpspipe~
-**** Process bufferBash, save secured chunk set to ~DIR_TMP~
-**** Append each secured chunk to ~PATHOUT_TAR~
-: tar --append --directory=DIR_TMP --file=PATHOUT_TAR $(basename PATHOUT_{NMEA,GPX,KML} )
-**** Remove secured chunk from ~DIR_TMP~
projects / EVA-2020-02.git / blobdiff