X-Git-Url: https://zdv2.bktei.com/gitweb/EVA-2020-02.git/blobdiff_plain/6c30388ff903fe9c9f2c31ced53f1f7003031fac..f17265fdc429b2fa2d42d5fe687d671a1b7a731e:/exec/bkgpslog-plan.org?ds=inline

diff --git a/exec/bkgpslog-plan.org b/exec/bkgpslog-plan.org
index bc98f9a..4ba6806 100644
--- a/exec/bkgpslog-plan.org
+++ b/exec/bkgpslog-plan.org
@@ -1,3 +1,131 @@
+* bkgpslog task list
+** DONE Add job control for short buffer length
+   CLOSED: [2020-07-02 Thu 16:04]
+2020-07-02T14:56Z; bktei> File write operations were bundled into a
+magicWriteBuffer function that is called then detached from the script
+shell (job control), but the detached job is not tracked by the main
+script. A problem may arise if two instances of magicWriteBuffer
+attempt to write to the same tar simultaneously. Two instances of
+magicWriteBuffer may exist if the buffer length is low (ex: 1 second);
+the default buffer length of 60 seconds should reduce the probability
+of a collision but it should be possible for the main script to track
+the process ID of a magicWriteBuffer() as soon as it detaches and then
+checking (via ~$!~ as described [[https://bashitout.com/2013/05/18/Ampersands-on-the-command-line.html][here]]) that the process is still alive.
+2020-07-02T15:23Z; bktei> I found that the Bash ~wait~ built-in can be
+used to delay processing until a specified job completes. The ~wait~
+command will pause script execution until all backgrounded processes
+complete.
+2020-07-02T16:03Z; bktei> Added ~wait~.
+** DONE Rewrite tar initialization function
+   CLOSED: [2020-07-02 Thu 17:23]
+2020-07-02T17:23Z; bktei> Simplify tar initialization function so
+VERSION file is used to test appendability of tar as well as to mark
+when a new session is started.
+** DONE Consolidate tar checking/creation into function
+   CLOSED: [2020-07-02 Thu 18:33]
+2020-07-02T18:33Z; bktei> Simplify how the output tar file's existence
+is checked and its status as a valid tar file is validated. This was
+done using a new function ~checkMakeTar~.
+** DONE Add VERSION if output tar deleted between writes
+
+   CLOSED: [2020-07-02 Thu 20:22]
+2020-07-02T20:21Z; bktei> Added bkgpslog-specified function
+magicWriteVersion() to be called whenever a new time-stamped ~VERSION~
+file needs to be generated and appended to the output tar file
+~PATHOUT_TAR~.
+** DONE Rewrite buffer loop to reduce lag between gpspipe runs
+
+   CLOSED: [2020-07-03 Fri 20:57]
+2020-07-03T17:10Z; bktei> As is, there is still a 5-6 second lag
+between when ~gpspipe~ times out at the end of a buffer round and when
+~gpspipe~ is called by the subsequent buffer round. I believe this can
+be reduced by moving variable manipulations inside the
+asynchronously-executed magicWriteBuffer() function. Ideally, the
+while loop should look like:
+
+#+BEGIN_EXAMPLE
+while( $SECONDS < $SCRIPT_TTL); do
+    gpspipe-r > "$DIR_TMP"/buffer.nmea
+    writeBuffer &
+done
+#+END_EXAMPLE
+2020-07-03T20:56Z; bktei> I simplified it futher to something like
+this:
+#+BEGIN_EXAMPLE
+while( $SECONDS < $SCRIPT_TTL); do
+    writeBuffer &
+    sleep $SCRIPT_TTL
+done
+#+END_EXAMPLE
+
+Raspberry Pi Zero W shows approximately 71ms of drift per buffer round
+with 10s buffer.
+** TODO Feature: Recipient watch folder
+2020-07-03T21:28Z; bktei> This feature would be to scan the contents
+of a specified directory at the start of every buffer round in order
+to determine encryption (age) recipients. This would allow a device to
+dynamically encrypt location data in response to automated changes
+made by other tools. For example, if such a directory were
+synchronized via Syncthing and changes to such a directory were
+managed by a trusted remote server, then that server could respond to
+human requests to secure location data.
+
+Two specific privacy subfeatures come to mind:
+
+1. Parallel encryption: Given a set of ~n~ public keys, encrypt data
+   with a single ~age~ command with options causing all ~n~ pubkeys to
+   be recipients. In order to decrypt the data, any individual private
+   key could be used. No coordination between key owners would be
+   required to decrypt.
+
+2. Sequential encryption: Given a set of ~n~ public keys, encrypt data
+   with ~n~ sequential ~age~ commands all piped in series with each
+   ~age~ command utilizing only one of the ~n~ public keys. In order
+   to decrypt the data, all ~n~ private keys would be required to
+   decrypt the data. Since coordination is required, it is less
+   convenient than parallel encryption.
+
+In either case, a directory would be useful for holding configuration
+files specifying how to execute which or combination of which features
+at the start of every buffer round.
+
+I don't yet know how to program the rules, although I think it'd be
+easier to simply add an option providing ~bkgpslog~ with a directory
+to watch. When examining the directory, check for a file with the
+appropriate file extension (ex: .pubkey) and then read the first line
+into the script's pubKey array.
+
+** TODO Feature: Simplify option to reduce output size
+
+~gpsbabel~ [[https://www.gpsbabel.org/htmldoc-development/filter_simplify.html][features]] a ~simplify~ option to trim data points from GPS
+data. There are several methods for prioritizing which points to keep
+and which to trim, although the following seems useful given some
+sample data I've recorded in a test run of ninfacyzga-01:
+
+#+BEGIN_EXAMPLE
+gpsbabel -i nmea -f all.nmea -x simplify,error=10,relative -o gpx \
+-F all-simp-rel-10.gpx
+#+END_EXAMPLE
+
+An error level of "10" with the "relative" option seems to retain all
+desireable features for GPS data while reducing the number of points
+along straightaways. File size is reduced by a factor of
+about 11. Noise from local stay-in-place drift isn't removed; a
+relative error of about 1000 is required to remove stay-in-place drift
+noise but this also trims all but 100m-size features of the recorded
+path. A relative error of 1000 reduces file size by a factor of
+about 450.
+
+#+BEGIN_EXAMPLE
+ 67M relerror-0.001.kml
+ 66M relerror-0.01.kml
+ 58M relerror-0.1.kml
+ 21M relerror-1.kml
+5.8M relerror-10.kml
+797K relerror-100.kml
+152K relerror-1000.kml
+#+END_EXAMPLE
+
 * bkgpslog narrative
 ** Initialize environment
 *** Init variables