doc(exec):Create coding plan for bklog, retire bkgpslog plan

Create coding plan for `bklog` at `exec/bklog-plan.org`.

Retire coding plan for `bkgpslog` at `exec/bkgpslog-plan.org`.

diff --git a/exec/bkgpslog-plan.org b/exec/bkgpslog-plan.org
index e1009621d3bad384cc5dbee9826144696b12f0bc..cc5c8390219147363ee8f319111733b706c32f5b 100644 (file)
--- a/exec/bkgpslog-plan.org
+++ b/exec/bkgpslog-plan.org
@@ -1,3 +1,6 @@
+2020-07-12T21:16Z; bktei> Note: This file is now retired since ~bklog~
+has replaced ~bkgpslog~.
+

 * bkgpslog task list
 ** DONE Add job control for short buffer length
    CLOSED: [2020-07-02 Thu 16:04]
 * bkgpslog task list
 ** DONE Add job control for short buffer length
    CLOSED: [2020-07-02 Thu 16:04]


@@ -60,7 +63,8 @@ done
 
 Raspberry Pi Zero W shows approximately 71ms of drift per buffer round
 with 10s buffer.
 
 Raspberry Pi Zero W shows approximately 71ms of drift per buffer round
 with 10s buffer.

-** TODO Feature: Recipient watch folder
+** DONE Feature: Recipient watch folder
+   CLOSED: [2020-07-12 Sun 21:08]

 2020-07-03T21:28Z; bktei> This feature would be to scan the contents
 of a specified directory at the start of every buffer round in order
 to determine encryption (age) recipients. This would allow a device to
 2020-07-03T21:28Z; bktei> This feature would be to scan the contents
 of a specified directory at the start of every buffer round in order
 to determine encryption (age) recipients. This would allow a device to


@@ -95,7 +99,11 @@ to watch. When examining the directory, check for a file with the
 appropriate file extension (ex: .pubkey) and then read the first line
 into the script's pubKey array.
 
 appropriate file extension (ex: .pubkey) and then read the first line
 into the script's pubKey array.
 

-** TODO Feature: Simplify option to reduce output size
+2020-07-12T21:08Z; bktei> ~-R~ watch directory option added in ~bkgpslog~ ver
+~0.4.0~.
+
+** DONE Feature: Simplify option to reduce output size
+   CLOSED: [2020-07-12 Sun 21:15]

 
 ~gpsbabel~ [[https://www.gpsbabel.org/htmldoc-development/filter_simplify.html][features]] a ~simplify~ option to trim data points from GPS
 data. There are several methods for prioritizing which points to keep
 
 ~gpsbabel~ [[https://www.gpsbabel.org/htmldoc-development/filter_simplify.html][features]] a ~simplify~ option to trim data points from GPS
 data. There are several methods for prioritizing which points to keep


@@ -126,7 +134,16 @@ about 450.
 152K relerror-1000.kml
 #+END_EXAMPLE
 
 152K relerror-1000.kml
 #+END_EXAMPLE
 

-** TODO Feature: Generalize bkgpslog to bklog function
+2020-07-12T21:13Z; bktei> Instead of programming data simplification
+in ~bkgpslog~, the data simplification step should be performed via
+~bklog~'s ~-p~ option which specifies a processing command string to
+be ~eval~'d before data is compressed, encrypted, and written to
+disk. In other words, handling the simplification of data beyond
+allowing for a general command string specified by ~-p~ is outside the
+scope of ~bkgpslog~ or its successor ~bklog~.
+
+** DONE Feature: Generalize bkgpslog to bklog function
+   CLOSED: [2020-07-12 Sun 21:11]

 2020-07-05T02:42Z; bktei> Transform ~bkgpslog~ into a modular
 component called ~bklog~ such that it processes a stdout stream of any
 external command, not just ~gpspipe -r~. This would permit reuse of
 2020-07-05T02:42Z; bktei> Transform ~bkgpslog~ into a modular
 component called ~bklog~ such that it processes a stdout stream of any
 external command, not just ~gpspipe -r~. This would permit reuse of


@@ -160,7 +177,12 @@ unsure in my ability to program script that could run for long periods
 of time without causing a runaway usage of memory. I still think it's
 a good idea to offer a script TTL option to the user but I think the
 default should be to simply run forver.
 of time without causing a runaway usage of memory. I still think it's
 a good idea to offer a script TTL option to the user but I think the
 default should be to simply run forver.

-** TODO: Evaluate ~rsyslog~ as stand-in for this work
+
+2020-07-12T21:11Z; bktei> ~bklog~ script created and tested as of
+commit ~aedd19f~.
+
+** DONE TODO: Evaluate ~rsyslog~ as stand-in for this work
+   CLOSED: [2020-07-12 Sun 21:09]

 2020-07-05T02:57Z; bktei> I searched for "debian iot logging" ("iot"
 as in "Internet of Things", the current buzzword for small low-power
 computers being used to provide microservices for owners in their own
 2020-07-05T02:57Z; bktei> I searched for "debian iot logging" ("iot"
 as in "Internet of Things", the current buzzword for small low-power
 computers being used to provide microservices for owners in their own


@@ -222,6 +244,10 @@ think converting ~bkgpslog~ into a ~bklog~ script that appends
 encrypted and compressed data to a tar file for later extraction
 (possibly the same script with future features) would be best.
 
 encrypted and compressed data to a tar file for later extraction
 (possibly the same script with future features) would be best.
 

+2020-07-12T21:10Z; bktei> rsyslog is outside the scope of what
+~bkgpslog~ does (record location observations). A different tool
+should be used to retrieve and synchronize data. The dumb storage
+method of "tar files in a syncthing folder" works for now.

 ** TODO: Place persistent recip. updates in asynchronous coproc
 2020-07-06T19:37Z; bktei> In order to update the recipient list, the
 magicParseRecipientDir() function needs to be run each buffer period
 ** TODO: Place persistent recip. updates in asynchronous coproc
 2020-07-06T19:37Z; bktei> In order to update the recipient list, the
 magicParseRecipientDir() function needs to be run each buffer period

diff --git a/exec/bklog-plan.org b/exec/bklog-plan.org
new file mode 100644 (file)
index 0000000..9908efb
--- /dev/null
+++ b/exec/bklog-plan.org
@@ -0,0 +1,58 @@
+* bklog task list
+** TODO Adjust filename duration dynamically
+2020-07-12T21:17Z; bktei> Currently, the "duration" component of the
+output filename for a given chunk is calculated from the ~bufferTTL~
+variable which does not necessarily reflect the amount of buffer lines
+being read into a given chunk, especially since lines of stdin may be
+received while synchronous processing occurs before a new
+~magicProcessWriteBuffer &~ job is spun off.
+
+A better way to specify the duration is to maintain a pair of
+timestamp varriables before each ~magicProcessWriteBuffer~ command is
+run. Within the asynchronous write job the time difference between the
+two time stamps may be evaluated in order to determine how much time
+has passed since the last write operation started. The last line read
+into the buffer when the ~while read~ loop finishes should be the most
+current value and so one of the two timestamps should be recorded
+then. The other time stamp should simply be the previous loop's
+timestamp value.
+
+For example:
+
+#+BEGIN_EXAMPLE
+    # MAIN LOOP: Run until script TTL seconds pass
+    bufferRound=0;
+    while [[ $SECONDS -lt "scriptTTL" ]]; do
+       vbm "STATUS:$fn:Starting buffer round:$bufferRound";
+       bufferTOD="$((SECONDS + bufferTTL))"; # Set buffer round time-of-death
+       # Consume stdin to fill buffer until buffer time-of-death (TOD) arrives
+       while read -r -t "$bufferTTL" line && [[ $SECONDS -lt "$bufferTOD" ]]; do
+           # Append line to buffer array
+           buffer+=("$line");
+       done;
+        #====BEGIN NEW CODE HERE====
+        bufferTimestampOld="$bufferTimestampNew";
+        bufferTimeStampNew="$(date --iso-8601=seconds)";
+        #====END NEW CODE HERE====
+       # Create dir_tmp if missing
+       if ! [[ -d "$dir_tmp" ]]; then
+           yell "ERROR:$fn:dir_tmp existence failure:$dir_tmp";
+           try mkdir "$dir_tmp" && vbm "DEBUG :$fn:Working dir recreated dir_tmp:$dir_tmp"; fi
+       # Update cmd_encrypt, cmd_encrypt_suffix
+       magicParseRecipients;
+       # Export buffer to asynchronous processing.
+       magicProcessWriteBuffer &
+       unset buffer; # Clear buffer array for next bufferRound
+       # Increment buffer round
+       ((bufferRound++));
+    done;
+#+END_EXAMPLE
+
+Then, within the ~magicProcessWriteBuffer()~ function, the difference
+in seconds between ~bufferTimestampOld~ and ~bufferTimestampNew~ may
+be calculated and an appropriate duration string generated from the
+~timeDuration()~ function.
+
+
+* bklog narrative
+