+2020-07-12T21:16Z; bktei> Note: This file is now retired since ~bklog~
+has replaced ~bkgpslog~.
+
* bkgpslog task list
** DONE Add job control for short buffer length
CLOSED: [2020-07-02 Thu 16:04]
Raspberry Pi Zero W shows approximately 71ms of drift per buffer round
with 10s buffer.
-** TODO Feature: Recipient watch folder
+** DONE Feature: Recipient watch folder
+ CLOSED: [2020-07-12 Sun 21:08]
2020-07-03T21:28Z; bktei> This feature would be to scan the contents
of a specified directory at the start of every buffer round in order
to determine encryption (age) recipients. This would allow a device to
appropriate file extension (ex: .pubkey) and then read the first line
into the script's pubKey array.
-** TODO Feature: Simplify option to reduce output size
+2020-07-12T21:08Z; bktei> ~-R~ watch directory option added in ~bkgpslog~ ver
+~0.4.0~.
+
+** DONE Feature: Simplify option to reduce output size
+ CLOSED: [2020-07-12 Sun 21:15]
~gpsbabel~ [[https://www.gpsbabel.org/htmldoc-development/filter_simplify.html][features]] a ~simplify~ option to trim data points from GPS
data. There are several methods for prioritizing which points to keep
152K relerror-1000.kml
#+END_EXAMPLE
-** TODO Feature: Generalize bkgpslog to bklog function
+2020-07-12T21:13Z; bktei> Instead of programming data simplification
+in ~bkgpslog~, the data simplification step should be performed via
+~bklog~'s ~-p~ option which specifies a processing command string to
+be ~eval~'d before data is compressed, encrypted, and written to
+disk. In other words, handling the simplification of data beyond
+allowing for a general command string specified by ~-p~ is outside the
+scope of ~bkgpslog~ or its successor ~bklog~.
+
+** DONE Feature: Generalize bkgpslog to bklog function
+ CLOSED: [2020-07-12 Sun 21:11]
2020-07-05T02:42Z; bktei> Transform ~bkgpslog~ into a modular
component called ~bklog~ such that it processes a stdout stream of any
external command, not just ~gpspipe -r~. This would permit reuse of
of time without causing a runaway usage of memory. I still think it's
a good idea to offer a script TTL option to the user but I think the
default should be to simply run forver.
-** TODO: Evaluate ~rsyslog~ as stand-in for this work
+
+2020-07-12T21:11Z; bktei> ~bklog~ script created and tested as of
+commit ~aedd19f~.
+
+** DONE TODO: Evaluate ~rsyslog~ as stand-in for this work
+ CLOSED: [2020-07-12 Sun 21:09]
2020-07-05T02:57Z; bktei> I searched for "debian iot logging" ("iot"
as in "Internet of Things", the current buzzword for small low-power
computers being used to provide microservices for owners in their own
encrypted and compressed data to a tar file for later extraction
(possibly the same script with future features) would be best.
+2020-07-12T21:10Z; bktei> rsyslog is outside the scope of what
+~bkgpslog~ does (record location observations). A different tool
+should be used to retrieve and synchronize data. The dumb storage
+method of "tar files in a syncthing folder" works for now.
** TODO: Place persistent recip. updates in asynchronous coproc
2020-07-06T19:37Z; bktei> In order to update the recipient list, the
magicParseRecipientDir() function needs to be run each buffer period
--- /dev/null
+* bklog task list
+** TODO Adjust filename duration dynamically
+2020-07-12T21:17Z; bktei> Currently, the "duration" component of the
+output filename for a given chunk is calculated from the ~bufferTTL~
+variable which does not necessarily reflect the amount of buffer lines
+being read into a given chunk, especially since lines of stdin may be
+received while synchronous processing occurs before a new
+~magicProcessWriteBuffer &~ job is spun off.
+
+A better way to specify the duration is to maintain a pair of
+timestamp varriables before each ~magicProcessWriteBuffer~ command is
+run. Within the asynchronous write job the time difference between the
+two time stamps may be evaluated in order to determine how much time
+has passed since the last write operation started. The last line read
+into the buffer when the ~while read~ loop finishes should be the most
+current value and so one of the two timestamps should be recorded
+then. The other time stamp should simply be the previous loop's
+timestamp value.
+
+For example:
+
+#+BEGIN_EXAMPLE
+ # MAIN LOOP: Run until script TTL seconds pass
+ bufferRound=0;
+ while [[ $SECONDS -lt "scriptTTL" ]]; do
+ vbm "STATUS:$fn:Starting buffer round:$bufferRound";
+ bufferTOD="$((SECONDS + bufferTTL))"; # Set buffer round time-of-death
+ # Consume stdin to fill buffer until buffer time-of-death (TOD) arrives
+ while read -r -t "$bufferTTL" line && [[ $SECONDS -lt "$bufferTOD" ]]; do
+ # Append line to buffer array
+ buffer+=("$line");
+ done;
+ #====BEGIN NEW CODE HERE====
+ bufferTimestampOld="$bufferTimestampNew";
+ bufferTimeStampNew="$(date --iso-8601=seconds)";
+ #====END NEW CODE HERE====
+ # Create dir_tmp if missing
+ if ! [[ -d "$dir_tmp" ]]; then
+ yell "ERROR:$fn:dir_tmp existence failure:$dir_tmp";
+ try mkdir "$dir_tmp" && vbm "DEBUG :$fn:Working dir recreated dir_tmp:$dir_tmp"; fi
+ # Update cmd_encrypt, cmd_encrypt_suffix
+ magicParseRecipients;
+ # Export buffer to asynchronous processing.
+ magicProcessWriteBuffer &
+ unset buffer; # Clear buffer array for next bufferRound
+ # Increment buffer round
+ ((bufferRound++));
+ done;
+#+END_EXAMPLE
+
+Then, within the ~magicProcessWriteBuffer()~ function, the difference
+in seconds between ~bufferTimestampOld~ and ~bufferTimestampNew~ may
+be calculated and an appropriate duration string generated from the
+~timeDuration()~ function.
+
+
+* bklog narrative
+
projects / EVA-2020-02.git / commitdiff