Steven Baltakatei Sandoval [Thu, 16 Jul 2020 01:01:44 +0000 (01:01 +0000)]
feat(README):Convert main README to org file from markdown
Steven Baltakatei Sandoval [Wed, 15 Jul 2020 23:51:21 +0000 (23:51 +0000)]
doc(bklog-plan):Note possible future feature: user config file
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 22:27:50 +0000 (22:27 +0000)]
fix(bklog):Update plan, bump version (bklog working)
Complete fix for `bklog` duration string bug.
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 19:47:38 +0000 (19:47 +0000)]
fix(bklog):Fix initial edge case bug in duration time
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 19:40:03 +0000 (19:40 +0000)]
fix(bklog):Fix flip-flopped buffer start/end timestamps
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 17:23:40 +0000 (17:23 +0000)]
debug(bklog):Testing duration code
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 17:20:42 +0000 (17:20 +0000)]
debug(bklog):Debugging duration code
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 16:58:48 +0000 (16:58 +0000)]
fix(bklog):Fix gap in duration times
Gap caused by not accounting for synchronous calcs between timestamps
marking duration. Using method planned in `bklog-plan.org`.
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 03:23:37 +0000 (03:23 +0000)]
fix(bklog):timeDuration():Fix duration 0 error.
Duration of "0" was erroneously being converted to "P" instead of
"PT0S".
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 03:08:18 +0000 (03:08 +0000)]
doc(bklog-plan):Add TODO item:account for input script failure
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 03:00:09 +0000 (03:00 +0000)]
Merge branch 'develop' into feature/bash-logging-filter/BK-2020-02
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 02:44:05 +0000 (02:44 +0000)]
debug(bklog):Fix SECONDS advancing
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 02:39:33 +0000 (02:39 +0000)]
debug(bklog):Add minor debug msg
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 02:31:40 +0000 (02:31 +0000)]
debug(bklog):Add debug messages
bklog is malfunctioning. Adding debug messages.
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 02:08:49 +0000 (02:08 +0000)]
fix(bklog):Fix buffer drift via SECONDS
==Change 1==
Fix drift in when bklog starts reading data via `while read` by
enforcing buffer termination when SECONDS (script variable that
increments every second) rises above a number determined by the
current bufferRound number and bufferTTL.
==Change 2==
Also initially advance SECONDS so it aligns with the remainder of
dividing the current seconds remaining in the day by bufferTTL. The
result should be that, if the user specifies a bufferTTL value that is
a clean fraction of a time element (ex: a bufferTTL of 5 minutes = 300
seconds), then each buffer round should start at a clean fraction of
absolute time with respect to the day (ex: 02:05, 02:10, 02:15,
etc.). This may assist searching through saved files since instead of
files named:
20200714T000019+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T000521+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T001023+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T001525+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T002027+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T002529+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T003031+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T003533+0000--PT5M0S..hostname_location.nmea.gz.age
they will be named something more like:
20200714T000000+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T000500+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T001000+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T001500+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T002000+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T002500+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T003000+0000--PT5M0S..hostname_location.nmea.gz.age
20200714T003500+0000--PT5M0S..hostname_location.nmea.gz.age
==Change 3==
Fix how the ISO-8601 duration string is calculated. Instead of using a
fixed value of `bufferTTL` seconds, record the epoch start and end
times (in seconds since 1970-01-01) of the main data gathering loop
and then calculate the difference. Now, actual deviations in buffer
gathering durations (ex: the first buffer round will likely be shorter
than a normal one) will be accurately represented.
Steven Baltakatei Sandoval [Tue, 14 Jul 2020 01:03:27 +0000 (01:03 +0000)]
doc(bklog-plan):Add TODO task: correct drift of SECONDS
Steven Baltakatei Sandoval [Mon, 13 Jul 2020 22:31:14 +0000 (22:31 +0000)]
fix(bklog):Add 2nd try to checkMakeTar() if tar --list fails
Steven Baltakatei Sandoval [Sun, 12 Jul 2020 21:35:16 +0000 (21:35 +0000)]
doc(exec):Create coding plan for bklog, retire bkgpslog plan
Create coding plan for `bklog` at `exec/bklog-plan.org`.
Retire coding plan for `bkgpslog` at `exec/bkgpslog-plan.org`.
Steven Baltakatei Sandoval [Sun, 12 Jul 2020 21:05:23 +0000 (21:05 +0000)]
doc(bklog):Update location readme with bklog details
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 09:28:00 +0000 (09:28 +0000)]
chore(bklog):Update vbm() to include script name
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 09:09:31 +0000 (09:09 +0000)]
fix(pressure):bmp388.py:Make executable
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 08:51:45 +0000 (08:51 +0000)]
doc(pressure):Note python-smbus dependency for bmp388.py script
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 08:42:46 +0000 (08:42 +0000)]
feat(pressure):Add python script for outputting pressure to stdout
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 08:23:00 +0000 (08:23 +0000)]
fix(bklog):Fix debug and status messages
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 08:02:18 +0000 (08:02 +0000)]
test(bklog):Uncomment cleanup line since script functional
Use of appendFileTar() to append a file to a tar using the file's same
location as a temporary working file caused problems. More direct
'eval' command string formation and 'tar --append' usage solved empty
file problem.
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 07:54:48 +0000 (07:54 +0000)]
debug(bklog):Test tar --append
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 07:51:48 +0000 (07:51 +0000)]
debug(bklog):Test more direct tar --append operation
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 07:37:04 +0000 (07:37 +0000)]
chore(bklog):Bump version for recent commit change.
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 07:35:43 +0000 (07:35 +0000)]
fix(bklog):Add missing appendArgTar() function
Note:Used by magicWriteVersion() function.
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 07:33:28 +0000 (07:33 +0000)]
debug(bklog):Fix version appending; test filewrite change
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 07:10:45 +0000 (07:10 +0000)]
debug(bklog):Debugging write command
Note: bklog appending appropriately named files to tar but they are
all empty.
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 06:43:07 +0000 (06:43 +0000)]
debug(bklog):Add debug msgs in magicParseProcessStrings
Add debug msgs to try and debug error in procStrings and procFileExts
arrays.
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 06:39:19 +0000 (06:39 +0000)]
fix(bklog):Fix how procStrings and procFileExts arrays assembled
Default raw data was not being included in output file processing
since procStrings and procFileExts arrays were being completely
overwritten instead of being appended by argProcStrings array
assembled in magicParseProcessStrings() function.
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 06:21:32 +0000 (06:21 +0000)]
style(bklog):Combine functions that parse '-r' and '-R'
Combine functions magicParseRecipientArgs() and
magicParseRecipientDir() into magicParseRecipients().
Group showUsage() function with other script-specific functions.
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 05:00:08 +0000 (05:00 +0000)]
debug(bklog):Format DEBUG messages to align better
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:58:57 +0000 (04:58 +0000)]
debug(bklog):Add debug msgs to write function
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:51:21 +0000 (04:51 +0000)]
debug(bklog):Add various debug msgs with function names
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:32:26 +0000 (04:32 +0000)]
fix(bklog):Add missing function dateTimeShort()
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:30:51 +0000 (04:30 +0000)]
test(bklog):Add debug message in main loop
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:24:41 +0000 (04:24 +0000)]
test(bklog):Add debug messages in magic functions
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:17:53 +0000 (04:17 +0000)]
debug(bklog):Remove debug msgs from main loop
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:16:03 +0000 (04:16 +0000)]
fix(bklog):Convert debug echos into vbm()
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:13:32 +0000 (04:13 +0000)]
fix(bklog):Add missing functions, remove ref to removed func
+checkMakeTar()
+timeUntilNextDay()
+timeUntilNextHour()
-magicParseStoreRaw() ref
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:10:40 +0000 (04:10 +0000)]
fix(bklog):Fix boolean '-' test in magicParseProcessStrings()
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:07:18 +0000 (04:07 +0000)]
test(bklog):Add debug msg to magicParseProcessStrings()
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:04:49 +0000 (04:04 +0000)]
test(bklog):Add debug msgs to magicParseProcessStrings()
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 04:01:45 +0000 (04:01 +0000)]
fix(bklog):Add missing validateInput()
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 03:59:12 +0000 (03:59 +0000)]
fix(bklog):Add missing checkAgePubkey(), reorder functions
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 03:53:01 +0000 (03:53 +0000)]
chore(bklog):Remove extraneous echoerr references
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 03:46:45 +0000 (03:46 +0000)]
test(bklog):Minor usage correction
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 01:19:19 +0000 (01:19 +0000)]
fix(bklog):Add missing function timeDuration() from bkgpslog
Rearrange function declarations
Steven Baltakatei Sandoval [Sat, 11 Jul 2020 01:11:02 +0000 (01:11 +0000)]
feat(bklog):Copy bkgpslog functions, rewrite main
Steven Baltakatei Sandoval [Fri, 10 Jul 2020 05:42:25 +0000 (05:42 +0000)]
Merge branch 'develop' into feature/bash-logging-filter/BK-2020-02
Steven Baltakatei Sandoval [Fri, 10 Jul 2020 05:29:19 +0000 (05:29 +0000)]
style(bkgpslog):Change VAR_NAMES to varNames
Change all-caps variable names to mix of lowercase and uppercase names
in order to avoid naming conflicts with environment variables.
Ref: https://mywiki.wooledge.org/BashGuide/InputAndOutput
Steven Baltakatei Sandoval [Fri, 10 Jul 2020 02:44:45 +0000 (02:44 +0000)]
chore(bkgpslog):Change allcaps varnames to lower/uppercase
Steven Baltakatei Sandoval [Fri, 10 Jul 2020 02:32:50 +0000 (02:32 +0000)]
chore(bklog):Add more metadata
Steven Baltakatei Sandoval [Fri, 10 Jul 2020 02:26:56 +0000 (02:26 +0000)]
feat(bklog):Add metadata, version
Steven Baltakatei Sandoval [Fri, 10 Jul 2020 02:10:29 +0000 (02:10 +0000)]
feat(bklog):Create barebones logging script for general logger
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 23:59:56 +0000 (23:59 +0000)]
fix(bkgpslog):Recreate working dir if missing
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 23:51:32 +0000 (23:51 +0000)]
test(bkgpslog):Add sticky bit to DIR_TMP
Add debug message when DIR_TMP removal is completed.
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 20:32:00 +0000 (20:32 +0000)]
style(bkgpslog):Minor boilerplate modification
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 20:10:47 +0000 (20:10 +0000)]
fix(bkgpslog):Fix PATHOUT_TAR filename dateShort() call
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 19:47:15 +0000 (19:47 +0000)]
test(bkgpslog):Test date format bug
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 18:55:21 +0000 (18:55 +0000)]
test(bkgpslog):Add debug messages
Bug still exists if `bkgpslog` run by cron instead of directly by
user.
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 13:38:43 +0000 (13:38 +0000)]
test(bkgpslog):Minor debug msg
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 12:32:25 +0000 (12:32 +0000)]
test(bkgpslog):Add debug tests
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 12:15:58 +0000 (12:15 +0000)]
chore(bkgpslog):Minor verbose message fix
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 12:02:47 +0000 (12:02 +0000)]
fix(bkgpslog):Make static initial BUFFER_TTL dynamic
Static value was used during debugging but should have been made
dynamic.
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 11:45:52 +0000 (11:45 +0000)]
chore(bkgpslog):Add minor comment
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 11:37:53 +0000 (11:37 +0000)]
feat(bkgpslog):Add PID control loop for buffer start sched
Add PID loop to adjust time delay between when buffer read-write jobs
are spun off in main while loop. The adjustments are made in order to
keep startup times of write jobs BUFFER_TTL seconds apart. This PID
loop calculation dynamically accounts for any future synchronous
calculations that may be added in the main while loop (including
itself).
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 11:35:36 +0000 (11:35 +0000)]
Merge branch 'feature/location-logging/EVA-2020-02' into develop
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 10:39:43 +0000 (10:39 +0000)]
fix(bkgpslog):Remove debug sleep
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 10:36:03 +0000 (10:36 +0000)]
test(bkgpslog):More debug msgs
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 10:16:38 +0000 (10:16 +0000)]
test(bkgpslog):Add debug messages
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 10:04:57 +0000 (10:04 +0000)]
fix(bkgpslog):fix pid float calc
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 10:02:51 +0000 (10:02 +0000)]
test(bkgpslog):more pid float testing
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 09:59:26 +0000 (09:59 +0000)]
fix(bkgpslog):Fix float calc in buffer PID adjust
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 09:37:07 +0000 (09:37 +0000)]
test(bkgpslog):More PID testing
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 09:30:07 +0000 (09:30 +0000)]
test(bkgpslog):Fix debug msgs
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 09:27:46 +0000 (09:27 +0000)]
test(bkgpslog):Test different PID calc
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 09:13:17 +0000 (09:13 +0000)]
test(bkgpslog):debug:Shift magicBufferSleepPID
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 09:04:33 +0000 (09:04 +0000)]
test(bkgpslog):Add debug code and negative int catch
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 08:45:42 +0000 (08:45 +0000)]
test(bkgpslog):Debug testing PID loop
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 08:31:59 +0000 (08:31 +0000)]
feat(bkgpslog):Add PID correction of buffer round timing
Add function to apply PID (proportional, integral, derivative)
adjustments to the sleep period between each buffer round in order to
compensate for lag caused by synchronous calculations between each
buffer round (including the PID adjustment calculation itself).
Steven Baltakatei Sandoval [Tue, 7 Jul 2020 01:48:52 +0000 (01:48 +0000)]
fix(bkgpslog):Remove nuisance error if -R not set
Error was appearing if -e were set but -R wasn't. -R is not a required
option.
Steven Baltakatei Sandoval [Mon, 6 Jul 2020 20:01:18 +0000 (20:01 +0000)]
fix(bkgpslog):Fix recPubKeysValid array bloat
Implemented recPubKeysValidStatic array for magicParseRecipientDir to
process instead of recPubKeysValid array.
Steven Baltakatei Sandoval [Mon, 6 Jul 2020 19:47:03 +0000 (19:47 +0000)]
fix(bkgpslog):Actually add recipient watch dir check
Steven Baltakatei Sandoval [Mon, 6 Jul 2020 19:10:32 +0000 (19:10 +0000)]
test(bkgpslog):Add verbose debug msg to recipient dir code
Steven Baltakatei Sandoval [Mon, 6 Jul 2020 19:02:09 +0000 (19:02 +0000)]
fix(bkgpslog):Fix error in encryption option handling, bump ver
Note: 0.4.0 version includes new '-b', '-B' options for specifying buffer and
script time-to-live values.
Note: 0.4.0 version includes new '-R' option for specifying directory
containing files specifying recipient pubkeys.
Steven Baltakatei Sandoval [Mon, 6 Jul 2020 18:42:14 +0000 (18:42 +0000)]
feat(bkgpslog):Enable custom TTL for buffer and script
New options: '-b' (buffer), '-B' (script).
Updated showUsage().
Steven Baltakatei Sandoval [Mon, 6 Jul 2020 16:55:37 +0000 (16:55 +0000)]
fix(bkgpslog):Fix broken tempdir var name
Steven Baltakatei Sandoval [Sun, 5 Jul 2020 17:31:07 +0000 (17:31 +0000)]
fix(bkgpslog):Add '-e'/'-R' checks in -R parse
Also add comments to magic functions in main()
Steven Baltakatei Sandoval [Sun, 5 Jul 2020 17:17:36 +0000 (17:17 +0000)]
fix(bkgpslog):Check -e, -r, -R option combinations; update usage
Steven Baltakatei Sandoval [Sun, 5 Jul 2020 16:56:56 +0000 (16:56 +0000)]
feat(bkgpslog):Add functions, group code into functions
Add several fucntions to add future new feature (recipient dir watch)
as well as to group existing code blocks into their own functions for
readability. Changes include:
* Add checkAgePubkey() function: checks that a provided string doesn't
throw an error when fed as a recipient string to `age`.
* Pushed several code blocks into their own "magic" functions
** magicParseRecipientDir() : (future feature): To be run at each
buffer session to update the recipient array in response to changes in
a specified recipient directory.
** magicParseRecipientArgs(): Parses strings provided by '-r'
option and format output accordingly. Existing code.
** magicParseCompressionArgs(): Responds to '-c' option by specifying
compression function and output file suffix.
* Update showUsage() to document future '-R' recipient dir option.
Steven Baltakatei Sandoval [Sat, 4 Jul 2020 22:13:09 +0000 (22:13 +0000)]
Merge branch 'feature/location-logging/EVA-2020-02' into develop
Steven Baltakatei Sandoval [Sat, 4 Jul 2020 22:12:43 +0000 (22:12 +0000)]
chore(bkgpslog):Bump version number to include recent change
Steven Baltakatei Sandoval [Sat, 4 Jul 2020 22:11:45 +0000 (22:11 +0000)]
Merge branch 'feature/location-logging/EVA-2020-02' into develop
Steven Baltakatei Sandoval [Sat, 4 Jul 2020 21:46:37 +0000 (21:46 +0000)]
feat(bkgpslog):Add additional validation for age pubkeys
Steven Baltakatei Sandoval [Sat, 4 Jul 2020 00:32:15 +0000 (00:32 +0000)]
chore(bkgpslog):Confirm age can encrypt to ssh pubkey string
Ssh pubkey strings have a space in them which was breaking
appendFileTar() function. 'eval' was used to fix how appendFileTar()
and appendArgTar() execute command strings.
Steven Baltakatei Sandoval [Sat, 4 Jul 2020 00:22:45 +0000 (00:22 +0000)]
fix(bkgpslog):Use eval to build command strings
Use eval to build command strings instead of simply putting bare
unquoted variable. This is necessary because the `recipients` variable
used to build a command to encrypt files may contain quotes if one of
the recipients is an ssh key which may have a space:
ssh-rsa AAAAB...
This should fix the inability of `bkgpslog` to accept quoted ssh
pubkey strings as a specified `-r` option argument.