fix(bkgpslog):Increase buffer period 60s to 5m to reduce write freq
[EVA-2020-02.git] / exec / bkgpslog-plan.org
... / ...
CommitLineData
1* bkgpslog task list
2** DONE Add job control for short buffer length
3 CLOSED: [2020-07-02 Thu 16:04]
42020-07-02T14:56Z; bktei> File write operations were bundled into a
5magicWriteBuffer function that is called then detached from the script
6shell (job control), but the detached job is not tracked by the main
7script. A problem may arise if two instances of magicWriteBuffer
8attempt to write to the same tar simultaneously. Two instances of
9magicWriteBuffer may exist if the buffer length is low (ex: 1 second);
10the default buffer length of 60 seconds should reduce the probability
11of a collision but it should be possible for the main script to track
12the process ID of a magicWriteBuffer() as soon as it detaches and then
13checking (via ~$!~ as described [[https://bashitout.com/2013/05/18/Ampersands-on-the-command-line.html][here]]) that the process is still alive.
142020-07-02T15:23Z; bktei> I found that the Bash ~wait~ built-in can be
15used to delay processing until a specified job completes. The ~wait~
16command will pause script execution until all backgrounded processes
17complete.
182020-07-02T16:03Z; bktei> Added ~wait~.
19** DONE Rewrite tar initialization function
20 CLOSED: [2020-07-02 Thu 17:23]
212020-07-02T17:23Z; bktei> Simplify tar initialization function so
22VERSION file is used to test appendability of tar as well as to mark
23when a new session is started.
24** DONE Consolidate tar checking/creation into function
25 CLOSED: [2020-07-02 Thu 18:33]
262020-07-02T18:33Z; bktei> Simplify how the output tar file's existence
27is checked and its status as a valid tar file is validated. This was
28done using a new function ~checkMakeTar~.
29** DONE Add VERSION if output tar deleted between writes
30 CLOSED: [2020-07-02 Thu 20:22]
312020-07-02T20:21Z; bktei> Added bkgpslog-specified function
32magicWriteVersion() to be called whenever a new time-stamped ~VERSION~
33file needs to be generated and appended to the output tar file
34~PATHOUT_TAR~.
35* bkgpslog narrative
36** Initialize environment
37*** Init variables
38**** Save timeStart (YYYYmmddTHHMMSS±zz)
39*** Define Functions
40**** Define Debugging functions
41**** Define Argument Processing function
42**** Define Main function
43** Run Main Function
44*** Process Arguments
45*** Set output encryption and compression option strings
46*** Check that critical apps and dirs are available, displag missing ones.
47*** Set lifespans of script and buffer
48*** Init temp working dir ~DIR_TMP~
49Make temporary dir in tmpfs dir: ~/dev/shm/$(nonce)..bkgpslog/~ (~DIR_TMP~)
50*** Initialize ~tar~ archive
51**** Write ~bkgpslog~ version to ~$DIR_TMP/VERSION~
52**** Create empty ~tar~ archive in ~DIR_OUT~ at ~PATHOUT_TAR~
53
54Set output file name to:
55: PATHOUT_TAR="$DIR_OUT/YYYYmmdd..hostname_location.gz.age.tar"
56Usage: ~iso8601Period $timeStart $timeEnd~
57
58**** Append ~VERSION~ file to ~PATHOUT_TAR~
59
60Append ~$DIR_TMP/VERSION~ to ~PATHOUT_TAR~ via ~tar --append~
61
62*** Read/Write Loop (Record gps data until script lifespan ends)
63**** Determine output file paths
64**** Define GPS conversion commands
65**** Fill Bash variable buffer from ~gpspipe~
66**** Process bufferBash, save secured chunk set to ~DIR_TMP~
67**** Append each secured chunk to ~PATHOUT_TAR~
68: tar --append --directory=DIR_TMP --file=PATHOUT_TAR $(basename PATHOUT_{NMEA,GPX,KML} )
69**** Remove secured chunk from ~DIR_TMP~