X-Git-Url: https://zdv2.bktei.com/gitweb/EVA-2020-02.git/blobdiff_plain/f75428fef49de12c858fe5c8dd0ac4fca1d09a82..2038018c4d5ddbc5cc3502ac4a4235f13f017d26:/exec/bkgpslog-plan.org?ds=inline diff --git a/exec/bkgpslog-plan.org b/exec/bkgpslog-plan.org index b7cc736..3e43633 100644 --- a/exec/bkgpslog-plan.org +++ b/exec/bkgpslog-plan.org @@ -33,7 +33,9 @@ done using a new function ~checkMakeTar~. magicWriteVersion() to be called whenever a new time-stamped ~VERSION~ file needs to be generated and appended to the output tar file ~PATHOUT_TAR~. -** TODO Rewrite buffer loop to reduce lag between gpspipe runs +** DONE Rewrite buffer loop to reduce lag between gpspipe runs + + CLOSED: [2020-07-03 Fri 20:57] 2020-07-03T17:10Z; bktei> As is, there is still a 5-6 second lag between when ~gpspipe~ times out at the end of a buffer round and when ~gpspipe~ is called by the subsequent buffer round. I believe this can @@ -47,6 +49,178 @@ while( $SECONDS < $SCRIPT_TTL); do writeBuffer & done #+END_EXAMPLE +2020-07-03T20:56Z; bktei> I simplified it futher to something like +this: +#+BEGIN_EXAMPLE +while( $SECONDS < $SCRIPT_TTL); do + writeBuffer & + sleep $SCRIPT_TTL +done +#+END_EXAMPLE + +Raspberry Pi Zero W shows approximately 71ms of drift per buffer round +with 10s buffer. +** TODO Feature: Recipient watch folder +2020-07-03T21:28Z; bktei> This feature would be to scan the contents +of a specified directory at the start of every buffer round in order +to determine encryption (age) recipients. This would allow a device to +dynamically encrypt location data in response to automated changes +made by other tools. For example, if such a directory were +synchronized via Syncthing and changes to such a directory were +managed by a trusted remote server, then that server could respond to +human requests to secure location data. + +Two specific privacy subfeatures come to mind: + +1. Parallel encryption: Given a set of ~n~ public keys, encrypt data + with a single ~age~ command with options causing all ~n~ pubkeys to + be recipients. In order to decrypt the data, any individual private + key could be used. No coordination between key owners would be + required to decrypt. + +2. Sequential encryption: Given a set of ~n~ public keys, encrypt data + with ~n~ sequential ~age~ commands all piped in series with each + ~age~ command utilizing only one of the ~n~ public keys. In order + to decrypt the data, all ~n~ private keys would be required to + decrypt the data. Since coordination is required, it is less + convenient than parallel encryption. + +In either case, a directory would be useful for holding configuration +files specifying how to execute which or combination of which features +at the start of every buffer round. + +I don't yet know how to program the rules, although I think it'd be +easier to simply add an option providing ~bkgpslog~ with a directory +to watch. When examining the directory, check for a file with the +appropriate file extension (ex: .pubkey) and then read the first line +into the script's pubKey array. + +** TODO Feature: Simplify option to reduce output size + +~gpsbabel~ [[https://www.gpsbabel.org/htmldoc-development/filter_simplify.html][features]] a ~simplify~ option to trim data points from GPS +data. There are several methods for prioritizing which points to keep +and which to trim, although the following seems useful given some +sample data I've recorded in a test run of ninfacyzga-01: + +#+BEGIN_EXAMPLE +gpsbabel -i nmea -f all.nmea -x simplify,error=10,relative -o gpx \ +-F all-simp-rel-10.gpx +#+END_EXAMPLE + +An error level of "10" with the "relative" option seems to retain all +desireable features for GPS data while reducing the number of points +along straightaways. File size is reduced by a factor of +about 11. Noise from local stay-in-place drift isn't removed; a +relative error of about 1000 is required to remove stay-in-place drift +noise but this also trims all but 100m-size features of the recorded +path. A relative error of 1000 reduces file size by a factor of +about 450. + +#+BEGIN_EXAMPLE + 67M relerror-0.001.kml + 66M relerror-0.01.kml + 58M relerror-0.1.kml + 21M relerror-1.kml +5.8M relerror-10.kml +797K relerror-100.kml +152K relerror-1000.kml +#+END_EXAMPLE + +** TODO Feature: Generalize bkgpslog to bklog function +2020-07-05T02:42Z; bktei> Transform ~bkgpslog~ into a modular +component called ~bklog~ such that it processes a stdout stream of any +external command, not just ~gpspipe -r~. This would permit reuse of +the ~bkgpslog~ code for logging not just GPS data but things like +pressure, temperature, system statistics, etc. +2020-07-05T16:35Z; bktei> +: bklog -r age1asdf -o log.tar # encrypt/compress stdin to log.tar +: bklog -x -f log.tar -i age.key -O /tmp # extract and decrypt + +Making ~bklog~ follow the [[https://en.wikipedia.org/wiki/Unix_philosophy][Unix philosophy]] means that it shouldn't care +what kind of text is fed to it. + +*** ~bklog~ Design vs. Unix Philosophy +**** Pubkey dir watching +The feature of periodically checking a directory for changes in the +pubkeys it contains should be justified by its usefulness; if the +complexity cannot be justified then the feature should be removed. +**** Defaults vs options +Many options can cause the tool to become complex in unjustifiable +ways. Currently I am adding options because I want the ability to +modify the script's behavior without having to modify the source code +on the machine in which the code is running. I should consider +removing features at some point and having the program force defaults +on the user. For example, allowing the specification of a temporary +directory, while useful for me, is probably not useful for most people +who don't know or care about the difference between ~/tmp~ and +~/dev/shm~. +**** Script time to live (TTL) +I initially implemented a script time-to-live feature because I was +unsure in my ability to program script that could run for long periods +of time without causing a runaway usage of memory. I still think it's +a good idea to offer a script TTL option to the user but I think the +default should be to simply run forver. +** TODO: Evaluate ~rsyslog~ as stand-in for this work +2020-07-05T02:57Z; bktei> I searched for "debian iot logging" ("iot" +as in "Internet of Things", the current buzzword for small low-power +computers being used to provide microservices for owners in their own +home) and came across several search results mentioning ~syslog~ and +~rsyslog~. + +https://www.thissmarthouse.net/consolidating-iot-logs-into-mysql-using-rsyslog/ +https://rsyslog.readthedocs.io/en/latest/tutorials/tls.html +https://serverfault.com/questions/20840/how-would-you-send-syslog-securely-over-the-public-internet +https://www.rsyslog.com/ + +My impression is that ~rsyslog~ is a complex software package designed +to offer many features, some of which possibly might satisfy my +needs. + +However, as stated in the repository README, the objective of the +~ninfacyzga-01~ project is "Observing facts of the new". This means +that the goal is not only to record location data but any data that +can be captured by a sensor. This means the capture of the following +environmental phenomena are within the scope of this device: + +*** Sounds (microphone) +*** Light (camera) +*** Temperature (thermocouple) +*** Air Pressure (barometer) +*** Acceleration Vector (acceleromter / gyroscope) +*** Magnetic Field Vector (magnetometer) + +This brings up the issue of respecting privacy of others in shared +spaces through which ~ninfacyzga-01~ may pass through. ~ninfacyzga-01~ +should encrypt data it records according to rules set by its +owner. + +One permissive rule could be that if ~ninfacyzga-01~ detects that a +person (let's call her Alice) enters a room, it should add Alice's +encryption public key to the list of recipients against which it +encrypts data without Alice having to know how ~ninfacyzga-01~ is +programmed (she might have a ~calkuptcana~ agent on her person that +broadcasts her privacy preferences). Meanwhile, ~ninfacyzga-01~ may +publish its observations to a repository that Alice and other members +of the shared communal space have access to (ex: a read-only shared +directory on a local network WiFi). Alice could download all the files +in the shared repository but she would only be able to decrypt files +generated when she was physically near enough to ~ninfacyzga-01~ for +it to detect that her presence was within some spatial boundary. + +A more restrictive rule could resemble the permissive rule in that +~ninfacyzga-01~ uses Alice's encryption public key only when she is +physically near by, except that it encrypts logged files against +public keys in a sequential manner. This would mean that all people +who were near ~ninfacyzga-01~ would have to pass around each log file +to eachother so that they could decrypt the content. + +That said, according to [[https://www.rsyslog.com/doc/master/tutorials/database.html][this ~rsyslog~ page]], ~rsyslog~ is more a data +wrangling system for collecting data from disparate sources of +different types and outputting data to text files on disk than a +system committed to the server-client model of database storage. So, I +think converting ~bkgpslog~ into a ~bklog~ script that appends +encrypted and compressed data to a tar file for later extraction +(possibly the same script with future features) would be best. * bkgpslog narrative ** Initialize environment