debug(bklog):Fix SECONDS advancing
[EVA-2020-02.git] / doc / location / README.org
... / ...
CommitLineData
1* Location Logging
2This document was created by Steven Baltakatei Sandoval on
3~2020-06-29T12:14Z~ under a [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons BY-SA 4.0 license]]. It
4was updated by Steven Baltakatei Sandoval on ~2020-07-12T21:04Z~
5#+TITLE: Ninfacyzga-01 Manual
6#+AUTHOR: Steven Baltakatei Sandoval
7#+EMAIL: baltakatei@gmail.com
8** Narrative
9Ninfacyzga-01 records (logs) its position in time and space using a
10[[https://en.wikipedia.org/wiki/Satellite_navigation_device][GPS receiver]]. The NMEA location data produced by the receiver is
11converted into the more commonly used GPS data storage formats of GPX
12and KML. All three types of data are then compressed and encrypted
13against a set of public keys. The encrypted data is then written to
14disk. Data produced by the receiver is segmented into 60-second chunks
15before being processed and written to disk.
16** Description
17*** Hardware
18**** Raspberry Pi Zero W
19See the [[https://www.raspberrypi.org/pi-zero-w/][OEM]] webpage for this product.
20**** PiZ UpTime 2.0
21See the [[https://alchemy-power.com/piz-uptime-2-0/][OEM]] webpage for this product.
22*** Software
23~bklog~ : A bash script that saves its stdin stream to a tar file. The
24file may be compressed by ~gzip~ and encrypted by ~age~. It is an
25executable file contained within this repository at ~exec/bklog~. It
26should be copied to ~$HOME/.local/bin~.
27
28~bkgpslog~ : A legacy bash script similar to ~bklog~ but narrower in
29scope in that it only records output from ~gpspipe~.
30
31~gpsd~ : A background daemon app capable of interfacing with the
32Ozzmaker BerryGPS-IMU's GPS submodule. Installed and initialized by
33~apt~.
34
35~gpspipe~ : A command line app that polls ~gpsd~ and produces a stream
36stdout consisting of GPS data lines in NMEA format. Installed via
37~apt~.
38
39~gpsbabel~ : A command line app that converts GPS data from one format
40into another. ~bklog~ may be used to convert NMEA data into GPX and
41KML. Installed via ~apt~.
42
43~gzip~ : A simple command line app that compresses stdin into a
44smaller stdout stream.
45
46~age~ : A simple command line app that encrypts stdin against public
47keys specified in its options. Produces encrypted stdout. Is an
48executable file contained within this repository at ~exec/age~. It
49should be copied to ~$HOME/.local/bin~.
50
51**** Narrative
52~bklog~ may be used to log location data by receiving stdout produced
53by ~gpspipe~. ~bklog~ contains options that allow file writes to be
54performed at adjustable time intervals (default: 300 seconds),
55compressed (with ~gzip~), and encrypted (with ~age~). Files are
56written in the form of appendages to a ~tar~ archive saved to a
57specified location. The NMEA data produced by ~gpspipe~ may be
58processed via a ~gpsbabel~ command string specified as an option to
59~bklog~, assuming ~gpsbabel~ is installed.
60
61*** Output
62
63Several output file formats have been tested with ~bklog~.
64
65**** File Formats
66***** NMEA
67See the [[https://en.wikipedia.org/wiki/NMEA_0183][Wikipedia page]] for this. This file format is the default
68output of the ~gpspipe -r~ command.
69***** GPX
70See the [[https://en.wikipedia.org/wiki/GPS_Exchange_Format][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. An NMEA file
71may be converted to this format using ~gpsbabel~.
72***** KML
73See the [[https://en.wikipedia.org/wiki/Keyhole_Markup_Language][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. An NMEA file
74may be converted to this format using ~gpsbabel~.
75
76**** Encryption Method
77Files produced by the bkgpslog script are encrypted against a set of
78public keys using [[https://github.com/FiloSottile/age][~age~]], a simple command line encryption tool
79selected over ~gpg~ because of ~age~'s deliberate lack of
80configurability.
81
82The public keys are bech32 strings supplied as options to bkgpslog
83when called. The secret key should *NOT* be stored in Ninfacyzga-01.
84
85If a key pair was generated using ~age-keygen~, then it is an [[https://en.wikipedia.org/wiki/Curve25519][~X25519~]]
86key pair. See the [[https://age-encryption.org/v1][~age~ Version 1 specification]].
87
88An ~ssh-rsa~ or ~ssh-ed25519~ SSH public key string may be used instead of
89the bech32 public key string produced by ~age-keygen~ for convenience.
90
91Help information for ~age~ is available by running ~$ age --help~.
92***** Encryption Commands
93****** Encryption through ~age~
94In order to illustrate how ~bklog~ encrypts files, below is an example
95command illustrating how ~age~ may be used to encrypt a file.
96
97#+BEGIN_EXAMPLE
98$ echo "asdf" | age -r \
99age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \
100> "$HOME/secret_file"
101#+END_EXAMPLE
102
103The resulting ~secret-file~ is a binary blob with a plaintext header
104indicating how the blob was encrypted (which version of age was used,
105which public key was used).
106
107****** Encryption through ~bklog~
108~bklog~ may instructed to encrypt files via the ~-e~ and ~-r [pubkey
109string]~ options. An example is shown below:
110
111#+BEGIN_EXAMPLE
112$ gpspipe -r | bklog -e \
113-r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \
114-r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \
115-r age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 \
116-o "$HOME/Location"
117#+END_EXAMPLE
118
119~bklog~ may be instructed via the ~-e~ and ~-R~ options to watch a
120directory in order to locate public key strings in its files. ~bklog~
121reads the first line of each file and interprets it as a public key
122string.
123
124In this example, the strings beginning with ~age1...~ are
125bech32-formatted public key strings. Please see the [[*Key Generation][Key Generation]]
126section for an explanation.
127
128Since ~age~ also accepts ~ssh~ public key strings, these may also be
129used if they are of the following form (no comment).
130
131: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABA…AACAQDLnJbPs7CjwPT+OxXd
132
133***** Decryption Commands
134Files may be decrypted using a command similar to:
135
136#+BEGIN_EXAMPLE
137cat location.gpx.age | age -d -i key.txt > location.gpx
138#+END_EXAMPLE
139
140The version of ~age~ used to perform the encryption
141
142** Operating Procedures
143*** Initial Startup
144See OEM (Ozzmaker) [[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][quickstart guide for the BerryGPS-IMU]].
145
146**** Physical Setup
147
148BerryGPS-IMU must be electrically connected to the correct pins on the
149GPIO header of a Raspberry Pi Zero W.
150
151*Optional*: stack together with PiZ Uptime 2.0 module. No GPIO pins
152conflict so a simple stacking and soldering with long header pins is
153possible.
154
155**** Software Setup
156***** Install Executables
157
158Install Raspbian 10 Buster onto an SD card image. See the Raspberry Pi
159Foundation [[https://www.raspberrypi.org/documentation/installation/installing-images/README.md][installation instructions]]. Configure WiFi to permit log
160file transfer. Configure SSH to permit remote administration via the
161command line interface.
162
163Make sure to install the ~unattended-upgrades~ package to make sure
164the latest security patches for packages are installed. See [[https://linux-audit.com/using-unattended-upgrades-on-debian-and-ubuntu/][this page]]
165for a description of how ~unattended-upgrades~ works.
166
167Install ~gpsd~, ~gpspipe~, ~git~, and this repository for location
168logging capability.
169
170Install ~syncthing~ for log file transfer capability.
171
172Place ~age~ binary (the one compiled for ARM CPU architecture for
173Linux) in ~$HOME/.local/bin~.
174
175***** Disable Swap File
176Since standard Raspbian 10 (Buster) install involves copying
177unencrypted file system image to SD card which is mounted by the
178Raspberry Pi, system memory may be written to disk in the form of a
179Swap file as described [[https://ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/][here]]. In order to reduce the chance that
180location log data is ever written to disk, swap file functionality
181must be disabled[fn:ideaheap_20130731_disableswap].
182
183Raspbian 10 uses dphys-swapfile to manage a swap file. It may be
184disabled persistently[fn:rpf_20190702_disableswappersist] by running
185the following command:
186
187: sudo systemctl disable dphys-swapfile.service
188
189To view the status of the swap file in Raspbian 10, run ~free -m~:
190
191#+BEGIN_EXAMPLE
192pi@ninfacyzga-01:~$ free -m
193 total used free shared buff/cache available
194Mem: 432 86 36 21 309 268
195Swap: 99 0 99
196#+END_EXAMPLE
197
198After disabling the swap file and rebooting:
199
200#+BEGIN_EXAMPLE
201pi@ninfacyzga-01:~$ free -m
202 total used free shared buff/cache available
203Mem: 432 89 214 3 128 289
204Swap: 0 0 0
205#+END_EXAMPLE
206
207[fn:ideaheap_20130731_disableswap] Explanation:
208https://ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/
209
210[fn:rpf_20190702_disableswappersist] Persistant disabling of swap in
211Raspbian 10 Buster:
212https://www.raspberrypi.org/forums/viewtopic.php?p=1490692&sid=5c596a124b7805d6b10dab8d3d7caf16#p1490692
213
214***** Automatic Start Configuration
215
216It is recommended to create a daily ~cron~ job that executes a bash
217script that starts ~bklog~ jobs to record location along with other
218types of data recording. An example of such a script is below:
219
220#+BEGIN_EXAMPLE
221#!/bin/bash
222
223# Log location
224gpspipe -r | /bin/bash "$HOME/.local/bin/bklog" -c -e -z "UTC" -t "/dev/shm" \
225 -r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \
226 -r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \
227 -R "$HOME/.config/bklog/recipients" -w ".nmea" -b "300" -B "day" \
228 -o "$HOME/Sync/Evanescent_Location" -l "location" \
229 -p "gpsbabel -i nmea -f - -o gpx -F - " ".gpx" \
230 -p "gpsbabel -i nmea -f - -o kml -F - " ".kml" &
231# Log pressure
232python ~/.local/bin/bmp388.py | /bin/bash "$HOME/git-OC/ninfacyzga-01/exec/bklog" \
233 -c -e -z "UTC" -t "/dev/shm" \
234 -r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \
235 -r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \
236 -R "$HOME/.config/bklog/recipients" -w ".txt" -b "300" -B "day" \
237 -o "$HOME/Sync/Evanescent_Location" -l "pressure" &
238#+END_EXAMPLE
239
240This script, if it were saved at
241~"$HOME/.local/bin/cron/dailylog.sh"~ would then be added as a
242line in the ~crontab~ file as shown below:
243
244#+BEGIN_EXAMPLE
245$ crontab -e
2460 0 * * * /bin/bash ~/.local/bin/cron/dailylog.sh
247@reboot /bin/bash ~/.local/bin/cron/dailylog.sh
248#+END_EXAMPLE
249
250In the example script, the options are:
251
252: -c : tells bklog to compress output
253: -e : tells bklog log to encrypt output
254: -r : tells bklog to interpret the next argument as a pubkey string
255: -R : tells bklog to interpret the next argument as a directory
256: where public keys may be found (first line of each file is
257: read).
258: -o : tells bklog to write output files to the directory represented
259: -t : tells bklog to interpret the next argument as a directory
260: for storing temporary files
261: by the next argument
262: -p : tells bklog a command string through which output is piped
263: before being compressed and encrypted. Also expected is a
264: file extension to be appended before the compression and
265: encryption file extensions.
266: -w : tells bklog to save the unprocessed stdin with a specified
267: file extension (instead of the default '.stdin').
268: -b : tells bklog how long each buffer round (time between file
269: writes) lasts in seconds.
270: -B : specifies the time-to-live for the bklog script. A valid value may
271: one of the time elements such as "day" or "hour".
272: -l : specfies a custom string to be used in output file names to
273: help differentiate tar files produced via bklog from different
274: sources of data.
275: -z : specifies a time zone to be used to determine the script time-to-live.
276: By default, bklog uses whatever time is specified by the TZ
277: environment variable.
278
279***** Log Transfer Configuration
280Log files may be shared to other machines via ~syncthing~. See [[https://docs.syncthing.net/][this]]
281manual for how to set up a shared folder and add Ninfacyzga-01 as a
282device. Syncthing's directory synchronization capability allows a
283remote machine to delete files from Ninfacyzga-01 by deleting from the
284shared folder that they both share.
285
286When log files are removed from Ninfacyzga-01 is not within the scope
287of this document.
288
289***** Key Generation
290An ~age~ encryption key may be generated like so:
291#+BEGIN_EXAMPLE
292$ umask # Gets current umask
2930022 # Note: This is the default umask for Raspbian 10
294$ umask 066 # So key.txt will have no perms except for owner (you)
295$ umask # Confirm umask set to 066
2960066
297$ age-keygen > key.txt
298Public key: age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5
299$ ls -al key.txt
300-rw------- 1 baltakatei baltakatei 184 Jun 29 18:28 key.txt
301$ umask 0022 # Return umask to default value
302$ umask
3030022
304#+END_EXAMPLE
305
306The resulting public/private keypair data looks like:
307#+BEGIN_EXAMPLE
308$ cat key.txt
309# created: 2020-06-29T18:01:56Z
310# public key: age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5
311AGE-SECRET-KEY-1NEUU5U2XGZGL9UYWNPU5DL99TGJJHFSN4F2E2WCCSDJJ6L5ZMLESNTVTU0
312#+END_EXAMPLE
313
314The file ~key.txt~ is not password-protected by default and should be
315secured like an SSH public key should. The ~$ umask 066~ command run
316before the ~$ age-keygen > key.txt~ command ensures ~key.txt~ will not
317be readable, writeable, or executable to anyone except the owner
318(you).
319
320*** Normal Startup
321Turn on Ninfacyzga-01 by supplying 5VDC power to the Raspberry Pi. No
322further interaction should be required.
323*** Normal Operation
324No interaction beyond continually supplying approximately 100mA of
3255VDC power and occasionally removing log files to conserve disk space
326is required.
327**** Log Transfer
328Log files may be transferred by use of ~syncthing~ shared folders.
329**** Automatic Updates
330The ~automatic-upgrades~ package, if installed, should automatically
331install security patches to packages installed via ~apt~.
332*** Normal Shutdown
333The system may be shutdown via SSH by running:
334
335: $ sudo shutdown -r 0
336
337*** Unscheduled Shutdown
338Ninfacyzga-01 as described and setup should tolerate unscheduled power
339loss. Log files being written every 60 seconds means, at most, 60
340seconds worth of location data may be lost.
341*** End of Life Disposal
342LiPo batteries used by the PiZ Uptime 2.0 module should be disposed of
343properly with their potential ignitability in mind, especially if they
344are not fully discharged.
345
346Consult your local municipality for its "E-Waste Disposal" (or
347equivalent) policy. Metals used in the Raspberry Pi and related
348components may be recycled.
349
350Take extra precuation if lead solder was used in assembling the
351electronics. Consumer electronics in early 21st century should use
352lead-free solder.