From: Steven Baltakatei Sandoval Date: Fri, 16 Oct 2020 22:41:44 +0000 (+0000) Subject: Merge branch 'feature/bash-logging-filter/BK-2020-02' of https://zdv2.bktei.com/gitwe... X-Git-Url: https://zdv2.bktei.com/gitweb/EVA-2020-02.git/commitdiff_plain/a8cd68d3a839a40c0142e61ff3013a70d5e461f7?hp=8151f8ce6e2cfd2629e7a26b1ce94d6b860be50c Merge branch 'feature/bash-logging-filter/BK-2020-02' of https://zdv2.bktei.com/gitweb/ninfacyzga-01 into feature/bash-logging-filter/BK-2020-02 --- diff --git a/README.md b/README.md deleted file mode 100644 index b7724e9..0000000 --- a/README.md +++ /dev/null @@ -1,103 +0,0 @@ - - - -# Environment Sensor ninfacyzga-01 - -Created by [Steven Baltakatei Sandoval][bktei_2020_homepage] on -2020-06-07T21:56Z under a [CC BY-SA 4.0][cc_20131125_bysa] license and -last updated on 2020-06-28T16:41Z. - - -**Table of Contents** - -- [Environment Sensor ninfacyzga-01](#environment-sensor-ninfacyzga-01) - - [Abstract](#abstract) - - [Setup](#setup) - - [Location](#location) - - [Name Etymology](#name-etymology) - - [Origin](#origin) - - [Meaning](#meaning) - - [Pronunciation](#pronunciation) - - [Directory structure](#directory-structure) - - [Subsection](#subsection) - - [References](#references) - - - - -## Abstract - -This git repository contains software and documentation necessary to -set up a sensor package for logging new fact observe. - -Sensor data collected include: - -- location (GPS WGS84) - -## Setup - -See the following documents for instructions on setting up hardware -for each type of data collected. - -### Location - -See files within [`doc/location`](doc/location). - -## Name Etymology - -### Origin - -The name "ninfacyzga" is a -lojban[[1]](#lojwiki_20200212_main) -lujvo[[2]](#lojwiki_20140930_lujvo) consisting of the tanru -"cnino fatci zgana". The three gismu in the tanru contain the -following meanings: - -* **cnino**: new -* **fatci**: fact -* **zgana**: observe - -### Meaning - -Hence, the primary mission of a ninfacyzga device is to observe facts -of the new. The primary mission is NOT to process the facts into -stories since that is the mission of a temlisru'e device. Neither is -the primary mission to network with other devices over distances of -space for the purpose of sharing information. - -In practice, this means a ninfacyzga device records data about the -environment but does not attempt to generate summaries or averages of -the data. It simply transmits what it sees (ex: camera), hears (ex: -microphone), feels (ex: accelerometer), smells (ex: gas detector), -mags (ex: magnetometer). - -### Pronunciation - -"ninfacyzga" is pronounced /ninfaʃəzga/. - -## Directory structure - -Directories in this repository are structured as follows: - -* `doc`: Contains documentation explaining in human-readable format - what each file does. - -* `archive`: Contains legacy files from previous iterations of the - project. - -* `exec`: Contains executable components of the project. - -### Subsection - -## References -- 1. ["Lojban"][1]. 2020-02-12. [Logical Language Group"](https://mw.lojban.org/papri/Logical_Language_Group). Date Accessed: 2020-06-07. [Archive link](https://web.archive.org/web/20200304102316/https://mw.lojban.org/papri/Logical_Language_Group). Archive date: 2020-03-04. -- 2. ["lujvo"][2]. 2014-09-30. [Logical Language Group"](https://mw.lojban.org/papri/Logical_Language_Group). Date Accessed: 2020-06-07. [Archive link](https://web.archive.org/web/20160401100124/https://mw.lojban.org/papri/lujvo). Archive date: 2016-04-01. - -[1]: https://mw.lojban.org/papri/Lojban -[2]: https://mw.lojban.org/papri/lujvo -[bktei_2020_homepage]: http://baltakatei.com -[cc_20131125_bysa]: http://creativecommons.org/licenses/by-sa/4.0/ - - -
-

This work by Steven Baltakatei Sandoval is licensed under CC BY-SA 4.0

diff --git a/README.org b/README.org new file mode 100644 index 0000000..7dbe360 --- /dev/null +++ b/README.org @@ -0,0 +1,93 @@ +* Environment Sensor ninfacyzga-01 +** Introduction +This document created by Steven Baltakatei Sandoval on +2020-06-07T21:56Z under a [[http://creativecommons.org/licenses/by-sa/4.0/][CC BY-SA 4.0]] license and last updated on +2020-07-23T22:26Z. +** Abstract +This git repository contains software and documentation necessary to +set up a sensor package for logging observables in the local +environment. + +Collected sensor data include: + +- Location (GPS WGS84) +- Time (via ~gpsd~) +- Air Pressure (pascals) +- Alignment (degrees X, degrees Y, degrees from magnetic north) + +** Setup + +See the following documents for instructions on setting up hardware +for each type of data collected. + +*** Location + +See the location [[file:doc/location/README.org][readme]]. + +*** Pressure + +See the pressure [[file:doc/pressure/README.org][readme]]. + +*** Alignment + +See the alignment [[file:doc/alignment/README.org][readme]]. + +** Name Etymology + +*** Origin + +The name ~ninfacyzga~ is a [[https://mw.lojban.org/papri/Logical_Language_Group][lojban]] [[https://mw.lojban.org/papri/lujvo][lujvo]] consisting of the tanru "cnino +fatci zgana". The three gismu in the tanru contain the following +meanings: + +**** ~cnino~: new +**** ~fatci~: fact +**** ~zgana~: observe + +*** Meaning + +Hence, the primary mission of a ~ninfacyzga~ device is to observe facts +of the new. The primary mission is NOT to process facts into +stories. Neither is the primary mission to network with other devices +for the purpose of sharing information. This means a ~ninfacyzga~ +device records data about the environment but does not attempt to +generate summaries or reports of archived data as a historian +would. Instead, the device simply transmits what it sees (ex: camera), +hears (ex: microphone), feels (ex: accelerometer, gyroscope, or +magnetometer), smells (ex: gas detector), etc. + +The observations of a ~nincafyzga~ device are the input of another +class of devices called ~temlisru'e~ whose primary mission is to +process observations into stories over time. + +The transmission and verification of observations and analyses +produced by ~ninfacyzga~ or ~temlisru'e~ devices is managed by another +class of devices called ~calkuptcana~. + +*** Pronunciation + +~ninfacyzga~ is pronounced ~/ninfaʃəzga/~. + +~temlisru'e~ is pronounced ~/tɛmlisruhe/~. + +~calkuptcana~ is pronounced ~/ʃalkuptʃana/~. + +** Directory structure + +Directories in this repository are structured as follows: + +- ~doc~: Contains documentation explaining in human-readable format + what each file does. + +- ~archive~: Contains legacy files from previous iterations of the + project. + +- ~exec~: Contains executable components of the project. + +** References + +1. "Lojban". 2020-02-12. [[https://mw.lojban.org/papri/Logical_Language_Group]["Logical Language Group"]]. Date Accessed: + 2020-06-07. [[https://web.archive.org/web/20200304102316/https://mw.lojban.org/papri/Logical_Language_Group][Archive link]]. Archive date: 2020-03-04. + +2. "lujvo". 2014-09-30. [[https://mw.lojban.org/papri/Logical_Language_Group]["Logical Language Group"]]. Date Accessed: + 2020-06-07. [[https://web.archive.org/web/20160401100124/https://mw.lojban.org/papri/lujvo][Archive link]]. Archive date: 2016-04-01. diff --git a/TODO.org b/TODO.org new file mode 100644 index 0000000..c85e44f --- /dev/null +++ b/TODO.org @@ -0,0 +1,13 @@ +* Task List +** TODO ~doc/setup~: Disable bluetooth +** TODO ~/doc/time~: Migrate from ~ntpsec~ to ~chrony~ +The ~ntpsec~ package is in the ~dfsg~ Debian repository instead of the +~main~ Debian repository. This is likely due to ~ntpsec~ using a mix +of ~ntp~'s original "NTP License", ~BSD 2~, and ~MIT~ licenses. + +In contrast, ~chrony~ uses the ~GPLv2~ license which allows it to be +in the Debian main repository. + +~gpsd~ documentation for communicating to ~chrony~ via +~/run/chrony.XXXX.sock~ and ~/run/chrony.YYYY.sock~ sockets (for GPS +and PPS data respectively) can be found [[https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html#_feeding_chrony_from_gpsd][here]]. diff --git a/doc/location/README.org b/doc/location/README.org index 1ffdccd..3676e32 100644 --- a/doc/location/README.org +++ b/doc/location/README.org @@ -1,10 +1,12 @@ * Location Logging -This document was created by Steven Baltakatei Sandoval on -~2020-06-29T12:14Z~ under a [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons BY-SA 4.0 license]]. It -was updated by Steven Baltakatei Sandoval on ~2020-07-12T21:04Z~ #+TITLE: Ninfacyzga-01 Manual #+AUTHOR: Steven Baltakatei Sandoval #+EMAIL: baltakatei@gmail.com +** About +This document was created by Steven Baltakatei Sandoval on +~2020-06-29T12:14Z~ under a [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons BY-SA 4.0 license]]. It +was updated by Steven Baltakatei Sandoval on ~2020-10-16T22:33Z~ + ** Narrative Ninfacyzga-01 records (logs) its position in time and space using a [[https://en.wikipedia.org/wiki/Satellite_navigation_device][GPS receiver]]. The NMEA location data produced by the receiver is @@ -25,12 +27,9 @@ file may be compressed by ~gzip~ and encrypted by ~age~. It is an executable file contained within this repository at ~exec/bklog~. It should be copied to ~$HOME/.local/bin~. -~bkgpslog~ : A legacy bash script similar to ~bklog~ but narrower in -scope in that it only records output from ~gpspipe~. - -~gpsd~ : A background daemon app capable of interfacing with the -Ozzmaker BerryGPS-IMU's GPS submodule. Installed and initialized by -~apt~. +[[https://tracker.debian.org/pkg/gpsd][~gpsd~]] : A background daemon app capable of interfacing with the +[[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][Ozzmaker BerryGPS-IMU]]'s GPS submodule. Installed and initialized by +~apt~. Should be installed along with the ~gpsd-clients~ package. ~gpspipe~ : A command line app that polls ~gpsd~ and produces a stream stdout consisting of GPS data lines in NMEA format. Installed via @@ -64,8 +63,16 @@ Several output file formats have been tested with ~bklog~. **** File Formats ***** NMEA -See the [[https://en.wikipedia.org/wiki/NMEA_0183][Wikipedia page]] for this. This file format is the default -output of the ~gpspipe -r~ command. +~NMEA~ is an acronym for National marine Electronics Association. The +NMEA format described in this document follows the NMEA 0183 +standard. It is a newline-delimited streaming text format that encodes +global positioning system (GPS) data such as WGS84 location, time and +date information, satellite count, accuracy, and other +information. Each line is an "NMEA sentence". Descriptions of various +NMEA sentences can be found on [[http://aprs.gids.nl/nmea/][this]] webpage. + +See the [[https://en.wikipedia.org/wiki/NMEA_0183][Wikipedia page for NMEA 0183]] for this. This file format is the +default output of the ~gpspipe -r~ command. ***** GPX See the [[https://en.wikipedia.org/wiki/GPS_Exchange_Format][Wikipedia page]] for this. [[http://wiki.gis.com/wiki/index.php/WGS84][WGS84]] is the datum used. An NMEA file may be converted to this format using ~gpsbabel~. @@ -74,75 +81,11 @@ See the [[https://en.wikipedia.org/wiki/Keyhole_Markup_Language][Wikipedia page] may be converted to this format using ~gpsbabel~. **** Encryption Method -Files produced by the bkgpslog script are encrypted against a set of -public keys using [[https://github.com/FiloSottile/age][~age~]], a simple command line encryption tool -selected over ~gpg~ because of ~age~'s deliberate lack of -configurability. - -The public keys are bech32 strings supplied as options to bkgpslog -when called. The secret key should *NOT* be stored in Ninfacyzga-01. - -If a key pair was generated using ~age-keygen~, then it is an [[https://en.wikipedia.org/wiki/Curve25519][~X25519~]] -key pair. See the [[https://age-encryption.org/v1][~age~ Version 1 specification]]. - -An ~ssh-rsa~ or ~ssh-ed25519~ SSH public key string may be used instead of -the bech32 public key string produced by ~age-keygen~ for convenience. - -Help information for ~age~ is available by running ~$ age --help~. -***** Encryption Commands -****** Encryption through ~age~ -In order to illustrate how ~bklog~ encrypts files, below is an example -command illustrating how ~age~ may be used to encrypt a file. - -#+BEGIN_EXAMPLE -$ echo "asdf" | age -r \ -age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ -> "$HOME/secret_file" -#+END_EXAMPLE - -The resulting ~secret-file~ is a binary blob with a plaintext header -indicating how the blob was encrypted (which version of age was used, -which public key was used). - -****** Encryption through ~bklog~ -~bklog~ may instructed to encrypt files via the ~-e~ and ~-r [pubkey -string]~ options. An example is shown below: - -#+BEGIN_EXAMPLE -$ gpspipe -r | bklog -e \ --r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ --r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \ --r age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 \ --o "$HOME/Location" -#+END_EXAMPLE - -~bklog~ may be instructed via the ~-e~ and ~-R~ options to watch a -directory in order to locate public key strings in its files. ~bklog~ -reads the first line of each file and interprets it as a public key -string. - -In this example, the strings beginning with ~age1...~ are -bech32-formatted public key strings. Please see the [[*Key Generation][Key Generation]] -section for an explanation. - -Since ~age~ also accepts ~ssh~ public key strings, these may also be -used if they are of the following form (no comment). - -: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABA…AACAQDLnJbPs7CjwPT+OxXd - -***** Decryption Commands -Files may be decrypted using a command similar to: - -#+BEGIN_EXAMPLE -cat location.gpx.age | age -d -i key.txt > location.gpx -#+END_EXAMPLE - -The version of ~age~ used to perform the encryption +See [[file:../setup/README.org][Main Setup]] procedures. ** Operating Procedures *** Initial Startup See OEM (Ozzmaker) [[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][quickstart guide for the BerryGPS-IMU]]. - **** Physical Setup BerryGPS-IMU must be electrically connected to the correct pins on the @@ -154,65 +97,60 @@ possible. **** Software Setup ***** Install Executables +Follow the [[file:../setup/README.org][Main Setup]] procedures to obtain required files from this +repository. + +Install ~gpsd~, ~gpsd-clients~, and ~gpsbabel~. -Install Raspbian 10 Buster onto an SD card image. See the Raspberry Pi -Foundation [[https://www.raspberrypi.org/documentation/installation/installing-images/README.md][installation instructions]]. Configure WiFi to permit log -file transfer. Configure SSH to permit remote administration via the -command line interface. +: $ sudo apt install gpsd gpsd-clients gpsbabel -Make sure to install the ~unattended-upgrades~ package to make sure -the latest security patches for packages are installed. See [[https://linux-audit.com/using-unattended-upgrades-on-debian-and-ubuntu/][this page]] -for a description of how ~unattended-upgrades~ works. +***** Setup Serial for BerryGPS +The Ozzmaker BerryGPS-IMU unit requires that the serial console be +disabled and the serial port enabled. (see [[https://ozzmaker.com/berrygps-setup-guide-raspberry-pi/][ref]]). -Install ~gpsd~, ~gpspipe~, ~git~, and this repository for location -logging capability. +: $ sudo raspi-config -Install ~syncthing~ for log file transfer capability. +Navigate to ~5 Interfacing Options~, then ~P6 Serial~. -Place ~age~ binary (the one compiled for ARM CPU architecture for -Linux) in ~$HOME/.local/bin~. +When prompted "Would you like a login shell to be accessible over +serial?", answer ~No~. -***** Disable Swap File -Since standard Raspbian 10 (Buster) install involves copying -unencrypted file system image to SD card which is mounted by the -Raspberry Pi, system memory may be written to disk in the form of a -Swap file as described [[https://ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/][here]]. In order to reduce the chance that -location log data is ever written to disk, swap file functionality -must be disabled[fn:ideaheap_20130731_disableswap]. +When prompted "Would you like the serial port hardware to be +enabled?", answer ~Yes~. -Raspbian 10 uses dphys-swapfile to manage a swap file. It may be -disabled persistently[fn:rpf_20190702_disableswappersist] by running -the following command: +***** Setup ~gpsd~ +~gpsd~ needs to know which serial port to look at for NMEA location +data generated by the GPS unit. This can be done by modifying the +~gpsd~ configuration file at ~/etc/default/gpsd~. -: sudo systemctl disable dphys-swapfile.service +: sudo nano /etc/default/gpsd -To view the status of the swap file in Raspbian 10, run ~free -m~: +Make sure the following lines are present: #+BEGIN_EXAMPLE -pi@ninfacyzga-01:~$ free -m - total used free shared buff/cache available -Mem: 432 86 36 21 309 268 -Swap: 99 0 99 +START_DAEMON="true" +USBAUTO="false" +DEVICES="/dev/ttyAMA0" +GPSD_OPTIONS="-n" #+END_EXAMPLE -After disabling the swap file and rebooting: +NOTE: The ~DEVICES=~ line must be adjusted if the the ~ninfacyzga~ +device is configured to also track time. -#+BEGIN_EXAMPLE -pi@ninfacyzga-01:~$ free -m - total used free shared buff/cache available -Mem: 432 89 214 3 128 289 -Swap: 0 0 0 -#+END_EXAMPLE +NOTE: The ~-n~ option causes ~gpsd~ to begin polling GPS devices +without waiting for a client to connect. This is important if ~gpsd~ +provides data to other applications (ex: ~gpspipe~, ~ntpsec~, +~chrony~). -[fn:ideaheap_20130731_disableswap] Explanation: -https://ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/ +Restart ~gpsd~ via: -[fn:rpf_20190702_disableswappersist] Persistant disabling of swap in -Raspbian 10 Buster: -https://www.raspberrypi.org/forums/viewtopic.php?p=1490692&sid=5c596a124b7805d6b10dab8d3d7caf16#p1490692 +: $ sudo systemctl restart gpsd.service -***** Automatic Start Configuration +Run ~gpsmon~ to verify that location data is being recorded. +: $ gpsmon + +***** Automatic Start Configuration It is recommended to create a daily ~cron~ job that executes a bash script that starts ~bklog~ jobs to record location along with other types of data recording. An example of such a script is below: @@ -277,6 +215,8 @@ In the example script, the options are: : environment variable. ***** Log Transfer Configuration +See [[file:../setup/README.org][Main Setup]] procedures. + Log files may be shared to other machines via ~syncthing~. See [[https://docs.syncthing.net/][this]] manual for how to set up a shared folder and add Ninfacyzga-01 as a device. Syncthing's directory synchronization capability allows a @@ -287,35 +227,7 @@ When log files are removed from Ninfacyzga-01 is not within the scope of this document. ***** Key Generation -An ~age~ encryption key may be generated like so: -#+BEGIN_EXAMPLE -$ umask # Gets current umask -0022 # Note: This is the default umask for Raspbian 10 -$ umask 066 # So key.txt will have no perms except for owner (you) -$ umask # Confirm umask set to 066 -0066 -$ age-keygen > key.txt -Public key: age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 -$ ls -al key.txt --rw------- 1 baltakatei baltakatei 184 Jun 29 18:28 key.txt -$ umask 0022 # Return umask to default value -$ umask -0022 -#+END_EXAMPLE - -The resulting public/private keypair data looks like: -#+BEGIN_EXAMPLE -$ cat key.txt -# created: 2020-06-29T18:01:56Z -# public key: age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 -AGE-SECRET-KEY-1NEUU5U2XGZGL9UYWNPU5DL99TGJJHFSN4F2E2WCCSDJJ6L5ZMLESNTVTU0 -#+END_EXAMPLE - -The file ~key.txt~ is not password-protected by default and should be -secured like an SSH public key should. The ~$ umask 066~ command run -before the ~$ age-keygen > key.txt~ command ensures ~key.txt~ will not -be readable, writeable, or executable to anyone except the owner -(you). +See [[file:../setup/README.org][Main Setup]] procedures. *** Normal Startup Turn on Ninfacyzga-01 by supplying 5VDC power to the Raspberry Pi. No @@ -339,14 +251,4 @@ Ninfacyzga-01 as described and setup should tolerate unscheduled power loss. Log files being written every 60 seconds means, at most, 60 seconds worth of location data may be lost. *** End of Life Disposal -LiPo batteries used by the PiZ Uptime 2.0 module should be disposed of -properly with their potential ignitability in mind, especially if they -are not fully discharged. - -Consult your local municipality for its "E-Waste Disposal" (or -equivalent) policy. Metals used in the Raspberry Pi and related -components may be recycled. - -Take extra precuation if lead solder was used in assembling the -electronics. Consumer electronics in early 21st century should use -lead-free solder. +See [[file:../setup/README.org][Main Setup]] procedures. diff --git a/doc/setup/README.org b/doc/setup/README.org new file mode 100644 index 0000000..1a2f9fa --- /dev/null +++ b/doc/setup/README.org @@ -0,0 +1,422 @@ +#+TITLE:Ninfacyzga-1 Main Setup +#+AUTHOR:Steven Baltakatei Sandoval +#+EMAIL:baltakatei@gmail.com +* Main Setup +** About +This document created by [[http://baltakatei.com][Steven Baltakatei Sandoval]] on +~2020-10-07T18:39Z~ under a [[http://creativecommons.org/licenses/by-sa/4.0/][CC BY-SA 4.0]] license and last updated on +~2020-10-16T22:26Z~. + +This document contains information regarding setup of the +ninfacyzga-01 hardware common to all operation modes. This includes: + +- Raspberry OS installation +- WiFi configuration +- Remote SSH login configuration + +** Scope +This document describes hardware and software installation steps +common to the various environmental sensing functions of +ninfacyzga-01. + +** Narrative +The Raspberry Pi Zero W is the platform in which environment data is +gathered, packaged, and stored for further forwarding to a remote +repository. The Raspberry OS 10 operating system is used. The device +may be equipped with a UPS module in order to allow it to function as +a mobile device for short periods of time. The system may use +executables such as ~bklog~ to append segments of observed compressed +(~gzip~) encrypted (~age~) data to a ~tar~ archive to local disk. This +document describes hardware and software configuration procedures +generally required by all environment sensing operations. + +** Description +*** Hardware +**** Raspberry Pi Zero W +See the [[https://www.raspberrypi.org/pi-zero-w/][OEM]] webpage for this product. +**** PiZ UpTime 2.0 +See the [[https://alchemy-power.com/piz-uptime-2-0/][OEM]] webpage for this product. + +*** Software +~bklog~ : A bash script that saves its stdin stream to a tar file. The +file may be compressed by ~gzip~ and encrypted by ~age~. It is an +executable file contained within this repository at ~exec/bklog~. It +should be copied to ~$HOME/.local/bin~. + +~bkgpslog~ : A legacy bash script similar to ~bklog~ but narrower in +scope in that it only records output from ~gpspipe~. + +~gzip~ : A simple command line app that compresses stdin into a +smaller stdout stream. + +~age~ : A simple command line app that encrypts stdin against public +keys specified in its options. Produces encrypted stdout. Is an +executable file contained within this repository at ~exec/age~. It +should be copied to ~$HOME/.local/bin~. + +*** Output +**** Encryption Method +Files produced by the bklog script are encrypted against a set of +public keys using [[https://github.com/FiloSottile/age][~age~]], a simple command line encryption tool +selected over ~gpg~ because of ~age~'s deliberate lack of +configurability. + +The public keys are bech32 strings supplied as options to bkgpslog +when called. The secret key should *NOT* be stored in Ninfacyzga-01. + +If a key pair was generated using ~age-keygen~, then it is an [[https://en.wikipedia.org/wiki/Curve25519][~X25519~]] +key pair. See the [[https://age-encryption.org/v1][~age~ Version 1 specification]]. + +An ~ssh-rsa~ or ~ssh-ed25519~ SSH public key string may be used instead of +the bech32 public key string produced by ~age-keygen~ for convenience. + +Help information for ~age~ is available by running ~$ age --help~. +***** Encryption Commands +****** Encryption through ~age~ +In order to illustrate how ~bklog~ encrypts files, below is an example +command illustrating how ~age~ may be used to encrypt a file. + +#+BEGIN_EXAMPLE +$ echo "asdf" | age -r \ +age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ +> "$HOME/secret_file" +#+END_EXAMPLE + +The resulting ~secret-file~ is a binary blob with a plaintext header +indicating how the blob was encrypted (which version of age was used, +which public key was used). + +****** Encryption through ~bklog~ +~bklog~ may instructed to encrypt files via the ~-e~ and ~-r [pubkey +string]~ options. An example is shown below: + +#+BEGIN_EXAMPLE +$ gpspipe -r | bklog -e \ +-r age1kza7pfshy7xwygf9349zgmk7x53mquvedgw9r98qwyyqhssh830qqjzlsw \ +-r age1ce3pvzrqfcn2pc6zqzglc8ac8yjk3fzukpy08cesqjjwns53xywqmaq7xw \ +-r age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 \ +-o "$HOME/Location" +#+END_EXAMPLE + +~bklog~ may be instructed via the ~-e~ and ~-R~ options to watch a +directory in order to locate public key strings in its files. ~bklog~ +reads the first line of each file and interprets it as a public key +string. + +In this example, the strings beginning with ~age1...~ are +bech32-formatted public key strings. Please see the [[*Key Generation][Key Generation]] +section for an explanation. + +Since ~age~ also accepts ~ssh~ public key strings, these may also be +used if they are of the following form (no comment). + +: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABA…AACAQDLnJbPs7CjwPT+OxXd + +***** Decryption Commands +Files may be decrypted using a command similar to: + +#+BEGIN_EXAMPLE +cat location.gpx.age | age -d -i key.txt > location.gpx +#+END_EXAMPLE + +The version of ~age~ used to perform the encryption + +** Operating Procedures +*** Initial Startup +**** Physical Setup +The device should be supplied with 5V power and an SD card with the +latest Raspberry Pi OS image installed. As of 2020-10-07, this will be +version 10 (e.g. Raspbian Buster 10). + +No additional hardware (ex: GPS module, UPS module, thermocouples) is +required to perform actions described in this document + +**** Software Setup +***** Install Operating System +Install Raspberry Pi OS onto an SD card image. See the Raspberry Pi +Foundation [[https://www.raspberrypi.org/documentation/installation/installing-images/README.md][installation instructions]]. + +Note: "Raspberry Pi OS" is the name used by the Raspberry Pi +Foundation to refer to their operating system images to be installed +on Raspberry Pi hardware. The change was made in order to facilitate +education of beginners not familiar with the wordplay between +"Raspberry" and "Debian". See [[https://www.raspberrypi.org/forums/viewtopic.php?f=66&t=275380&sid=1a468f226394ccddf4654a3d3d90cb7d#p1668466][this]] forum post made on 2020-05-28 by +plugwash. + +***** Configure Wireless +Configure WiFi in order to permit file transfer and remote +administration. For a Raspberry Pi W, the WiFi settings may be +programmed via a specific text file in the `boot` partition of a +freshly installed image of Raspberry OS. Raspberry Pi Foundation +instructions [[https://www.raspberrypi.org/documentation/configuration/wireless/headless.md][here]]. + +In summary, create a ~wpa_supplicant.conf~ file containing the +following text: +#+BEGIN_EXAMPLE +ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev +update_config=1 +country=US + +network={ + ssid="" + psk="" +} +#+END_EXAMPLE + +Replace ~~ with your WiFi network's SSID. + +Replace ~~ with your WiFi network's +passphrase. +***** Enable Remote SSH Login +Configure SSH to permit remote administration via the command line +interface. Raspberry Pi Foundation instructions [[https://www.raspberrypi.org/documentation/remote-access/ssh/README.md][here]]. + +In summary, remote SSH access may be enabled upon initial startup of a +freshly installed image of Raspberry Pi OS by making sure an empty +file named ~ssh~ is present on the ~boot~ partition. + +***** Add SSH public key +If the use has an SSH public key, it may be added as a line in +~~/.ssh/authorized_keys~. + +Follow [[https://superuser.com/a/925859/][these]] directions to set permissions. + +: $ chmod 700 ~/.ssh +: $ chmod 644 ~/.ssh/authorized_keys + +***** Change default passphrase +The default username is ~pi~ and the default passphrase is +~raspberry~. Change them to something unique. + +: $ passwd + +***** Update software +Update software with distribution repository. + +: $ sudo apt update +: $ sudo apt upgrade -y +: $ sudo apt dist-upgrade -y + +***** Change time zone +The time zone should be set to "UTC" for simplicity. + +: $ sudo raspi-config + +Navigate to ~4 Localisation Options~, ~I2 Change Time Zone~, ~None of the above~, ~UTC~. + +***** Update hostname +A unique hostname is required to uniquely identify the device on the +network. + +Start up the Raspberry Pi Software Configuration Tool by running: +: $ sudo raspi-config + +- Select `2 Network Options` +- Select `N1 Hostname` + +This document recommends a hostname beginning with the prefix: +: ninfacyzga-1- + +An example hostname would be ~ninfacyzga-1-2~. + +***** Install software +****** ~unattended-upgrades~ +Make sure to install the ~unattended-upgrades~ package to make sure +the latest security patches for packages are installed. See [[https://linux-audit.com/using-unattended-upgrades-on-debian-and-ubuntu/][this page]] +for a description of how ~unattended-upgrades~ works. + +The configuration file is located at: +~/etc/apt/apt.conf.d/50unattended-upgrades~ ([[https://linux-audit.com/using-unattended-upgrades-on-debian-and-ubuntu/][ref]]). Make sure that the +following lines are present and not commented out. + +#+BEGIN_EXAMPLE +Unattended-Upgrade::Automatic-Reboot "true"; +#+END_EXAMPLE + +****** ~syncthing~ +Install ~syncthing~ for log file transfer capability. + +: $ sudo apt install syncthing + +Enable automatic startup. (See [[https://docs.syncthing.net/users/autostart.html][ref]]). + +: $ sudo systemctl enable syncthing@pi.service +: $ sudo systemctl start syncthing@pi.service + +The WebUI of the local instance of syncthing (port 8384) can be +accessed by running the following command from a separate machine: + +: $ ssh -L 127.0.0.1:8390:127.0.0.1:8384 pi@ninfacyzga-1-x + +Then, the separate machine should navigate to ~localhost:8390~ in a +web browser in order to change the ninfacyzga-1 device's +configuration. The separate machine's Syncthing configuration options +are accessible via its own web browser via ~localhost:8384~. + +****** ~git~ +~git~ facilitates downloading files from this repository to the +device. It may be installed via: + +: $ sudo apt install git + +****** ninfacyzga-01 git repository +Create the directory ~/git-OC/~ . Within this directory, run the +following commands to clone the ~ninfacyzga-01~ git repository: +: $ git clone https://zdv2.bktei.com/gitweb/ninfacyzga-01.git +: $ cd ninfacyzga-01 + +Check out the ~develop~ branch (if the latest changes are desired over +those of the ~master~ branch). +: $ git checkout --track origin/develop + +****** ~age~ +~age~ is required for encrypting data at rest. + +Place ~age~ binary (the one compiled for ARM CPU architecture for +Linux) in ~$HOME/.local/bin~. A copy of binary may be found within the +~exec~ directory. + +: $ mkdir ~/.local/bin +: $ cp exec/age ~/.local/bin/ + +***** Disable Swap File +Since standard Raspberry OS 10 install involves copying unencrypted +file system image to SD card which is mounted by the Raspberry Pi, +system memory may be written to disk in the form of a Swap file as +described [[https://ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/][here]]. In order to reduce the chance that location log data +is ever written to disk, swap file functionality must be +disabled[fn:ideaheap_20130731_disableswap]. + +Raspbian 10 uses dphys-swapfile to manage a swap file. It may be +disabled persistently[fn:rpf_20190702_disableswappersist] by running +the following command: + +: sudo systemctl disable dphys-swapfile.service + +To view the status of the swap file in Raspbian 10, run ~free -m~: + +#+BEGIN_EXAMPLE +pi@ninfacyzga-01:~$ free -m + total used free shared buff/cache available +Mem: 432 86 36 21 309 268 +Swap: 99 0 99 +#+END_EXAMPLE + +After disabling the swap file and rebooting: + +#+BEGIN_EXAMPLE +pi@ninfacyzga-01:~$ free -m + total used free shared buff/cache available +Mem: 432 89 214 3 128 289 +Swap: 0 0 0 +#+END_EXAMPLE + +[fn:ideaheap_20130731_disableswap] Explanation: +https://ideaheap.com/2013/07/stopping-sd-card-corruption-on-a-raspberry-pi/ + +[fn:rpf_20190702_disableswappersist] Persistant disabling of swap in +Raspbian 10 Buster: +https://www.raspberrypi.org/forums/viewtopic.php?p=1490692&sid=5c596a124b7805d6b10dab8d3d7caf16#p1490692 + +***** Disable Bluetooth +In order to reduce power consumed by bluetooth transmissions, +bluetooth functionality should be disabled (see [[https://di-marco.net/blog/it/2020-04-18-tips-disabling_bluetooth_on_raspberry_pi/][link]]). + +Modify the ~/boot/config.txt~ file (the Pi's equivalent to BIOS +settings; see [[https://www.raspberrypi.org/documentation/configuration/config-txt/][link]]) to make sure the following lines are added: + +#+BEGIN_EXAMPLE +# Disable Bluetooth +dtoverlay=disable-bt +#+END_EXAMPLE + +The ~hciuart~ service is associated with bluetooth functionality via +UART which may conflict with location and time data provided via +~/dev/ttyAMA0~. It should be disabled like so: + +#+BEGIN_EXAMPLE +$ sudo systemctl disable hciuart +#+END_EXAMPLE + +***** Disable login console via serial port +Some ~ninfacyzga~ functions (location and time) require data transfer +via ~/dev/ttyAMA0~. In order to prevent serial login programs from +interfering with such functions, it is necessary to disable them. + +Run the following commands to disable login via ~ttyAMA0~: + +#+BEGIN_EXAMPLE +$ sudo systemctl stop serial-getty@ttyAMA0.service +$ sudo systemctl disable serial-getty@ttyAMA0.service +$ sudo systemctl disable hciuart +#+END_EXAMPLE + +Modify ~/boot/cmdline.txt~ to remove the console: + +#+BEGIN_EXAMPLE +$ sudo nano /boot/cmdline.txt +#+END_EXAMPLE + +Remove ~console=serial0,115200~ + +***** Log Transfer Configuration +Log files may be shared to other machines via ~syncthing~. See [[https://docs.syncthing.net/][this]] +manual for how to set up a shared folder and add Ninfacyzga-01 as a +device. Syncthing's directory synchronization capability allows a +remote machine to delete files from Ninfacyzga-01 by deleting from the +shared folder that they both share. + +When log files are removed from Ninfacyzga-01 is not within the scope +of this document. +***** Key Generation +An ~age~ encryption key may be generated like so: +#+BEGIN_EXAMPLE +$ umask # Gets current umask +0022 # Note: This is the default umask for Raspbian 10 +$ umask 066 # So key.txt will have no perms except for owner (you) +$ umask # Confirm umask set to 066 +0066 +$ age-keygen > key.txt +Public key: age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 +$ ls -al key.txt +-rw------- 1 baltakatei baltakatei 184 Jun 29 18:28 key.txt +$ umask 0022 # Return umask to default value +$ umask +0022 +#+END_EXAMPLE + +The resulting public/private keypair data looks like: +#+BEGIN_EXAMPLE +$ cat key.txt +# created: 2020-06-29T18:01:56Z +# public key: age1pu5usxm743sx7rf22985xv2f4s0luzv6r6yx4fa7p8c2zyvp9fvqus2xr5 +AGE-SECRET-KEY-1NEUU5U2XGZGL9UYWNPU5DL99TGJJHFSN4F2E2WCCSDJJ6L5ZMLESNTVTU0 +#+END_EXAMPLE + +The file ~key.txt~ is not password-protected by default and should be +secured like an SSH public key should. The ~$ umask 066~ command run +before the ~$ age-keygen > key.txt~ command ensures ~key.txt~ will not +be readable, writeable, or executable to anyone except the owner +(you). + +*** Normal Startup +*** Normal Operation +*** Normal Shutdown +*** Unscheduled Shutdown +*** End of Life Disposal +See [[file:../setup/README.org][Main Setup]] procedures. + +LiPo batteries used by the PiZ Uptime 2.0 module should be disposed of +properly with their potential ignitability in mind, especially if they +are not fully discharged. + +Consult your local municipality for its "E-Waste Disposal" (or +equivalent) policy. Metals used in the Raspberry Pi and related +components may be recycled. + +Take extra precuation if lead solder was used in assembling the +electronics. Consumer electronics in early 21st century should use +lead-free solder. + + + diff --git a/doc/time/README.org b/doc/time/README.org new file mode 100644 index 0000000..e662695 --- /dev/null +++ b/doc/time/README.org @@ -0,0 +1,480 @@ +#+TITLE: Ninfacyzga-1 Time Tracking +#+AUTHOR: Steven Baltakatei Sandoval +#+EMAIL: baltakatei@gmail.com +* Time Tracking +** About +This document was created by Steven Baltakatei Sandoval on +~2020-07-23T22:27Z~ under a [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons BY-SA 4.0 license]]. It +was updated by Steven Baltakatei Sandoval on ~2020-10-16T22:26Z~ + +** Narrative +The ~ninfacyzga-01~ device is equipped with an Ozzmaker BerryGPS-IMU +module which provides time and location data to ~gpsd~ and ~ntpsec~. The +time is provided by GPS satellites which themselves are +equipped [fn:nasa_20020408_atomicclock] with atomic clocks. This +extremely accurate set of clocks are needed since a GPS receiver +calculates its position in space using a General Relativity +calculation that uses the small variations in the time stamps received +from each satellite. This means that ~gpsd~ may be used to set the +system clock without a need for an internet connection to a default +Debian time server; ~ninfacyzga-01~ can be its own time server. + +[fn:nasa_20020408_atomicclock] Title:[[https://science.nasa.gov/science-news/science-at-nasa/2002/08apr_atomicclock/][Tick-Tock Atomic Clock]]; +Date:2002-04-08; Website:NASA.gov; [[https://web.archive.org/web/20100429141752/http://science.nasa.gov/science-news/science-at-nasa/2002/08apr_atomicclock/][Archive-link]]; Archive-date: +2010-04-29 + +** Description +*** Hardware +Ozzmaker BerryGPS-IMU, Version 3 (see [[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][ref]]). +*** Software +- [[https://tracker.debian.org/pkg/gpsd][~gpsd~]] : A background daemon app capable of interfacing with the + [[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][Ozzmaker BerryGPS-IMU]]'s GPS submodule. Installed and initialized by + ~apt~. Should be installed along with the ~gpsd-clients~ + package. This procedure was developed with ~gpsd~ version + +- [[https://tracker.debian.org/pkg/ntpsec][~ntpsec~]] : A security-hardned version of [[https://tracker.debian.org/pkg/ntp][~ntp~]] which is a "[[https://en.wikipedia.org/wiki/Network_Time_Protocol][Network + Time Protocol]]" app that synchronizes the system clock with local or + remote time servers or devices. It is capable of extracting time + data from a GPS signal handled by ~gpsd~. It is also capable of + using the high precision PPS (pulse-per-second) signal handled by + ~gpsd~. Installed via ~apt~. This procedure was developed with + ~ntpsec~ version ~1.1.3~. + +** Operating Procedures +*** Initial Startup +**** Perform initial setup. +See [[file:../setup/README.org][Main Setup]] procedure. +**** Install Hardware for time tracking +See [[https://ozzmaker.com/forums/topic/connecting-gps-pps-pin/][this]] Ozzmaker forum topic about connecting the BerryGPS-IMU +~T_PULSE~ pin to GPIO 18. + +#+CAPTION: An image showing how to connect the PPS signal from an Ozzmaker BerryGPS-IMU board to a Raspberry Pi Zero W. +#+NAME: fig:PPS_BERRYGPS_RASPIZW +[[../../img/Compact_Stratum_1_NTP_time_server_hardware,_October_2020.jpg]] + +Connect the ~T_PULSE~ connection on the BerryGPS-IMU-3 to GPIO pin 18 +(ex: with solder and wire) in order to provide the PPS data signal +generated by the BerryGPS-IMU to the Raspberry Pi. Processing of this +data signal is handled by adding a line to ~/boot/config.txt~ in the +next section ("Install Software"). + +Note: If it is desired to specify a custom GPIO pin besides the one +recommended, see this [[https://raspberryautomation.com/connect-multiple-ds18b20-temperature-sensors-to-a-raspberry-pi/][Raspberry Autom]] article. + +**** Install Software for time tracking +The time tracking function can be performed by two programs: ~gpsd~ +and ~ntpsec~. + +Basically, two things need to happen: + +1. ~gpsd~ needs to be pointed towards the correct device files for + incoming GPS data (in NMEA format) and the PPS signal ("pulse per + second"; a high precision time signal). + +2. ~ntpsec~ needs to be pointed towards the correct local IP addresses + where ~gpsd~ provides GPS data and the PPS signal. + +~gpsd~ then will provide GPS and PPS data to ~ntpsec~ via a "shared +memory driver". + +***** Install packages via ~apt~ +Run the following command to install the required packages. +: $ sudo apt install gpsd gpsd-clients python-gps pps-tools ntpsec + +***** Enable PPS device +Modify the ~/boot/config.txt~ file in order to tell the Raspberry Pi +to expect PPS data on ~BCM 18~ (pin number 12; see [[https://pinout.xyz/][link]]). This is done +by adding the following line to ~/boot/config.txt~ as described on +[[https://ozzmaker.com/forums/topic/problems-with-pps-on-a-pi0w-running-raspian-and-attached-to-a-berrygps-imuv3/][this Ozzmaker page]]: + +: dtoverlay=pps-gpio,gpiopin=18 + +The ~/boot/config.txt~ file can be modified via: + +: $ sudo nano /boot/config.txt + +PPS data can be confirmed by running: + +#+BEGIN_EXAMPLE +$ sudo su - +# ppstest /dev/pps0 +trying PPS source "/dev/pps0" +found PPS source "/dev/pps0" +ok, found 1 source(s), now start fetching data... +source 0 - assert 1595708074.003644641, sequence: 219 - clear 0.000000000, sequence: 0 +source 0 - assert 1595708075.003709620, sequence: 220 - clear 0.000000000, sequence: 0 +source 0 - assert 1595708076.003779580, sequence: 221 - clear 0.000000000, sequence: 0 +source 0 - assert 1595708077.003850580, sequence: 222 - clear 0.000000000, sequence: 0 +#+END_EXAMPLE + +Note: For older Raspberry Pi models, it may be necessary to enable +~pps-gpio~ via modifications to ~/etc/modules~ (see [[https://www.raspberrypi.org/forums/viewtopic.php?p=757747#p757747][link]]). + +***** Enable GPS device +The Ozzmaker BerryGPS-IMU makes NMEA sentences available via the +serial "UART" device ~/dev/ttyAMA0~. If bluetooth has not been +disabled, the Raspberry Pi OS automatically creates a software "UART" +device at ~/dev/serial0~. See the "[[file:~/git-OC/ninfacyzga-01/doc/setup/README.org::*Disable%20Bluetooth][Disable Bluetooth]]" section in the +[[file:../setup/README.org][Main Setup]] Initial Startup procedure for instructions on how to +disable bluetooth to free up ~/dev/ttyAMA0~ for use by ~gpsd~. + +Note: Older ~ntp~ documentation indicates that ~ntp~ (and presumably +~ntpsec~ might be able to access GPS data (ex: NMEA sentences) without +the aid of ~gpsd~ itself if the data is available at a device named +~/dev/ggp0~. Similarly, the same might apply to PPS data and the +~/dev/gpspps0~ device. However, since the Raspberry Pi OS overlay +system automatically creates ~/dev/pps0~ using the modifications to +~/boot/config.txt~ (described [[*Enable PPS device][elsewhere]] in this procedure). + +***** Setup ~gpsd~ +See the "[[file:~/git-OC/ninfacyzga-01/doc/location/README.org::*Setup%20~gpsd~][Setup ~gpsd~]]" subsection within the "Initial Startup" section +of the Location Logging [[file:~/git-OC/ninfacyzga-01/doc/location/README.org][~README.org~]] file. There is one additional +change that must be made which is to add a ~/dev/pps0~ item to the +~DEVICES=~ line in ~/etc/default/gpsd~ like so: + +: DEVICES="/dev/ttyAMA0 /dev/pps0" + +~/dev/ttyAMA0~ is where ~gpsd~ can get NMEA data from the GPS unit. + +~/dev/pps0~ is where ~gpsd~ can get a PPS signal. + +As an example, the following lines will be present in +~/etc/default/gpsd~ if both location and time tracking are set up: + +#+BEGIN_EXAMPLE +START_DAEMON="true" +USBAUTO="false" +DEVICES="/dev/ttyAMA0 /dev/pps0" +GPSD_OPTIONS="-n" +#+END_EXAMPLE + +***** Setup ~ntpsec~ +The ~ntpsec~ configuration file at ~/etc/ntpsec/ntp.conf~ must be +modified to tell ~ntpsec~ how to retrieve and interpret the GPS and +PPS data provided by ~gpsd~. + +In summary, lines need to be added telling ~ntpsec~ how often to poll +certain sources (local or remote) for time information as well as how +to prioritize and label information from each source. + +~ntpsec~ uses syntax simplified from that of ~ntp~. Specifically, it +recommends use of the ~refclock~ keyword for configuring local +hardware clocks such as GPS and PPS devices. + +Below is an example of lines that may be added to +~/etc/ntpsec/ntp.conf~ in order to ensure that the system clock will always have + +#+BEGIN_EXAMPLE +# Local time sources + +# Kernel PPS (precision seconds; driver 22) +refclock pps unit 0 refid kPPS flag3 1 minpoll 4 maxpoll 4 stratum 1 + +# GPS PPS reference (precision seconds; driver 28; root) (NTP1) +refclock shm unit 1 refid PPS minpoll 4 maxpoll 4 iburst stratum 1 + +# GPS Serial data reference (coarse time; driver28; root) (NTP0) +refclock shm unit 0 refid GPS prefer time1 0.109 minpoll 4 maxpoll 4 iburst stratum 1 + +# GPS PPS reference (precision seconds; driver 28; user) (NTP2) +refclock shm unit 2 refid PPSuser minpoll 4 maxpoll 4 iburst stratum 1 + +# Tolerate outliers (useful for coarse clocks) +tos mindist 0.500 +#+END_EXAMPLE + +A complete working example of an ~ntp.conf~ file is found in Appendix +A of this document. + +Commands useful for examining the shared memory driver used by ~gpsd~ +and ~ntpsec~ are: + +- ~sudo ntpshmmon~ : Shows live output of data using the shared memory + driver. ([[https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html][ref]]) + +- ~sudo ipcs -m~ : Show live segments of the shared memory. ([[https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html][ref]]) + +General references for this configuration file are: + +- "Raspberry Pi as a Stratum-1 NTP Server - Updating to the final NTP + configuration file" by David Taylor. [[http://www.satsignal.eu/ntp/Raspberry-Pi-NTP.html#ntp-conf][Link]]. Date: 2020-03-24. Note: + An example of kernel PPS transfer from ~gpsd~ to ~ntp~ via + ~127.127.22.0~. + +- "GPSD Time Service HOWTO" by Garry E. Miller. [[https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html][Link]]. Date: + 2020-06-28. Note: Explains how shared memory segments available at + ~127.127.28.0~, ~127.127.28.1~, ~127.127.28.2~, etc. work in context + of an ~ntp~ configuration file. + +- "Building a Stratum 1 NTP Server with a Raspberry Pi 4 and Adafruit + Ultimate GPS Hat". [[http://www.gregledet.net/computers/building-a-stratum-1-ntp-server-with-a-raspberry-pi-4-and-adafruit-ultimate-gps-hat/][Link]]. Date: 2020-02-23. Note: A concise example + between ~gpsd~ and ~ntp~ setup for a Raspberry Pi 4 using + ~127.127.28.0~ and ~127.127.28.1~. Recommends use of ~ntpsec~ + instead of ~ntp~. + +Expalantions of some ~ntpsec~ [[https://docs.ntpsec.org/latest/comdex.html][commands and options]] used in the +configuration file include: + +- ~refclock~ : An ~ntpsec~ keyword not present in ~ntp-4.2.8~ that is + meant to simplify configuration of reference clocks that use certain + [[https://docs.ntpsec.org/latest/refclock.html][drivers]] including ~pps~ ([[https://docs.ntpsec.org/latest/driver_pps.html][PPS Clock Discipline]]), ~shm~ ([[https://docs.ntpsec.org/latest/driver_shm.html][Shared Memory + Driver]]) . In ~ntp-4.2.8~ and earlier, in order to configure a clock, + instead of ~ntpsec~'s one keyword (~refclock~), two keywords would + be required (~server~ and ~fudge~). + +- ~server [address]~ : Tells ~ntpsec~ the address to query for time + information. May be IP address or URL. Additional options such as + ~minpoll~, ~maxpoll~, ~prefer~, and ~true~ may be added. + +- ~prefer~ : Indicates the source should be prioritized in some + way. How exactly the ~ntpsec~ mitigation rules (see [[https://docs.ntpsec.org/latest/prefer.html][ref]]) use the + ~prefer~ option is complicated but in the context of dealing with a + GPS source and a PPS source, the GPS source should have a ~prefer~ + option. The PPS source may have a ~prefer~ option but it is not + necessary; adding ~prefer~ to a PPS source is useful in the corner + case that the GPS source is less reliable than another source (ex: a + remote clock specified by ~server~). + +- ~... shm ... flag1 1~ : For an ~shm~ driver, tells ~ntpsec~ to + ignore the default sanity check that discards any time that is more + than 4-hours off from the system clock. Note that ~ntpsec~ treats + ~shm flag1 1~ in the opposite manner as ~ntp~ (see [[https://docs.ntpsec.org/latest/driver_shm.html][ref]]). + +- ~... shm ... time1 [float]~ : Tells ~ntpsec~ that a latency + time of ~[float]~ seconds is required for data to travel from the + address to ~ntpsec~ (ex: a USB hub in the case of a GPS device that + connects via USB). (see [[https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html#_feeding_ntpd_from_gpsd][ref]]) This number of seconds should be + adjusted to reduce the `offset` time of the address indicated in the + output of the ~ntpq -pn~ command to near ~0.000~. Note that the + units of ~offset~ displayed by ~ntpq -pn~ are in milliseconds. + +- ~... refid [string]~ : Tells ~ntpsec~ a string used to label the source. + +- ~fudge~ : An ~ntp~ keyword that ~ntpsec~ developers do not recommend + using; ~ntpsec~ documentation for ~fudge~ is widthdrawn. For local + hardware providing GPS and PPS time information, ~fudge~ can mostly + be replaced with use of the ~refclock~ keyword instead. + +- ~fudge [address]~ : Tells ~ntpsec~ to consider additional options + when evaluating the accuracy of time from a given address. + +- ~fudge [address] ... flag3 1~ : An option that tells ~ntpsec~ to + enable "kernel PPS discipline" when interpreting the PPS signal at + the address. (see [[http://doc.ntp.org/4.2.8p7/drivers/driver22.html][ref]]). + +- ~stratum 1~ : Tells ~ntpsec~ to label the source as a "stratum 1" + time server. This means that it is a clock using NTP that is a + canonical (?) time source. For example, a stratum 0 NTP time server + would be an atomic clock. A stratum 1 NTP time server could be a + clock that uses a GPS receiver to synchronize to said atomic clock. + +- ~iburst~ : Tells ~ntpsec~ how often to query a source. + +- ~fudge [address] ... flag1 [float]~ : See description of ~... shm + ... time1 [float]~. + +- ~tos mindist [float]~ : Tells ~ntpsec~ to increase the tolerance for + outliers in the clock selection algorithm. Units of ~[float]~ are + seconds. The default value is ~0.001~ seconds. This [[http://doc.ntp.org/4.2.6/miscopt.html][reference]] + indicates that it is acceptable to raise this value for "clocks with + high jitter and a PPS signal". For the device being configured in + this guide, this is the case. ~ntpsec~ receives a kernel PPS signal + on ~127.127.22.0~ indicating when each second occurs but without + information about which second of the day is being ticked. ~ntpsec~ + receives a coarse time from ~127.127.28.0~ supplied by ~gpsd~ that + is derived from the NMEA sentences generated by the GPS receiver; + this coarse time (high jitter) lacks the precision of the PPS signal + since NMEA sentence transmission times are not well-controlled. + +****** Disable DHCP services affecting ~ntpsec~ +Some processes that use [[https://en.wikipedia.org/wiki/Dynamic_Host_Configuration_Protocol][DHCP]] (Dynamic Host Configuration Protocol) may +cause configuration changes to configuration files of ~ntp~. Since +~ntpsec~ is similar to ~ntp~, it is reasonable to take action to +prevent this. + +Remove some files: + +: $ sudo rm /etc/dhcp/dhclient-exit-hooks.d/ntp +: $ sudo rm /lib/dhcpcd/dhcpcd-hooks/50-ntp.conf + +In the "request" block of ~dhclient.conf~, remove ~dhcp6.sntp-servers~ +and ~ntp-servers~. + +: $ sudo nano /etc/dhcp/dhclient.conf + +For example, the following section of ~dhclient.conf~: + +#+BEGIN_EXAMPLE +send host-name = gethostname(); +request subnet-mask, broadcast-address, time-offset, routers, + domain-name, domain-name-servers, domain-search, host-name, + dhcp6.name-servers, dhcp6.domain-search, dhcp6.fqdn, dhcp6.sntp-servers, + netbios-name-servers, netbios-scope, interface-mtu, + rfc3442-classless-static-routes, ntp-servers; +#+END_EXAMPLE + +should be changed to: + +#+BEGIN_EXAMPLE +send host-name = gethostname(); +request subnet-mask, broadcast-address, time-offset, routers, + domain-name, domain-name-servers, domain-search, host-name, + dhcp6.name-servers, dhcp6.domain-search, dhcp6.fqdn, + netbios-name-servers, netbios-scope, interface-mtu, + rfc3442-classless-static-routes; +#+END_EXAMPLE + +***** Disable CPU power saving +Power saving featurs of the Raspberry Pi Zero W may also be disabled +in order to improve accuracy. + +****** Configure CPU ~scaling_governor~ +If additional precision is required, the PPS signal may be made more +reliable at the cost of increasing CPU power by configuring the CPU to +always run at maximum frequency.[fn:se_20180320_raspicpugov] This +change can be performed by modifying the following file as root: + +: /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor + +The file should consist of one line. Change + +: ondemand + +to + +: performance + +. + +This change can be performed via the ~nano~ text editor by running the +following commands: + +: $ sudo su - +: # nano /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor + +Additionally, in order to prevent the ~raspi-config~ init script from +reverting this text file back to ~ondemand~ after a reboot, this +script must be disabled via: + +: $ sudo systemctl disable raspi-config + +****** Configure ~/boot/config.txt~ +Modify ~/boot/config.txt~ so that it contains these lines in order to +disable power saving functions: + +#+BEGIN_EXAMPLE +# Disable power saving +nohz=off +#+END_EXAMPLE + +[fn:se_20180320_raspicpugov] Title:[[https://raspberrypi.stackexchange.com/questions/9034/how-to-change-the-default-governor]["How to change the default governor?"]]; Author:[[https://raspberrypi.stackexchange.com/users/5538/goldilocks][goldilocks]]; Date: 2018-03-20; Website:stackexchange.com; + +*** Normal Startup +*** Normal Operation +*** Normal Shutdown +*** Unscheduled Shutdown +** Appendix A +*** Example ~ntp.conf~ for ~ntpsec~ +Below is an example ~ntp.conf~ file for use with ~ntpsec~. + +#+BEGIN_EXAMPLE +# /etc/ntpsec/ntp.conf, configuration for ntpd; see ntp.conf(5) for help + +# Drift file +driftfile /var/lib/ntpsec/ntp.drift + +# Leap seconds definition provided by tzdata +leapfile /usr/share/zoneinfo/leap-seconds.list + +# Log file +#logfile /var/log/ntp.log + + +# You must create /var/log/ntpsec (owned by ntpsec:ntpsec) to enable logging. +#statsdir /var/log/ntpsec/ +#statistics loopstats peerstats clockstats +#filegen loopstats file loopstats type day enable +#filegen peerstats file peerstats type day enable +#filegen clockstats file clockstats type day enable + + +# Comment this out if you have a refclock and want it to be able to discipline +# the clock by itself (e.g. if the system is not connected to the network). +#tos minclock 4 minsane 3 + + +# Local time sources + +# Kernel PPS (precision seconds; driver 22) +refclock pps unit 0 refid kPPS flag3 1 minpoll 4 maxpoll 4 stratum 1 + +# GPS PPS reference (precision seconds; driver 28; root) (NTP1) +refclock shm unit 1 refid PPS minpoll 4 maxpoll 4 iburst stratum 1 + +# GPS Serial data reference (coarse time; driver28; root) (NTP0) +refclock shm unit 0 refid GPS prefer time1 0.109 minpoll 4 maxpoll 4 iburst stratum 1 + +# GPS PPS reference (precision seconds; driver 28; user) (NTP2) +refclock shm unit 2 refid PPSuser minpoll 4 maxpoll 4 iburst stratum 1 + +# Tolerate outliers (useful for coarse clocks) +tos mindist 0.500 + + +# Specify one or more NTP servers. + +# pool.ntp.org maps to about 1000 low-stratum NTP servers. Your server will +# pick a different set every time it starts up. Please consider joining the +# pool: +server time.nist.gov iburst +server tick.usno.navy.mil iburst +server 0.us.pool.ntp.org iburst +#server 0.us.pool.ntp.org iburst +#server 0.us.pool.ntp.org iburst +#server 0.us.pool.ntp.org iburst +pool 0.debian.pool.ntp.org iburst +#pool 1.debian.pool.ntp.org iburst +#pool 2.debian.pool.ntp.org iburst +#pool 3.debian.pool.ntp.org iburst + + +# Access control configuration; see /usr/share/doc/ntpsec-doc/html/accopt.html +# for details. +# +# Note that "restrict" applies to both servers and clients, so a configuration +# that might be intended to block requests from certain clients could also end +# up blocking replies from your own upstream servers. + +# By default, exchange time with everybody, but don't allow configuration. +restrict default kod nomodify nopeer noquery limited + +# Local users may interrogate the ntp server more closely. +restrict 127.0.0.1 +restrict ::1 +#+END_EXAMPLE + +* Discard + +** Transitional ntp to ntpsec config file snippet +# Kernel PPS (precision seconds; driver 22) +#server 127.127.22.0 minpoll 4 maxpoll 4 true +#fudge 127.127.22.0 flag3 1 refid kPPS stratum 1 +refclock pps unit 0 refid kPPS flag3 1 minpoll 4 maxpoll 4 stratum 1 + +# GPS PPS reference (precision seconds; driver 28; root) (NTP1) +#server 127.127.28.1 minpoll 4 maxpoll 4 iburst +#fudge 127.127.28.1 flag1 1 refid PPS +refclock shm unit 1 refid PPS minpoll 4 maxpoll 4 iburst stratum 1 + +# GPS Serial data reference (coarse time; driver28; root) (NTP0) +#server 127.127.28.0 minpoll 4 maxpoll 4 iburst prefer +#fudge 127.127.28.0 flag1 1 time1 0.109 refid GPS stratum 1 +refclock shm unit 0 refid GPS prefer time1 0.109 minpoll 4 maxpoll 4 iburst stratum 1 + +# GPS PPS reference (precision seconds; driver 28; user) (NTP2) +#server 127.127.28.2 minpoll 4 maxpoll 4 iburst +#fudge 127.127.28.2 flag1 1 refid PPSuser stratum 1 +refclock shm unit 2 refid PPSuser minpoll 4 maxpoll 4 iburst stratum 1 + +# Tolerate outliers (useful for coarse clocks) +tos mindist 0.500 diff --git a/doc/time/ref/Schmidt-NTP-2019.pdf b/doc/time/ref/Schmidt-NTP-2019.pdf new file mode 100644 index 0000000..076161f Binary files /dev/null and b/doc/time/ref/Schmidt-NTP-2019.pdf differ diff --git a/exec/age/age-v1.0.0-beta2-linux-arm/age b/exec/age/age-v1.0.0-beta2-linux-arm/age new file mode 100755 index 0000000..d529bb4 Binary files /dev/null and b/exec/age/age-v1.0.0-beta2-linux-arm/age differ diff --git a/exec/age/age-v1.0.0-beta2-linux-arm/age-keygen b/exec/age/age-v1.0.0-beta2-linux-arm/age-keygen new file mode 100755 index 0000000..3d339ac Binary files /dev/null and b/exec/age/age-v1.0.0-beta2-linux-arm/age-keygen differ diff --git a/exec/bklog b/exec/bklog index 504ecbc..afb73be 100644 --- a/exec/bklog +++ b/exec/bklog @@ -11,8 +11,8 @@ dirTmpDefault="/dev/shm"; # Default parent of working directory # Script Metadata scriptName="bklog"; # Define basename of script file. -scriptVersion="0.1.29"; # Define version of script. -scriptURL="https://gitlab.com/baltakatei/ninfacyzga-01"; # Define wesite hosting this script. +scriptVersion="0.1.32"; # Define version of script. +scriptURL="https://gitlab.com/baltakatei/ninfacyzga-01"; # Define website hosting this script. scriptTimeStartEpoch="$(date +%s)"; # Save start time of script in epoch seconds scriptTimeStart="$(date +%Y%m%dT%H%M%S.%N)"; # YYYYmmddTHHMMSS.NNNNNNNNN scriptHostname=$(hostname); # Save hostname of system running this script. @@ -58,7 +58,7 @@ processArguments() { -p | --process-string) optionProcString="true" && argProcStrings+=("$2") && argProcFileExts+=("$3") && vbm "STATUS:file extension \"$3\" for output of processing string added:\"$2\""; shift; shift;; -l | --label) optionLabel="true" && argLabel="$2"; vbm "DEBUG :Custom label received:$argLabel"; shift;; -w | --store-raw) optionStoreRaw="true" && argRawFileExt="$2"; vbm "DEBUG :Raw stdin file extension received:$argRawFileExt"; shift;; - -W | --no-store-raw) optionNoStoreRaw="true"; vbm "DEBUG :Option selected to not store raw stdin data."; shift;; + -W | --no-store-raw) optionNoStoreRaw="true"; vbm "DEBUG :Option selected to not store raw stdin data.";; *) yell "ERROR: Unrecognized argument: $1"; yell "STATUS:All arguments:$*"; exit 1;; # Handle unrecognized options. esac shift @@ -1178,6 +1178,21 @@ magicSetScriptTTL() { fi; vbm "STATUS:$fn:Finished magicSetScriptTTL() function."; } # Set scriptTTL in seconds until next (day|hour). +magicVerboseReadout() { + vbm "$bufferTTL"; # Time-to-live (seconds) for each buffer round + vbm "$scriptTTL_TE"; # Time element at the end of which script terminates + vbm "$dirTmpDefault"; # Default parent of working directory + # Script Metadata + vbm "$scriptName"; # Basename of script file. + vbm "$scriptVersion"; # Version of script. + vbm "$scriptURL"; # Website hosting this script. + vbm "$scriptTimeStartEpoch"; # Start time of script in epoch seconds + vbm "$scriptTimeStart"; # YYYYmmddTHHMMSS.NNNNNNNNN + vbm "$scriptHostname" # Hostname of system running this script. + vbm "$PATH"; # PATH env. var. + vbm "$ageVersion"; # Version of age (encryption program) + vbm "$ageURL"; # Website hosting age. +} # Display script variables magicWriteVersion() { # Desc: Appends time-stamped VERSION to pathout_tar # Usage: magicWriteVersion @@ -1403,6 +1418,8 @@ main() { magicParseProcessStrings; # Sets arrays: procStrings, procFileExts ## React to "-l" (output file label) option magicParseLabel; # sets label (ex: "_location") + ## React to "-v" (verbose) option + magicVerboseReadout; # Display various script variables # Perform secondary setup operations ## Set script lifespan (scriptTTL from scriptTTL_TE) @@ -1439,12 +1456,14 @@ main() { done; # Mark time for buffer ## Initial time - if [[ ! -z "$timeBufferEndEpoch" ]]; then + if [[ bufferRound -gt 0 ]]; then ### Usual case timeBufferStartEpoch="$timeBufferEndEpoch" && vbm "DEBUG :$fn:timeBufferStartEpoch:$timeBufferStartEpoch"; - else + elif [[ bufferRound -eq 0 ]]; then ### Edge case: initial startup timeBufferStartEpoch="$scriptTimeStartEpoch" && vbm "DEBUG :$fn:timeBufferStartEpoch:$timeBufferStartEpoch"; + else + yell "ERROR:$fn:Invalid bufferRound value."; exit 1; fi; ## End Time timeBufferEndEpoch="$(date +%s)" && vbm "DEBUG :$fn:timeBufferEndEpoch:$timeBufferEndEpoch"; diff --git a/exec/bklog-plan.org b/exec/bklog-plan.org index 18bdf42..3b7f5f5 100644 --- a/exec/bklog-plan.org +++ b/exec/bklog-plan.org @@ -87,7 +87,14 @@ closed. *** 2020-07-14T22:25Z; bktei> Possible solution using ~dd~, ~od~, and ~if [ -z string ]~ [[https://unix.stackexchange.com/a/33055][here]]. -** TODO Support configuration file +** TODO Support configuration file for storing options +*** 2020-07-15T23:48Z;bktei> +Support storing options in a ~$HOME/.config/bklog/options.conf~ file +so individual calls to ~bklog~ don't have to be so long. Aliases can +be used instead to make ~bklog~ calls shorter, but cron jobs don't +necessarily source alias files. Explicitly reading options from a +configuration file would be more reliable, although it complicates +usage of ~bklog~. *** 2020-07-20T01:45Z; bktei> It should be possible to specify options via a configuration file stored within a ~~/.config/~ directory instead of specifying options diff --git a/img/Compact_Stratum_1_NTP_time_server_hardware,_October_2020.jpg b/img/Compact_Stratum_1_NTP_time_server_hardware,_October_2020.jpg new file mode 100644 index 0000000..4f4bfda Binary files /dev/null and b/img/Compact_Stratum_1_NTP_time_server_hardware,_October_2020.jpg differ diff --git "a/ref/age \342\200\224 A simple file encryption tool & format.odt" "b/ref/age \342\200\224 A simple file encryption tool & format.odt" new file mode 100644 index 0000000..eecb95e Binary files /dev/null and "b/ref/age \342\200\224 A simple file encryption tool & format.odt" differ diff --git "a/ref/age \342\200\224 A simple file encryption tool & format.pdf" "b/ref/age \342\200\224 A simple file encryption tool & format.pdf" new file mode 100644 index 0000000..7acaed7 Binary files /dev/null and "b/ref/age \342\200\224 A simple file encryption tool & format.pdf" differ