fix(exec):Camera script and daily job script

[EVA-2020-02.git] / doc / time / README.org

diff --git a/doc/time/README.org b/doc/time/README.org
index 7f845587978e3541eac623ebe170305caec3118b..5351973dd06e28f25bd5a5cc8d724cbab4d313ec 100644 (file)
--- a/doc/time/README.org
+++ b/doc/time/README.org
@@ -1,65 +1,101 @@
-* Time 
+#+TITLE: Ninfacyzga-1 Time Tracking
+#+AUTHOR: Steven Baltakatei Sandoval
+#+EMAIL: baltakatei@gmail.com
+* Time Tracking
+** About

 This document was created by Steven Baltakatei Sandoval on
 This document was created by Steven Baltakatei Sandoval on

-~2020-07-23T22:27Z~ under a [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons BY-SA 4.0 license]]. It
-was updated by Steven Baltakatei Sandoval on ~2020-07-25T19:34Z~
-** Development Task list
-*** Set up prototype unit
-**** DONE Solder wire for PPS signal
-***** 2020-07-25T20:03Z; bktei> Soldered pins to permit accessibility.
-     CLOSED: [2020-07-25 Sat 20:03]
-**** Follow guide
-- [[https://ozzmaker.com/forums/topic/connecting-gps-pps-pin/][Ozzmaker post]] on ~PPS~ wiring and a [[https://www.satsignal.eu/ntp/Raspberry-Pi-NTP.html][guide]] recommendation.
-- [[http://www.satsignal.eu/raspberry-pi/Schmidt-RPZ-NTP-2016.pdf][Schmidt article]] on Raspberry Pi Zero time server.
-- [[https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html][GPSD dev ref]] page for ~gpsd~ interfacing with ~ntp~.
-- [[https://raspberryautomation.com/connect-multiple-ds18b20-temperature-sensors-to-a-raspberry-pi/][Raspberry Autom]] article on specifying custom GPIO pin.
-*** Document prototype unit
+~2020-07-23T22:27Z~ under a [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons BY-SA 4.0 license]]. It was
+updated by Steven Baltakatei Sandoval on ~2020-10-18T00:11Z~
+

 ** Narrative
 The ~ninfacyzga-01~ device is equipped with an Ozzmaker BerryGPS-IMU
 ** Narrative
 The ~ninfacyzga-01~ device is equipped with an Ozzmaker BerryGPS-IMU

-module which provides time and location data to ~gpsd~. The time is
-provided by GPS satellites which themselves are [[https://science.nasa.gov/science-news/science-at-nasa/2002/08apr_atomicclock/][equipped]] with atomic
-clocks. This extremely accurate set of clocks are needed since a GPS
-receiver calculates its position in space using a General Relativity
+module which provides time and location data to ~gpsd~ and ~chrony~. The
+time is provided by GPS satellites which themselves are
+equipped [fn:nasa_20020408_atomicclock] with atomic clocks. This
+extremely accurate set of clocks are needed since a GPS receiver
+calculates its position in space using a General Relativity

 calculation that uses the small variations in the time stamps received
 from each satellite. This means that ~gpsd~ may be used to set the
 system clock without a need for an internet connection to a default
 Debian time server; ~ninfacyzga-01~ can be its own time server.
 calculation that uses the small variations in the time stamps received
 from each satellite. This means that ~gpsd~ may be used to set the
 system clock without a need for an internet connection to a default
 Debian time server; ~ninfacyzga-01~ can be its own time server.

+
+[fn:nasa_20020408_atomicclock] Title:[[https://science.nasa.gov/science-news/science-at-nasa/2002/08apr_atomicclock/][Tick-Tock Atomic Clock]];
+Date:2002-04-08; Website:NASA.gov; [[https://web.archive.org/web/20100429141752/http://science.nasa.gov/science-news/science-at-nasa/2002/08apr_atomicclock/][Archive-link]]; Archive-date:
+2010-04-29
+

 ** Description
 *** Hardware
 ** Description
 *** Hardware

-Ozzmaker BerryGPS-IMU, Version 3
+Ozzmaker BerryGPS-IMU, Version 3 (see [[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][ref]]).

 *** Software
 *** Software

-This guide describes steps that may be used to convert a Raspberry Pi
-into a time server using ~gpsd~.
+- [[https://www.raspberrypi.org/downloads/raspberry-pi-os/][Raspberry Pi OS]] : A GNU/Linux operating system derived from
+  Debian 10. This procedure was developed with version ~August 2020~.
+
+- [[https://tracker.debian.org/pkg/gpsd][~gpsd~]] : A background daemon app capable of interfacing with the
+  [[https://ozzmaker.com/berrygps-berrygps-imu-quick-start-guide/][Ozzmaker BerryGPS-IMU]]'s GPS submodule. Installed and initialized by
+  ~apt~. Should be installed along with the ~gpsd-clients~
+  package. This procedure was developed with ~gpsd~ version
+
+- [[https://chrony.tuxfamily.org/][~chrony~]] : A set of programs capable of continuously adjusting the
+  system clock until it is synchronized with configurable time sources
+  such as GPS and PPS data provided by ~gpsd~. ~chrony~ may be
+  configured to act as an NTP time client or server. It uses the same
+  protocol as ~ntp~ but is a GPLv2 implementation. This procedure was
+  developed with ~chrony~ version ~3.4-4~.
+

 ** Operating Procedures
 *** Initial Startup
 ** Operating Procedures
 *** Initial Startup

-**** Install Raspbian 10 (Buster) onto a Raspberry Pi Zero W.
-**** Install Hardware
+**** Perform initial setup.
+See [[file:../setup/README.org][Main Setup]] procedure.
+**** Install Hardware for time tracking

 See [[https://ozzmaker.com/forums/topic/connecting-gps-pps-pin/][this]] Ozzmaker forum topic about connecting the BerryGPS-IMU
 ~T_PULSE~ pin to GPIO 18.
 
 See [[https://ozzmaker.com/forums/topic/connecting-gps-pps-pin/][this]] Ozzmaker forum topic about connecting the BerryGPS-IMU
 ~T_PULSE~ pin to GPIO 18.
 

+#+CAPTION: An image showing how to connect the PPS signal from an Ozzmaker BerryGPS-IMU board to a Raspberry Pi Zero W.
+#+NAME: fig:PPS_BERRYGPS_RASPIZW
+[[../../img/Compact_Stratum_1_NTP_time_server_hardware,_October_2020.jpg]]
+

 Connect the ~T_PULSE~ connection on the BerryGPS-IMU-3 to GPIO pin 18
 (ex: with solder and wire) in order to provide the PPS data signal
 generated by the BerryGPS-IMU to the Raspberry Pi. Processing of this
 data signal is handled by adding a line to ~/boot/config.txt~ in the
 next section ("Install Software").
 
 Connect the ~T_PULSE~ connection on the BerryGPS-IMU-3 to GPIO pin 18
 (ex: with solder and wire) in order to provide the PPS data signal
 generated by the BerryGPS-IMU to the Raspberry Pi. Processing of this
 data signal is handled by adding a line to ~/boot/config.txt~ in the
 next section ("Install Software").
 

-**** Install Software
-These instructions assume that ~gpsd~ has already been setup to
-provide NMEA sentences to ~gpspipe~ for location. See the ~README.org~
-in ~doc/location~ for details. Basically, ~gpsd~ needs to be told via
-its ~/etc/default/gpsd~ configuration file of which ~/dev/ttyS0~ will
-provide the raw GPS module data.
+Note: If it is desired to specify a custom GPIO pin besides the one
+recommended, see this [[https://raspberryautomation.com/connect-multiple-ds18b20-temperature-sensors-to-a-raspberry-pi/][Raspberry Autom]] article.
+
+**** Install Software for time tracking
+The time tracking function can be performed by two programs: ~gpsd~
+and ~chrony~.
+
+Basically, two things need to happen:
+
+1. ~gpsd~ needs to be pointed towards the correct device files for
+   incoming GPS data (in NMEA format) and the PPS signal ("pulse per
+   second"; a high precision time signal).
+
+2. ~chrony~ needs to be pointed towards the correct local IP addresses
+   where ~gpsd~ provides GPS data and the PPS signal.
+
+~gpsd~ then will provide GPS and PPS data to ~chrony~ via a "shared
+memory driver".

 
 ***** Install packages via ~apt~
 Run the following command to install the required packages.
 
 ***** Install packages via ~apt~
 Run the following command to install the required packages.

-: $ sudo apt install usbmount eject gpsd gpsd-clients python-gps pps-tools ntp
+: $ sudo apt install gpsd gpsd-clients python-gps pps-tools chrony

 
 

-***** Modify ~/boot/config.txt~
-In order to tell the Raspberry Pi to expect PPS data on ~BCM 18~ (pin
-number 12; see [[https://pinout.xyz/][RaspPi pinout]]), add the following line to
-~/boot/config.txt~ as described on [[https://ozzmaker.com/forums/topic/problems-with-pps-on-a-pi0w-running-raspian-and-attached-to-a-berrygps-imuv3/][this Ozzmaker page]]:
+***** Enable PPS device
+Modify the ~/boot/config.txt~ file in order to tell the Raspberry Pi
+to expect PPS data on ~BCM 18~ (pin number 12; see [[https://pinout.xyz/][link]]). This is done
+by adding the following line to ~/boot/config.txt~ as described on
+[[https://ozzmaker.com/forums/topic/problems-with-pps-on-a-pi0w-running-raspian-and-attached-to-a-berrygps-imuv3/][this Ozzmaker page]]:

 
 

+: # Enable PPS on GPIO 18

 : dtoverlay=pps-gpio,gpiopin=18
 
 : dtoverlay=pps-gpio,gpiopin=18
 

+The ~/boot/config.txt~ file can be modified via:
+
+: $ sudo nano /boot/config.txt
+

 PPS data can be confirmed by running:
 
 #+BEGIN_EXAMPLE
 PPS data can be confirmed by running:
 
 #+BEGIN_EXAMPLE


@@ -74,71 +110,222 @@ source 0 - assert 1595708076.003779580, sequence: 221 - clear  0.000000000, sequ
 source 0 - assert 1595708077.003850580, sequence: 222 - clear  0.000000000, sequence: 0
 #+END_EXAMPLE
 
 source 0 - assert 1595708077.003850580, sequence: 222 - clear  0.000000000, sequence: 0
 #+END_EXAMPLE
 

-***** Add udev rule
-As per [[https://www.slsmk.com/how-to-setup-a-gps-pps-ntp-time-server-on-raspberry-pi/][the slsmk.com]] guide, create symbolic links expected by ~ntp~
-pointing to the PPS device. This may be performed by creating the file:
+Note: For older Raspberry Pi models, it may be necessary to enable
+~pps-gpio~ via modifications to ~/etc/modules~ (see [[https://www.raspberrypi.org/forums/viewtopic.php?p=757747#p757747][link]]).
+
+***** Enable GPS device
+The Ozzmaker BerryGPS-IMU makes NMEA sentences available via the
+serial "UART" device ~/dev/ttyAMA0~. If bluetooth has not been
+disabled, the Raspberry Pi OS automatically creates a software "UART"
+device at ~/dev/serial0~. See the "[[file:../setup/README.org::*Disable%20Bluetooth][Disable Bluetooth]]" section in the
+[[file:../setup/README.org][Main Setup]] Initial Startup procedure for instructions on how to
+disable bluetooth to free up ~/dev/ttyAMA0~ for use by ~gpsd~.
+
+***** Setup ~gpsd~
+See the "[[file:../location/README.org::*Setup%20~gpsd~][Setup ~gpsd~]]" subsection within the "Initial Startup" section
+of the Location Logging [[file:../location/README.org][~README.org~]] file. There is one additional
+change that must be made which is to add a ~/dev/pps0~ item to the
+~DEVICES=~ line in ~/etc/default/gpsd~ like so:
+
+: DEVICES="/dev/ttyAMA0 /dev/pps0"
+
+~/dev/ttyAMA0~ is where ~gpsd~ can get NMEA data from the GPS unit.
+
+~/dev/pps0~ is where ~gpsd~ can get a PPS signal.
+
+As an example, the following lines will be present in
+~/etc/default/gpsd~ if both location and time tracking are set up:
+
+#+BEGIN_EXAMPLE
+START_DAEMON="true"
+USBAUTO="false"
+DEVICES="/dev/ttyAMA0 /dev/pps0"
+GPSD_OPTIONS="-n"
+#+END_EXAMPLE
+
+Make sure to enable ~gpsd~ to automatically start as a system service.

 
 

-: /etc/udev/rules.d/09-pps.rules
+: $ sudo systemctl enable gpsd
+: $ sudo systemctl start gpsd

 
 

-with the contents:
+***** Setup ~chrony~
+Modify the configuration file for ~chrony~ at ~/etc/chrony/chrony.conf~.
+
+: $ sudo nano /etc/chrony/chrony.conf
+
+Add the following lines:

 
 #+BEGIN_EXAMPLE
 
 #+BEGIN_EXAMPLE

-KERNEL=="ttyAMA0", SUBSYSTEM=="tty", DRIVER=="", OWNER=="root", GROUP=="tty", MODE=="0777", SYMLINK+="gps0"
+# Get time from GPS (/dev/XXXX) and PPS (/dev/YYYY)
+#refclock SOCK /run/chrony.XXXX.sock refid GPS precision 1e-1 offset 0.0000
+#refclock SOCK /run/chrony.YYYY.sock refid PP precision 1e-7
+refclock SHM 0 refid GPS precision 1e-1 offset 0.0000 delay 0.2 stratum 1
+refclock SHM 1 refid PPS precision 1e-7 stratum 1

 #+END_EXAMPLE
 
 #+END_EXAMPLE
 

-***** Modify ~/etc/modules~
-For older versions of Raspberry Pi, it may be necessary to explicitly
-require the ~pps-gpio~ module to be loaded by making sure
-~/etc/modules~ contains the line:
+Where
+- ~XXXX~ : the basename of the GPS device's serial port. In this guide
+  it should be ~ttyAMA0~; other setups may use ~ttyS0~, ~ttyACM0~, or
+  ~serial0~.
+
+- ~YYYY~ : the basename of the PPS device's serial port. In this guide
+  it should be ~pps0~.
+
+Note: The ~refclock SOCK~ lines are left as comments in case ~gpsd~
+incorrectly maps the GPS and PPS data.

 
 

-: pps-gpio
+The following commands may be useful for testing ~gpsd~ and ~chrony~
+configurations.
+- ~chronyc sources -v~ : Shows time sources and associated accuracy
+  information.

 
 

-According to [[https://www.raspberrypi.org/forums/viewtopic.php?p=757747#p757747][this]] post on 2015-05-13T09:14 by [[https://www.raspberrypi.org/forums/memberlist.php?mode=viewprofile&u=31454][ame]] on the Raspberry Pi
-forums,
+- ~chronyc tracking~ : Shows the current time difference between
+  the reference clock and the system clock. Note: ~chrony~ gradually
+  attempts to reduce the difference by changing the system clock.

 
 

-#+BEGIN_QUOTE
-rowlandski wrote:
-> Hi,
-> 
-> I want the RPi to load/probe the w1-gpio module on boot.
-> 
-> I have added:
-> 
-> sudo modprobe w1-gpio
-> 
-> To the etc/modules file, but on boot it is not loaded.
-> I have checked this by the lsmod command. If i then enter the above line manually and then check it's there loaded.
-> 
-> What's wrong with my line on code in the modules file?
-> 
-> Thanks
+- ~sudo chronyc makestep~ : Force ~chrony~ to set the system clock to
+  match the reference clock immediately.

 
 

-You no longer need to load modules for some devices. It's handled by devicetree.
+- ~sudo systemctl enable chrony~ : Enable automatic startup of
+  ~chrony~ (Note: This command shouldn't be necessary since the act of
+  installing ~chrony~ via ~sudo apt install chrony~ should
+  automatically enable it).

 
 

-Search for "devicetree" on the forum, or start here:
-https://www.raspberrypi.org/forums/viewtopic.php?f=28&t=97314
-#+END_QUOTE
+- ~sudo systemctl stop chrony~ : Stop ~chrony~.

 
 

-As of 2020-07-25, using Raspbian Buster 10 with a BerryGPS-IMU v3,
-~/etc/modules~ did not need to be modified.
+- ~sudo systemctl restart chrony~ : Restart ~chrony~.

 
 

-***** Remove DHCP client hooks
-As per [[https://www.slsmk.com/how-to-setup-a-gps-pps-ntp-time-server-on-raspberry-pi/][this]] guide, "remove ntp DHCP hook (so that it doesn't override ntp settings)".
+- ~systemctl status chrony~ : Check status of ~chrony~ service.

 
 

-This can be performed by running the following command (shown in
-script form for readability):
+- ~sudo ntpshmmon~ : Shows live output of data using the shared memory
+  driver filled by ~gpsd~. ([[https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html][ref]])
+
+- ~sudo ipcs -m~ : Show live segments of the shared memory. ([[https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html][ref]])
+
+- ~sudo date -s '2020-07-07T00:00+0000'~ : Manually sets time to a
+  string.
+
+An example output of ~sudo chronyc sources -v~ will show something
+similar to this:

 
 #+BEGIN_EXAMPLE
 
 #+BEGIN_EXAMPLE

-if [ -f  "/etc/dhcp/dhclient-exit-hooks.d/ntp" ]; then
-    mv /etc/dhcp/dhclient-exit-hooks.d/ntp /etc/dhcp/dhclient-exit-hooks.d/xxxntp-orig;
-else
-    echo "NTP exit hook does not exist";
-fi;
+pi@ninfacyzga-1-x:~ $ sudo chronyc sources -v
+210 Number of sources = 6
+
+  .-- Source mode  '^' = server, '=' = peer, '#' = local clock.
+ / .- Source state '*' = current synced, '+' = combined , '-' = not combined,
+| /   '?' = unreachable, 'x' = time may be in error, '~' = time too variable.
+||                                                 .- xxxx [ yyyy ] +/- zzzz
+||      Reachability register (octal) -.           |  xxxx = adjusted offset,
+||      Log2(Polling interval) --.      |          |  yyyy = measured offset,
+||                                \     |          |  zzzz = estimated error.
+||                                 |    |           \
+MS Name/IP address         Stratum Poll Reach LastRx Last sample               
+===============================================================================
+#- GPS                           1   4   377    21   +110ms[ +110ms] +/-  200ms
+#* PPS                           1   4   377    22  +2496ns[+3045ns] +/- 1000ns
+^- vps-2d3ddab6.vps.ovh.ca       2   6   277    57  +1302us[+1304us] +/-  151ms
+^? time.richiemcintosh.com       2   6     1    59  +2626us[+2628us] +/-   92ms
+^- varuna.ga-group.nl            3   6   377    55  -3962us[-3960us] +/-  151ms
+^- ntp3.junkemailfilter.com      2   6   377    58  -4561us[-4558us] +/-   80ms

 #+END_EXAMPLE
 
 #+END_EXAMPLE
 

+General references for the ~chrony.conf~ file are:
+
+- The ~chrony~ ~4.0~ documentation. ([[https://chrony.tuxfamily.org/doc/4.0/chrony.conf.html][ref]])
+
+- The ~gpsd~ documentation for communicating with ~chrony~. ([[https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html#_feeding_chrony_from_gpsd][ref]])
+
+- Setup guide for a USB GPS with ~gpsd~ and ~chrony~. ([[https://photobyte.org/raspberry-pi-stretch-gps-dongle-as-a-time-source-with-chrony-timedatectl/][ref]])
+
+***** Disable CPU power saving
+Power saving featurs of the Raspberry Pi Zero W may also be disabled
+in order to improve accuracy.
+
+****** Configure CPU ~scaling_governor~
+If additional precision is required, the PPS signal may be made more
+reliable at the cost of increasing CPU power by configuring the CPU to
+always run at maximum frequency.[fn:se_20180320_raspicpugov] This
+change can be performed by modifying the following file as root:
+
+: /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
+
+The file should consist of one line. Change
+
+: ondemand
+
+to
+
+: performance
+
+.
+
+This change can be performed via the ~nano~ text editor by running the
+following commands:
+
+: $ sudo su -
+: # nano /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor
+
+Additionally, in order to prevent the ~raspi-config~ init script from
+reverting this text file back to ~ondemand~ after a reboot, this
+script must be disabled via:
+
+: $ sudo systemctl disable raspi-config
+
+****** Configure ~/boot/config.txt~
+Modify ~/boot/config.txt~ so that it contains these lines in order to
+disable power saving functions:
+
+#+BEGIN_EXAMPLE
+# Disable power saving
+nohz=off
+#+END_EXAMPLE
+
+[fn:se_20180320_raspicpugov] Title:[[https://raspberrypi.stackexchange.com/questions/9034/how-to-change-the-default-governor]["How to change the default governor?"]]; Author:[[https://raspberrypi.stackexchange.com/users/5538/goldilocks][goldilocks]]; Date: 2018-03-20; Website:stackexchange.com;
+

 *** Normal Startup
 *** Normal Operation
 *** Normal Shutdown
 *** Unscheduled Shutdown
 *** Normal Startup
 *** Normal Operation
 *** Normal Shutdown
 *** Unscheduled Shutdown

+** Appendix A
+*** Example ~chrony.conf~ for ~chrony~
+For Raspberry Pi OS, the configuration file should be installed at
+~/etc/chrony/chrony.conf~.
+
+#+BEGIN_EXAMPLE
+# Welcome to the chrony configuration file. See chrony.conf(5) for more
+# information about usuable directives.
+pool 2.debian.pool.ntp.org iburst
+
+# This directive specify the location of the file containing ID/key pairs for
+# NTP authentication.
+keyfile /etc/chrony/chrony.keys

 
 

+# This directive specify the file into which chronyd will store the rate
+# information.
+driftfile /var/lib/chrony/chrony.drift
+
+# Uncomment the following line to turn logging on.
+#log tracking measurements statistics
+
+# Log files location.
+logdir /var/log/chrony
+
+# Stop bad estimates upsetting machine clock.
+maxupdateskew 100.0
+
+# This directive enables kernel synchronisation (every 11 minutes) of the
+# real-time clock. Note that it can’t be used along with the 'rtcfile' directive.
+rtcsync
+
+# Step the system clock instead of slewing it if the adjustment is larger than
+# one second, but only in the first three clock updates.
+makestep 1 3
+
+# Get time from GPS (/dev/ttyAMA0) and PPS (/dev/pps0)
+#refclock SOCK /run/chrony.ttyAMA0.sock refid GPS precision 1e-1 offset 0.0000
+#refclock SOCK /run/chrony.pps0.sock refid PP precision 1e-7
+refclock SHM 0 refid GPS precision 1e-1 offset 0.0000 delay 0.2 stratum 1
+refclock SHM 1 refid PPS precision 1e-7 stratum 1
+#+END_EXAMPLE