Raspberry Pi SoC installation script

This documents the procedure for installing the operating system and additional modifications on a Raspberry Pi. The current operating system is Raspbian Stretch, a variant of Debian 9 Stretch.
When viewing in a notepad with language highlighting, specify the script as Python in language.
--------------------------------------------------------------------------------------------------------------------



#========================  NOTES  ===========================
"""
Check VOM for current measurement. No current is read on 2mA scale with 3 digits (N.NN)

tightvnc is installed on the older SD card and RealVNC is on the newer SD card

create 'ssh' file in boot root while in Windows
Due to problems with the installation of hostapd and dnsmasq 'empty filename' error,
compilation of the Epever driver is done last.

Currently, both .102 and .103 IPs are accessible.

If the time is wrong in a ssh sessionusing RPI's wifi, ntp will auto-correct as soon as an internet connection is made.
At this point (completed installation 5-6-2018, ntp is included in Stretch and will update the hardwasre clock automatically if an internet connection is available.

NOTE: when trying to copy to 'html' with www-dat ownership, A saw the 'empty filename' raspberrypi-kernel-headers error message. Those headers were never installed. The driver copied to the /home/pi/ directory, froma different SD card, works.

The RPi will break connections with a laptop on its onw wifi, if a router wifi becomes available. Example: If I'm running an ssh session from my laptop, over CD2G-a, the RPi will break connection if I boot the router and modem.
"""

#Today, 5-6-18, did sudo systemctl start/enable EpeverX-hdr.service successfully.
#No header was being written to the data file, but the resources subdirectory had not been copied to the RPi.

sudo ifconfig
192.168.0.102 : ethernet (LAN) address.
192.168.0.103 : WIFI address.

ping raspberrypi
#C:\Users\jimjulian>ping raspberrypi.msnethome
#Pinging raspberrypi.msnethome [198.105.244.24] with 32 bytes of data:
#Request timed out.


#TODO:
#Modification s/w: if the day's data file has already been written and the RPi rebooted, do not write another header line to the file.

#Modification s/w: edit relay controls for environmental state determination & response.

#Modification s/w: write python code for solar sensor array.
#Modification h/w: hook up solar sensor to RPi. The PCF8951 will be the converter.

#Options: insert current resistors and ADS115s for power sensors.
#Remove thermal sensor from PCF3231 and attach to heat sensitive device, e.x. power supply unit or RPi.

#Check for automated downloads and instalations using the -Y option.

#    "sudo apt install apache2 apache2-utils -y"

#==================== END NOTES  ========================================
"""
Changed config
expand memory -- option: after saving an image of the completed installation.
change password 'passwd Newton'
changed timezone, localization.
-----------------------------------
Set DNS servers
"""

sudo nano /etc/network/interfaces
#added
nameservers 208.67.220.220 208.67.222.222
#to the interfaces file.

sudo nano /etc/resolv.conf
#added:
nameserver 208.67.220.220
nameserver 208.67.222.222
#-----------------------------------


#Default python is 2.7
#Install 3.6
sudo apt-get update
sudo apt-get install build-essential tk-dev libncurses5-dev libncursesw5-dev libreadline6-dev libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev
Is the -Y option available for apt-get?

# https://www.python.org/downloads/source/ - latest version of Python
wget https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tar.xz
tar xf Python-3.6.5.tar.xz
cd Python-3.6.5
./configure
make
sudo make altinstall

sudo apt-get install python3-pip

#From Tutorials.com
##sudo rm -r Python-3.6.0 <-- remove previous user-installed versions
rm Python-3.6.5.tar.xz
sudo apt-get --purge remove build-essential tk-dev
sudo apt-get --purge remove libncurses5-dev libncursesw5-dev libreadline6-dev
sudo apt-get --purge remove libdb5.3-dev libgdbm-dev libsqlite3-dev libssl-dev
sudo apt-get --purge remove libbz2-dev libexpat1-dev liblzma-dev zlib1g-dev


#=======================  RealVNC ==============================
# VNC:
# RealVNC is included in the Stretch distribution.



#========================================================================
#--------------------------------------------------------------------

# RPi.GPIO is integrated into the Stretch distribution.



#================== pigpio  ====================
"""
pigpio has a built in daemon.
A service file can be written for scripts using RPI.GPIO <<----------------<<<<<

wget abyz.co.uk/rpi/pigpio/pigpio.zip
unzip pigpio.zip
cd PIGPIO
make
sudo make install
Test
The Python module requires the services of the pigpio daemon. As a quick test. Start the daemon.
sudo pigpiod
Start Python.
python
NOTE: the following code is legacy. The start method is deprecated. The owner website is archaic and not updated.
Enter the following code (you can cut & paste but do not use the SELECT ALL button as that corrupts the formatting).
import pigpio
pigpio.start()
for g in range(0,32):
   print('gpio {} is {}'.format(g, pigpio.read(g))) # May need double quotes here.
pigpio.stop()
The code prints the level of the first 32 gpios.
To exit Python type exit() or ctrl-D.
A script to show the status of the first 32 gpios with updates is at http://abyz.co.uk/rpi/pigpio/code/gpio_status_py.zip.
END NOTE

UPDATED TEST CODE
#!/usr/bin/env python
import pigpio
pi = pigpio.pi() # connect to Pi
if not pi.connected:
   exit()
for g in range(0,32):
  print('gpio {} is {}'.format(g, pi.read(g)))       # May need double quotes here.
pi.stop() # disconnect from Pi

FROM WEB (with modifications)
#!/usr/bin/env python
# sets GPIO 7 high, then low and
# prints each change to the screen
import pigpio
pi = pigpio.pi() # connect to Pi
if not pi.connected:
   exit()
pi.write(7,1) # set GPIO 7 high
level = pi.read(7) # read GPIO 7
print(level)
pi.write(7,0)  # setr GPIO 7 low
level = pi.read(7) # read GPIO 7
print(level)
pi.stop() # disconnect from Pi
"""
#================= END pigpio  ====================

#-----------------------------------------------
# LAMP stack
sudo apt install apache2 apache2-utils -y
systemctl status apache2
#if not running, already, :
systemctl start apache2
#For authentication, choose'pi' over 'root'. The default password is 'raspberry'.

# Apache, Python, MySQL, and so on.
#apt-get install -y runs with automatic yes to prompts.
# FROM: https://www.linuxbabe.com/debian/install-lamp-stack-debian-9-stretch--
sudo apt-get install apache2 apache2-utils -y
systemctl status apache2
#If it’s not running, use systemctl to start it.
sudo systemctl start apache2
#It’s also a good idea to enable Apache to automatically start at boot time.
sudo systemctl enable apache2
#Check Apache version:
sudo apache2 -v
#Get RPi IP

sudo apt install curl
curl http://icanhazip.com
24.158.81.79
curl 127.0.0.1
#Result is typing the 'index.html' file to the screen.

sudo chown www-data:www-data /var/www/html/ -R
# Had to change this back to 'pi' to copy files to the directory.

sudo apt install mariadb-server mariadb-client # (drop-in replacement for MySQL)
sudo apt install mysql-server mysql-client
systemctl status mariadb
#If it’s not running, start it with this command:
sudo systemctl start mariadb
#To enable MariaDB to automatically start at boot time, run
sudo systemctl enable mariadb
#Now run the post installation security script.
sudo mysql_secure_installation
#When it asks you to enter MariaDB root password, press Enter key as the root password isn’t set yet. Then enter y to set the root password for MariaDB server.
#Next, you can just press Enter to answer all remaining questions. This will remove anonymous user, disable remote root login and remove
#test database. This step is a basic requirement for MariaDB database security.
#Now you can use the following command to log in to MariaDB server.
sudo mariadb -u root
#To exit, run
exit;
#Check MariaDB server version information.
mariadb --version
#PHP
sudo apt install php7.0 libapache2-mod-php7.0 php7.0-mysql php-common php7.0-cli php7.0-common php7.0-json php7.0-opcache php7.0-readline
#Enable the Apache php7.0 module then restart Apache Web server.
sudo a2enmod php7.0
sudo systemctl restart apache2
#Check PHP version information.
php --version
#To test PHP scripts with Apache server, we need to create a info.php file in the Web root directory.
sudo nano /var/www/html/info.php
#Paste the following PHP code into the file.
#<?php phpinfo(); ?>
#Save and close the file. Now in the browser address bar, enter server-ip-address/info.php. Replace sever-ip-address with your actual #IP. #If you follow this tutorial on your local computer, then type 127.0.0.1/info.php or localhost/info.php.
#You should see your server’s PHP information. This means PHP scripts can run properly with Apache web server. You can find that Zend #OPcache is enabled.
cd /var/www/html
cd../
sudo chown pi html
cd html
sudo chown pi index.html
rm index.html
#Edit the name of index.html yo _index.html.
#Open browser at RPi IP. click php file. Should list data.
#-------------------------------------------------
pip install pysolar
pip install pyephem

sudo apt-get install i2c-tools
sudo i2cdetect -y 1
sudo pip install pyephem
sudo pip install pysolar - optional
sudo apt-get autoremove
sudo apt-get install tightvncserver
sudo apt-get install pi-bluetooth
(already installed)
sudo apt-get install bluez bluez-firmware
sudo apt-get install blueman
bluetoothctl --help
git clone https://github.com/adafruit/Adafruit_Python_ADS1x15.git
cd Adafruit_Python_ADS1x15
sudo python setup.py install
# (python 2.7 was used)
#( was installed in current directory: /var/www/html/rersources/Adafruit_Python_ADS1x15/)
#--------------------------------------------------------------------------------
#===============================================================================


#===============  Set Ethernet Address  ========================
sudo ifconfig
sudo cp /etc/dhcpcd.conf /etc/dhcdcp.backup
sudo nano /etc/dhcpcd.conf

#Place these lines at the top of the file too set the Ethernet port (eth0) to a static IP address:
interface eth0
static ip_address=10.11.44.124/24
static routers=10.11.44.1
static domain_name_servers=172.16.33.85

#Set the address (ip_address) to the IP address you want the Raspberry Pi to occupy.
#Set the routers value to the gateway address
#If available, set to domain_name_servers to you name servers
#Once the file has been updated, use ctrl x to save and exit.
#Use the "ping" command to confirm that the Raspberry Pi is on the network
#and talking to another computer also on the network.

#If you have trouble pinging other computers on the network work, check the following:
#1. Confirm that the ethernet cable is firmly connected to the Raspberry Pi and network switch.
#2. Confirm that the ip address, mask and gateway are correct.
#3. If pinging a Windows machine, sometimes security setting prevent responding to a ping request.

#================================================================

#============ BEGIN ACCESS POINT  ===============
# RPi as access point.
#Instalation of hostapdf and dnsmasq are an opportunity to set the fixed IP of the RPi to that set in the router.
192.168.0.103
# This may not be a good idea. When the RPi wifii is beiong used, the 192 series may not be available, or conflict with MS Windows.


#RPI wifi access point

sudo apt-get update
sudo apt-get upgrade
#Install all the required software in one go with this command:
sudo apt-get install dnsmasq hostapd

#Since the configuration files are not ready yet, turn the new software off as follows:
sudo systemctl stop dnsmasq
sudo systemctl stop hostapd

#To configure a static IP address, edit the dhcpcd configuration file with:
sudo nano /etc/dhcpcd.conf

#Go to the end of the file and edit it so that it looks like the following:
interface wlan0
#    static ip_address=192.168.0.103/24
    static ip_address=172.24.1.1/24

#Now restart the dhcpcd daemon and set up the new `wlan0` configuration:
sudo service dhcpcd restart

#Configuring the DHCP server (dnsmasq)
#The DHCP service is provided by dnsmasq. By default, the configuration file contains a lot #of information that is not needed, and it is easier to start from scratch. Rename this #configuration file, and edit a new one:

sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig 
sudo nano /etc/dnsmasq.conf
#Type or copy the following information into the dnsmasq configuration file and save it:

# Edited jlj
interface=wlan0      # Use the require wireless interface - usually wlan0
#  dhcp-range=192.168.0.2,192.168.0.200,255.255.255.0,24h
# Try this with 172.24.1.1
  dhcp-range=172.24.1.1,172.24.0.200,255.255.255.0,24h


#Configuring the access point host software (hostapd)
#You need to edit the hostapd configuration file, located at /etc/hostapd/hostapd.conf, to add the various parameters for your wireless network. After initial install, this will be a new/empty file.

sudo nano /etc/hostapd/hostapd.conf


#Add the information below to the configuration file. This configuration assumes we are using channel 7, with a network name of NameOfNetwork, and a password AardvarkBadgerHedgehog. Note that the name and password should not have quotes around them.

interface=wlan0
driver=nl80211
ssid=CD2G-a
hw_mode=g
channel=7
wmm_enabled=0
macaddr_acl=0
auth_algs=1
ignore_broadcast_ssid=0
wpa=2
wpa_passphrase=rxplc22549-a
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP


#We now need to tell the system where to find this configuration file.

sudo nano /etc/default/hostapd
#Find the line with #DAEMON_CONF, and replace it with this:
DAEMON_CONF="/etc/hostapd/hostapd.conf"

#Start it up
#Now start up the remaining services:
sudo service hostapd start 
sudo service dnsmasq start 
#Add routing and masquerade
#Edit /etc/sysctl.conf and uncomment this line:
net.ipv4.ip_forward=1
sudo nano /etc/sysctl.conf

#Add a masquerade for outbound traffic on eth0:
sudo iptables -t nat -A  POSTROUTING -o eth0 -j MASQUERADE

#Save the iptables rule.
sudo sh -c "iptables-save > /etc/iptables.ipv4.nat"

#Edit /etc/rc.local and add this just above "exit 0" to install these rules on boot.
iptables-restore < /etc/iptables.ipv4.nat

#++++++++++++++++++++Internet Access++++++++++++++++++
'''To get internet access, add the following step:
Step 8: Enable internet connection
Now the Raspberry Pi is acting as an access point to which other devices can connect. However, those devices can’t use the Pi to access the internet just yet. To make the possible, we need to build a bridge that will pass all traffic between the wlan0 and eth0 interfaces.
'''
#To build the bridge, let’s install one more package:
sudo apt-get install bridge-utils
#We’re ready to add a new bridge (called br0):
sudo brctl addbr br0
#Next, we’ll connect the eth0 interface to our bridge:
sudo brctl addif br0 eth0
#Finally, let’s edit the interfaces file:
sudo nano /etc/network/interfaces
#…and add the following lines at the end of the file:
auto br0
iface br0 inet manual
bridge_ports eth0 wlan0
#++++++++++++++++++++++++++++++++++++++++++++++++++++++

sudo reboot

#============ END ACCESS POINT  ===============



#===================================================================================
#PhpEpsolarTracer library -  this is the data collector.
#https://github.com/notro/rpi-source/wiki
#( part of the PHPepsolar togio installation that required recompilation of the Linux kernals)
#("getInfoData() function not returning info for Tracer Model A, here's the fix. · Issue #4 · #toggio/PhpEpsolarTracer · GitHub")

sudo apt-get install raspberrypi-kernel-headers
sudo apt-get install --reinstall raspberrypi-kernel-headers


#Files from RPi snapshots will already have needed changes made.
#===================================================================================
'''
The previous instructions:
1. sudo wget https://raw.githubusercontent.com/notro/rpi-source/master/rpi-source -O /usr/bin/rpi-source && sudo chmod +x /usr/bin/rpi-source && /usr/bin/rpi-source -q --tag-update
2. rpi-source

EPsolar-Tracer
https://github.com/kasbert/epsolar-tracer
This library uses python and is not used but includes the Tracer driver source code and instruction on installation.
Install the new headers from the web.

copy eps directory and go into xr_usb subdirectory. Type "make".
sudo insmod ./xr_usb_serial_common.ko
* Ensure that thecdc-acm module is not loaded (assumig that it is not needed)
# echo blacklist cdc-acm > /etc/modprobe.d/blacklist-cdc-acm.conf
# update-initramfs -u
lsusb
ls /dev/tty*
sudo rmmod cdc-acm
sudo modprobe -r usbserial
sudo reboot
FROM NOTES:
To remove the CDC-ACM driver and install the driver:
# rmmod cdc-acm
# modprobe -r usbserial
# modprobe usbserial
# insmod ./xr_usb_serial_common.ko
Better yet, examine or run epdriver.sh in /home/pi/.
Copy /var/www/html/ files from snapshot to RPi.
Run the client.php as a test of the Epsolar driver.
'''
Run epdriver.sh :
sudo rmmod cdc-acm
sudo insmod /home/pi/xr_usb_serial_common.ko

All that remains is the installation of service files.
Copy files to proper directory, /lib/systemd/system/.
From that directory
sudo cp /home/pi/epdriver.service epdriver.service
sudo cp /home/pi/EpeverX.service EpeverX.service
sudo cp /home/pi/EpeverX-hdr.service EpeverX-hdr.service

sudo systemctl start EpeverX.service
sudo systemctl enable EpeverX.service

sudo systemctl start EpeverX-hdr.service
sudo systemctl enable EpeverX-hdr.service

sudo systemctl start epdriver.service
sudo systemctl enable epdriver.service
# I got the script service running on boot, but failed 'start', yet passed 'enable'.

#NOTES: problems. EpeverX ran immediately. After some time, epdriver ran.
#Make sure the driver has not been loaded manually, when starting.

#//////////////////////////
# The RTC may need to be installed before hwclock is accessible.
#--------------------------------------
# The Stretch os has timedatectl.
# The change to EDT is also made by the timezone edit  using raspi-config.

timedatectl set-local-rtc 0 # for UTC
timedatectl set-local-rtc 1 # for EDT
timedatectl set-ntp true    # enable ntp
timedatectl set-ntp false   # disable ntp

timedatectl status
      Local time: Mon 2018-05-07 11:26:20 EDT
  Universal time: Mon 2018-05-07 15:26:20 UTC
        RTC time: Mon 2018-05-07 15:26:20
       Time zone: US/Eastern (EDT, -0400)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

#Read clock
#To check the current system clock time (presented both in local time and UTC) as well as the RTC (hardware clock):
 timedatectl
#Set system clock
#To set the local time of the system clock directly:
 timedatectl set-time "yyyy-MM-dd hh:mm:ss"
#For example:
 timedatectl set-time "2014-05-26 11:13:54"

# timezone
America/New_York
--------------------------------------


sudo nano /boot/config.txt
add: dtoverlay=i2c-rtc,ds3231
sudo nano /lib/udev/hwclock-set
and comment out the following lines (using the #)

if [ -e /run/systemd/system ] ; then
   exit 0
fi

It’s also a good idea to comment out all the –systz lines too (as shown below). These update the RTC from the RPi and can cause issues.

fi
if [ yes - "$BADYEAR" ] ; then
#  /sbin/hwclock --rtc=$dev --systz --badyear
  /sbin/hwclock --rtc=$dev --hctosys --badyear
else
#  /sbin/hwclock --rtc=$dev --systz --badyear
  /sbin/hwclock --rtc=$dev --hctosys
fi

then exit and save (Ctrl-X then y). This will stop that script exiting early and will update the clock on boot.
Next reboot and the configuration is done.
i2cdetect -y 1
#RTC Clock: '68' at 68x8
# In this case, as in others, following the installation of the RTC, the '68' becomes a 'UU'.

#============================================================
#============================================================
#Note: ntp may already be installed in Stretch.
sudo apt-get install ntp
#============================================================
#RTC On Stretch
#from:
#https://www.raspberrypi.org/forums/viewtopic.php?t=209700
sudo nano /boot/config.txt
#add
dtparam=i2c_arm=on
dtoverlay=i2c-rtc,ds3231

sudo hwclock --systohc
May need to add the -w switch
sudo hwclock -w --systohc

sudo apt-get purge fake-hwclock

sudo nano /etc/udev/rules.d/85-hwclock.rules
add
# On the Raspberry Pi the RTC isn't available when systemd tries,
# set the time from RTC now when it is available.
KERNEL=="rtc0", RUN+="/sbin/hwclock --rtc=$root/$name --hctosys"

#Done.
#-----------------------------------------------
#Date and hwclock may report in UTC.
#Run raspi-config and set localization to U.S.
#UTF-8 may be an option.

#============================================================
#============================================================

#//////////////////////////
#EpeverX and epdriver services are installed. EpeverX-hdr is not. The php file does work.
#Make sure the PHP designation in the service files reflects one of the links in the /usr/bin/directory. PHP7.0 replaces PHP5.


#==========================================================================
#For an alternative access to the Tracer:
pip3 install pymodbus <-- did not work.
pip install pymodbus <-- worked
sudo pip3 install pymodbus
sudo pip install pymodbus

run stackexchange.py
NOTE: sometimes python/pip will not produce an error is sudo is needed but not used.
sudo pip install adafruit-ads1x15
#==================================================


sudo apt-get autoremove
sudo apt-get clean



# Best for the end of installations:
sudo apt-get autoremove
sudo apt-get clean


-------------------------------------------------------------------------------