Adventures in Networking

Main menu:

Uncategorized

Real-time Audio Streaming with Raspberry Pi

References:

  • Basic Idea:
    • https://tech.lds.org/forum/viewtopic.php?t=4812&start=30
    • https://tech.lds.org/forum/viewtopic.php?t=18758
  • Raspberry Pi Setup:
    • https://www.zdnet.com/article/raspberry-pi-extending-the-life-of-the-sd-card/
    • https://www.raspberrypi.org/forums/viewtopic.php?p=462982#p462982
      • Not needed with Pi 3!
    • https://raymii.org/s/tutorials/Autossh_persistent_tunnels.html
    • https://ritsch.io/2017/08/02/execute-script-at-linux-startup.html
    • https://medium.com/@mikestreety/use-a-raspberry-pi-with-multiple-wifi-networks-2eda2d39fdd6
  • Sound adapter:
    • https://smile.amazon.com/gp/product/B016CU2PEU/ref=oh_aui_detailpage_o01_s00?ie=UTF8&psc=1
  • Darkice, Icecast, etc:
    • https://stmllr.net/blog/live-mp3-streaming-from-audio-in-with-darkice-and-icecast2-on-raspberry-pi/
    • https://github.com/tim273/weather_radio_scripts
    • https://w7apk.com/streaming-radio
    • http://icecast.org/docs/icecast-2.4.1/basic-setup.html
    • http://icecast.org/docs/icecast-2.4.1/auth.html
    • http://wiki.sunfounder.cc/index.php?title=To_use_USB_mini_microphone_on_Raspbian
    • https://wiki.radioreference.com/index.php/Live_Audio/Ubuntu_Darkice
    • http://www.hamblog.co.uk/top-10-amateur-radio-uses-for-raspberry-pi/
  • Internet hosting by:
    • Linode
    • Debian, UFW
  • Client:
    • VLC

rtl_fm.sh

rtl_fm -f 72.1M -s 48000 | play -q -r 48000 -t raw -e s -b 16 -c 1 -V1 -v 4 - sinc 125-3.2k

darkice.cfg

[general]
duration = 0 # duration in s, 0 forever
bufferSecs = 1 # buffer, in seconds
reconnect = yes # reconnect if disconnected

[input]
device = plughw:1,1 # or `pulse' for Pulseaudio
sampleRate = 22050 # sample rate 11025, 22050 or 44100
bitsPerSample = 16 # bits
channel = 1 # 2 = stereo

[icecast2-0]
bitrateMode = vbr # variable bit rate (`cbr' constant, `abr' average)
quality = 1.0 # 1.0 is best quality
format = mp3 # format. Choose `vorbis' for OGG Vorbis
bitrate = 128 # bitrate
server = xx.xx.xx.xx # IP address of the IceCast server
port = 8000 # port for IceCast2 access
password = snajd # source password to the IceCast2 server
mountPoint = lcw.mp3 # mount point on the IceCast2 server .mp3 or .ogg
name = Lewis Center Ward

darkice.service

[Unit]
Description=darkice
After=network.target

[Service]
Type=simple
User=root
ExecStart=/usr/bin/darkice -c /xxxxx/darkice.cfg

[Install]
WantedBy=multi-user.target

autossh.service

[Unit]
Description=autossh
Wants=network-online.target
After=network-online.target
# sshd.service

[Service]
Type=simple
User=root
ExecStart=/usr/bin/autossh -M 0 -N -q -o "ExitOnForwardFailure=yes" -M 1234 -i /home/xxxxxxxx/.ssh/id_rsa -R 3333:localhost:22 user@example.com
#Restart=always
#RestartSec=60
#ExecStop=/usr/bin/killall -s autossh

[Install]
WantedBy=multi-user.target

Finding Files or Directories in Linux

find . -type d -exec ls -la {} \;

Reverse Differential Backup

Setup for Backup Server:

  • Configure forwarding of root’s email to GroupWise System Admin resource:
    echo user@test.com > /root/.forward
  • Create/modify /etc/rsyncd.conf. See example Backup Server rsyncd.conf.
  • Enable rsync daemon: chkconfig rsyncd on
  • Start rsyncd: rcrsyncd start
  • Change into the backup volume/directory:
    • For a NSS BACKUP volume: cd /media/nss/BACKUP
    • For a XFS /backup directory: cd /backup
  • Create the backup folders for the primary server:
    mkdir primaryservername
    mkdir primaryservername/APPS
    mkdir primaryservername/backup
    mkdir primaryservername/boot
    mkdir primaryservername/etc
    mkdir primaryservername/GW
    mkdir primaryservername/mbr
    mkdir primaryservername/SHARED
    mkdir primaryservername/USERS
  • Create the backup folders for the backup server:
    mkdir backupservername
    mkdir backupservername/backup
    mkdir backupservername/boot
    mkdir backupservername/etc
    mkdir backupservername/mbr
  • Modify /etc/exports to export the BACKUP volume and /install directory:
    • /install *(ro,sync,root_squash) 10.1.1.0/255.255.255.0(rw,no_root_squash,sync) 10.1.2.0/255.255.255.0(rw,no_root_squash,sync)
    • For NSS BACKUP Volume:
      /media/nss/BACKUP primaryservername(rw,no_root_squash,sync,fsid=20) backup1(rw,no_root_squash,sync,fsid=20)
    • For XFS /backup directory:
      /backup primaryservername(rw,no_root_squash,sync) backup1(rw,no_root_squash,sync)
  • Enable nfsserver: chkconfig nfsserver on
  • Start nfsserver: rcnfsserver start
  • Create backup script bkp_backupservername.sh. See example Backup Server Backup Script.
  • Set executable permission: chmod +x bkp_backupservername.sh
  • Create lst file. See example exclude.lst.
  • Use crontab -e to edit the cron jobs and add the backup script:
    • For a NSS BACKUP volume: @daily /media/nss/BACKUP/bkp_backupservername.sh
    • For a XFS /backup directory: @daily /backup/bkp_backupservername.sh

 

Backup Setup for Primary Server:

  • Configure forwarding of root’s email to GroupWise System Admin resource:
    echo user@test.com > /root/.forward
  • Create the backup folder: mkdir /backup
  • Change into backup folder: cd /backup
  • Create the mount folder: mkdir primaryservername
  • Modify /etc/fstab to auto-mount the backup volume:
    • NSS BACKUP volume:
      backupservername:/media/nss/BACKUP/primaryservername /backup/primaryservername nfs rw,intr,proto=tcp,rsize=32768,wsize=32768 0 0
    • XFS /backup directory:
      backupservername:/backup/primaryservername /backup/primaryservername nfs rw,intr,proto=tcp,rsize=32768,wsize=32768 0 0
  • Mount the backup volume: mount -a
  • Create backup script bkp_primaryservername.sh. See example Primary Server Backup Script.
  • Set executable permission: chmod +x bkp_primaryservername.sh
  • Create lst file. See example exclude.lst.
  • Use crontab -e to edit the cron jobs and add:
    @daily /backup/bkp_primaryservername.sh

 

Example: Backup Server rsyncd.conf (NSS BACKUP volume):

uid = root
gid = root
read only = false
use chroot = true
transfer logging = false
log format = %h %o %f %l %b
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = primaryservername.test.com backup.test.com
slp refresh = 300

[primaryservername_APPS]
path = /media/nss/BACKUP/primaryservername/APPS

[primaryservername_boot]
path = /media/nss/BACKUP/primaryservername/boot

[primaryservername_etc]
path = /media/nss/BACKUP/primaryservername/etc

[primaryservername_GW]
path = /media/nss/BACKUP/primaryservername/GW

[primaryservername_USERS]
path = /media/nss/BACKUP/primaryservername/USERS

[primaryservername_SHARED]
path = /media/nss/BACKUP/primaryservername/SHARED

 

Example: Backup Server rsyncd.conf (XFS /backup directory):

uid = root
gid = root
read only = false
use chroot = true
transfer logging = false
log format = %h %o %f %l %b
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
hosts allow = primaryservername.test.com backup1.test.com
slp refresh = 300

[primaryservername_APPS]
path = /backup/primaryservername/APPS

[primaryservername_boot]
path = /backup/primaryservername/boot

[primaryservername_etc]
path = /backup/primaryservername/etc

[primaryservername_GW]
path = /backup/primaryservername/GW

[primaryservername_USERS]
path = /backup/primaryservername/USERS

[primaryservername_SHARED]
path = /backup/primaryservername/SHARED

 

Example: Backup Server Backup Script:

#!/bin/bash
# Version: 2008-02-27 03:10pm
# Modified By: Dan O.

StartTime=`date`
Today=`date +%F`
Yesterday=`date --date=yesterday +%F`
PurgeDate=`date --date='31 days ago' +%F`

echo "**********************************************************************"
echo "Starting backup of system files on $HOSTNAME at $StartTime..."

echo -e "\nChanging into backup directory '/backup/$HOSTNAME'..."
cd /backup/$HOSTNAME

echo -e "\nPurging backup folders older than 30 days..."
echo -e "backup
boot
etc
mbr" |
while read DirName
do
  find $DirName/* -type d ! -name . -prune -mtime +30 -exec rm -rf {} \;
done

echo -e "\nCreating new backup directories for $Today:"
echo -e "backup
boot
etc
mbr" |
while read DirName
do
  mkdir -v $DirName/$Today
done

echo -e "\nRotating files from $Yesterday/ to $Today/..."
# No rotation for /backup/ scripts or mbr (too small to worry about differential)
echo -e "boot
etc" |
while read DirName
do
  mv $DirName/$Yesterday/* $DirName/$Today/
done

echo -e "\nSaving backup scripts..."
cp /backup/* backup/$Today/
touch backup/$Today

echo -e "\nSaving Master Boot Record for sda & sdb..."
dd if=/dev/sda of=mbr/$Today/sda-mbr bs=512 count=1
dd if=/dev/sdb of=mbr/$Today/sdb-mbr bs=512 count=1
touch mbr/$Today

echo -e "\nBacking up /boot/ directory:"
rsync -azb --stats --delete --backup-dir boot/$Yesterday/ /boot/ boot/$Today/
touch boot/$Today

echo -e "\nBacking up /etc/ directory:"
rsync -azb --stats --delete --backup-dir etc/$Yesterday/ /etc/ etc/$Today/
touch etc/$Today

EndTime=`date`

echo -e "\n\nBackup of system files on $HOSTNAME completed at $EndTime."
echo "**********************************************************************"

 

Example: exclude.lst:

._NETWARE
~DFSINFO.8-P
VOLDATA.TDF
.Trash-root
Thumbs.db
thumbs.db
THUMBS.DB

 

Example: Primary Server Backup Script:

#!/bin/bash
# Version: 2008-02-27 03:00pm
# Modified By: Dan O.

StartTime=`date`
Today=`date +%F`
Yesterday=`date --date=yesterday +%F`
PurgeDate=`date --date='31 days ago' +%F`
BackupHost="backupservername"

echo "**********************************************************************"
echo "Starting backup from $HOSTNAME to $BackupHost at $StartTime..."

echo -e "\nMaking sure backup directory is mounted with 'mount -a'..."
mount -a
# Would rather do a check to make sure it's already mounted. If not, try to mount. If it fails, abort.

echo -e "\nChanging into backup directory '/backup/$HOSTNAME'..."
cd /backup/$HOSTNAME
# Need to add check to make sure this directory change was successful, or abort.

echo -e "\nPurging backup folders older than 30 days..."
echo -e "backup
boot
etc
GW
mbr
SHARED
USERS" |
while read DirName
do
#  rm -rf $DirName/$PurgeDate
#  find $DirName/* -type d ! -name . -prune -mtime +30 -exec echo {} \;
  find $DirName/* -type d ! -name . -prune -mtime +30 -exec rm -rf {} \;
done

# Creating a new new "$Today" directory, then moving $Yesterday/* $Today/
# keeps the timestamp on $Yesterday/, and makes the script better able to handle
# manual backups (no nested $Today/$Yesterday, which can happen on a restart)
echo -e "\nCreating new backup directories for $Today:"
echo -e "backup
boot
etc
GW
mbr
SHARED
USERS" |
while read DirName
do
  mkdir -v $DirName/$Today
done

echo -e "\nRotating files from $Yesterday/ to $Today/..."
# No rotation for /backup/ scripts or mbr (too small to worry about differential)
echo -e "boot
etc
GW
SHARED
USERS" |
while read DirName
do
  mv $DirName/$Yesterday/* $DirName/$Today/
done

echo -e "\nSaving backup scripts..."
cp /backup/* backup/$Today/
touch backup/$Today

echo -e "\nSaving Master Boot Record for sda & sdb..."
dd if=/dev/sda of=mbr/$Today/sda-mbr bs=512 count=1
dd if=/dev/sdb of=mbr/$Today/sdb-mbr bs=512 count=1
touch mbr/$Today

echo -e "\n\nBacking up APPS volume:"
rsync -az --stats --delete --exclude-from=/backup/exclude.lst /media/nss/APPS/ "$BackupHost"::"$HOSTNAME"_APPS/
touch APPS/

BackupOptions="-azb --stats --delete --exclude-from=/backup/exclude.lst --backup-dir /$Yesterday/"
echo -e "\nBackup options set to '$BackupOptions'..."

echo -e "\nBacking up /boot/ directory:"
rsync $BackupOptions /boot/ "$BackupHost"::"$HOSTNAME"_boot/$Today/
touch boot/$Today

echo -e "\nBacking up /etc/ directory:"
rsync $BackupOptions /etc/ "$BackupHost"::"$HOSTNAME"_etc/$Today/
touch etc/$Today

echo
/etc/init.d/grpwise stop
echo "Backing up GW volume:"
rsync $BackupOptions /media/nss/GW/ "$BackupHost"::"$HOSTNAME"_GW/$Today/
/etc/init.d/grpwise start
touch GW/$Today

echo -e "\n\nSaving Trustees for USERS volume..."
/opt/novell/nss/sbin/metamig save USERS -m t > /media/nss/USERS/trustees.xml
echo "Backing up USERS volume:"
rsync $BackupOptions /media/nss/USERS/ "$BackupHost"::"$HOSTNAME"_USERS/$Today/
touch USERS/$Today

echo -e "\n\nSaving Trustees for SHARED volume..."
/opt/novell/nss/sbin/metamig save SHARED -m t > /media/nss/SHARED/trustees.xml
echo "Backing up SHARED volume:"
rsync $BackupOptions /media/nss/SHARED/ "$BackupHost"::"$HOSTNAME"_SHARED/$Today/
touch SHARED/$Today

EndTime=`date`

echo -e "\n\nBackup of $HOSTNAME to $BackupHost completed at $EndTime."
echo "**********************************************************************"