Vote!
When
travelling a few years ago, I needed my laptop to check the weather,
book train and flight tickets, check emails and much more.
Now I can do all this stuff with my smartphone and carrying a big laptop with power-supply seems a little out-dated. And there is one function that my smartphone can not provide at the moment: Backup all my pictures and videos from several cameras and sources!
But, you don't need a big laptop for that. A small Wifi-Backup-HDD with SD-card slot is enough. There are devices like this out there, but I don't trust them, because they don't have a display, I don't see what is happening and they cost twice as much as a simple 2.5 USB-HDD.
So I set up a Raspberry Pi 3 to do the job and it works great!
You can use as much USB-HDDs as you want and have full control!
And you can even add your own wifi-hotspot to backup your mobile data too!
Let's Start!
Now I can do all this stuff with my smartphone and carrying a big laptop with power-supply seems a little out-dated. And there is one function that my smartphone can not provide at the moment: Backup all my pictures and videos from several cameras and sources!
But, you don't need a big laptop for that. A small Wifi-Backup-HDD with SD-card slot is enough. There are devices like this out there, but I don't trust them, because they don't have a display, I don't see what is happening and they cost twice as much as a simple 2.5 USB-HDD.
So I set up a Raspberry Pi 3 to do the job and it works great!
You can use as much USB-HDDs as you want and have full control!
And you can even add your own wifi-hotspot to backup your mobile data too!
Let's Start!
Step 1: Hardware Setup
What do you need:
- Raspberry Pi 3 (2 or the older 1 will work too)
- USB external HDD ( like 1TB or bigger)
- a good 5V 2A Power-supply
- a raspberry Pi housing or a cardboardbox or similar
- Wifi-Dongle for a Raspberry Pi 2 or 1, the 3rd generation has Wifi on board!
- one to three LEDs with 1k resistors
- a small piece of breadboard
- a pushbutton
Step 2: Software Setup
This is the list of software we are going to use for this project:
- usbmount: for automatic mounting any usb-stick.
- Raspberry Python GPIO: for controlling the pushbuttons
- Bash-GPIO: for direct control of the LEDs from within a bash-script.
- hostapd: for creating a wifi-hotspot with your pi.
- samba: for easy file-sharing with your phone
Step 3: Optional LED-message Board
Because I operate the Raspi without a keyboard and a display, it is hard to say when copying is really finished.
Most external HDDs or USB-sticks have a built-in LED that show read or write access, but this is only a weak indicator.
To shut the raspberry down safely, I use a simple pushbutton. This way I don't have to remotely log in via smartphone to issue the shutdown command. This will improve the data integrity a lot! And it only costs a few cents!
So get yourself a small breadboard, a single 20-pin female connector and start soldering the connector to the bread board. (You can also use a dual-row connector, but these are more expensive and the breadboard might get crowded)
I chose to directly connect the LED-Anode to a Raspberry Pin and via a 1k-Ohm resistor to a ground-pin. You should try and measure if 3.3V (what the rapsberry provides) is enough to light up the LED via the 1k-resistor. I measured my LEDs and they took 1.5mA while providing a good brightness.
The Pushbutton also directly connects a pin with a ground pin.
After some initial testing I found out, that starting the rsync command from usbmount directly didn't work well. Therefore I decided to have another pushbutton, solely for the purpose of starting the copy-process.
Most external HDDs or USB-sticks have a built-in LED that show read or write access, but this is only a weak indicator.
To shut the raspberry down safely, I use a simple pushbutton. This way I don't have to remotely log in via smartphone to issue the shutdown command. This will improve the data integrity a lot! And it only costs a few cents!
So get yourself a small breadboard, a single 20-pin female connector and start soldering the connector to the bread board. (You can also use a dual-row connector, but these are more expensive and the breadboard might get crowded)
I chose to directly connect the LED-Anode to a Raspberry Pin and via a 1k-Ohm resistor to a ground-pin. You should try and measure if 3.3V (what the rapsberry provides) is enough to light up the LED via the 1k-resistor. I measured my LEDs and they took 1.5mA while providing a good brightness.
The Pushbutton also directly connects a pin with a ground pin.
After some initial testing I found out, that starting the rsync command from usbmount directly didn't work well. Therefore I decided to have another pushbutton, solely for the purpose of starting the copy-process.
Step 4: SD-card Setup and Preparation
Get the most recent version of raspbian-lite from here:
https://www.raspberrypi.org/downloads/raspbian/
I used raspbian lite, because I don't need a desktop and all the other stuff, that makes the full blown raspbian so big.
Prepare your sd-card with your PC in the way that works for you: https://www.raspberrypi.org/documentation/installation/installing-images/README.md
Then hook up your raspi with a network cable and power it up. If you don't know how to remotely access your Pi with ssh, have a look at some tutorials or use a keyboard and a display on the Pi itself.
The next step should be the initial change of the password and a software update.
To change the password simply start the raspi-config, there you can also change the locales and other stuff:
https://www.raspberrypi.org/downloads/raspbian/
I used raspbian lite, because I don't need a desktop and all the other stuff, that makes the full blown raspbian so big.
Prepare your sd-card with your PC in the way that works for you: https://www.raspberrypi.org/documentation/installation/installing-images/README.md
Then hook up your raspi with a network cable and power it up. If you don't know how to remotely access your Pi with ssh, have a look at some tutorials or use a keyboard and a display on the Pi itself.
The next step should be the initial change of the password and a software update.
To change the password simply start the raspi-config, there you can also change the locales and other stuff:
sudo raspi-configTo update the software:
sudo apt-get update sudo apt-get upgrade
Step 5: Basic Interfacing and GPIO-Setup
Well,
the setup might not be ideal, but I chose to use a combination of
direct access to the LEDs from the console or bash-scripts and
python-scripts.
With bash-scripts one can very easily handle LEDs, turn them on and off. But you have a problem with handling interrupts.
This is where python comes in handy. But on the other hand, it is hard to perform file-system copy-processes within a python script. Therefore I use the python script to handle the pushbuttons-Interrupt and then call a bash-script to do the job!
The "gpiosetup.sh"-script initializes all LEDs and pushbuttons to be handled by further software.
The shutdownbutton.py handles the shutdown and copy-button and blinks the little green led. So I know, that the python-script is still running.
The little yellow led only shows that the software has boot up and is ready to copy.
The initial gpio-setup could be done by the user pi, while the python script should be run by root, because we need the rights to shutdown the pi. Just have a look at the crontab-entries how to start the job after boot.
With bash-scripts one can very easily handle LEDs, turn them on and off. But you have a problem with handling interrupts.
This is where python comes in handy. But on the other hand, it is hard to perform file-system copy-processes within a python script. Therefore I use the python script to handle the pushbuttons-Interrupt and then call a bash-script to do the job!
The "gpiosetup.sh"-script initializes all LEDs and pushbuttons to be handled by further software.
The shutdownbutton.py handles the shutdown and copy-button and blinks the little green led. So I know, that the python-script is still running.
The little yellow led only shows that the software has boot up and is ready to copy.
The initial gpio-setup could be done by the user pi, while the python script should be run by root, because we need the rights to shutdown the pi. Just have a look at the crontab-entries how to start the job after boot.
shutdownbuttonhandler.py.txt
root_cron.txt
pi_cron.txt
Step 6: Connect the USB-HDD to the System
So
first of all, we have to connect the usb-hdd to the system. This should
be done during boot-up, but if the usb-hdd is not present for any
reason, then the boot-process should of course not fail!
There are several good tutorial how to do this, depending on your hdd and the file-system on it:
https://www.instructables.com/id/Using-a-USB-external-hard-drive-with-your-Raspberr/
https://www.modmypi.com/blog/how-to-mount-an-external-hard-drive-on-the-raspberry-pi-raspian
https://jankarres.de/2013/01/raspberry-pi-usb-stick-und-usb-festplatte-einbinden/
In my setup I chose to mount the usb-hdd in the folder: /media/usbhdd/
The line in my fstab looks like this:
utf8: this is to enable german special characters like ä, ü or ö
uid=pi,gid=pi: mount the usb-hdd in the name of pi
noexec prevents any executable to be run from the hdd
nofail prevents the boot-process to fail if the harddisc is not plugged in.
There are several good tutorial how to do this, depending on your hdd and the file-system on it:
https://www.instructables.com/id/Using-a-USB-external-hard-drive-with-your-Raspberr/
https://www.modmypi.com/blog/how-to-mount-an-external-hard-drive-on-the-raspberry-pi-raspian
https://jankarres.de/2013/01/raspberry-pi-usb-stick-und-usb-festplatte-einbinden/
In my setup I chose to mount the usb-hdd in the folder: /media/usbhdd/
The line in my fstab looks like this:
UUID=3284-E8C7 /media/usbhdd/ vfat utf8,uid=pi,gid=pi,auto,noexec,noatime,rw,nofail 0 0explanation:
utf8: this is to enable german special characters like ä, ü or ö
uid=pi,gid=pi: mount the usb-hdd in the name of pi
noexec prevents any executable to be run from the hdd
nofail prevents the boot-process to fail if the harddisc is not plugged in.
Step 7: Install and Configure Usbmount
Next
step is to install and configure usbmount. Usbmount is a piece of
software that automatically mounts any usb-mass-storage device
available. You can configure how the drives are mounted and where.
To install it:
The second line allows user to access the mounted usb-sticks.
To install it:
sudo apt-get install usbmountTo configure it, you need to edit the config-file:
sudo nano /etc/usbmount/usbmount.confI changed two distinctive lines in the config-file, these are:
MOUNTOPTIONS="noexec,nodev,noatime,nodiratime,iocharset=utf8" FS_MOUNTOPTIONS="-fstype=vfat,gid=users,dmask=0007,fmask=0117"The first line is like the configuration in the fstab, it prevents executables to be run from the usb-sticks and the utf8 takes care of special characters in the german language.
The second line allows user to access the mounted usb-sticks.
Step 8: Optional: Install and Configure Wifi-access Point
This
step is optional. You only have to do it, if you want to remotely
access your pi with your smartphone or tablet. Which is a good thing,
because you can log in and have a look, what is already there in the
backup. Or check if a specific file is still present or not. And you can
even use it to backup your phone!
This step is not a big deal, but there are already a lot of good tutorials out there, so I only refere to some tutorials:
https://frillip.com/using-your-raspberry-pi-3-as-a-wifi-access-point-with-hostapd/
https://cdn-learn.adafruit.com/downloads/pdf/setting-up-a-raspberry-pi-as-a-wifi-access-point.pdf
This step is not a big deal, but there are already a lot of good tutorials out there, so I only refere to some tutorials:
https://frillip.com/using-your-raspberry-pi-3-as-a-wifi-access-point-with-hostapd/
https://cdn-learn.adafruit.com/downloads/pdf/setting-up-a-raspberry-pi-as-a-wifi-access-point.pdf
Step 9: Optional: Install Samba for Easy File Access
Once
you have your wifi-access point running, you should think of installing
samba. This provides the service you need to access the raspberry pi
with a file-manager on your phone.
But why to use samba, when NFS works much faster and is safer?
Well, I found out that using a NFS-share from an Android phone is not that easy. From a windows phone or a PC it might be even harder.
And because this server is not running 24/7 and "normally" not connected to the internet, the samba-service is not that big issue.
Just follow this simple tutorial and adjust to your need: http://www.raspberry-pi-geek.com/howto/Samba-Deploying
By adding additional shares in the smb.conf you can even provide access to the usb-hdd or other folders of the system.
But why to use samba, when NFS works much faster and is safer?
Well, I found out that using a NFS-share from an Android phone is not that easy. From a windows phone or a PC it might be even harder.
And because this server is not running 24/7 and "normally" not connected to the internet, the samba-service is not that big issue.
Just follow this simple tutorial and adjust to your need: http://www.raspberry-pi-geek.com/howto/Samba-Deploying
By adding additional shares in the smb.conf you can even provide access to the usb-hdd or other folders of the system.
Step 10: The Essential BACKUP-SCRIPT!
After so many steps of preparation we now come to the most important script of this device: the backup-script itself!
What do I want it to do?
Well, backup each usb-stick into a special directory on the usb-hdd and only copy new files while still keeping the old ones of course.
O.k. a two step approach:
That is what in principle happens in the script provided by this step. No rocket science, but sophisticated bash scripting triggered by the python-script which handles the pushbutton interrupt.
What do I want it to do?
Well, backup each usb-stick into a special directory on the usb-hdd and only copy new files while still keeping the old ones of course.
O.k. a two step approach:
- Get the UUID from the usb-stick and use this as a directory name on the hdd for the backup.
- Use rsync to do the backup.
That is what in principle happens in the script provided by this step. No rocket science, but sophisticated bash scripting triggered by the python-script which handles the pushbutton interrupt.
Step 11: Get Everything Together and Test It!
Now that we have everything together and running it is time to test the function:
Start up the raspberry with the usb-hdd plugged in and wait until the little green LED is blinking.
Now insert an usb-stick and press the copy button. Then the red LED should start to light up and the stick and the hdd should start to show some kind of access. While the copy process is still ongoing, when you issue "df" from the console, it should at least contain a line with "/media/usb0". When the red LED is off again and you issue "df", the line with "/media/usb0" should be gone. This indicates that the usb-stick was unmounted correctly. Now go to the hdd-folder and have a look if all the data from the stick was correctly backuped.
If this is the case you are done here and have a very nice small backup-device powered by raspberry pi!
Start up the raspberry with the usb-hdd plugged in and wait until the little green LED is blinking.
Now insert an usb-stick and press the copy button. Then the red LED should start to light up and the stick and the hdd should start to show some kind of access. While the copy process is still ongoing, when you issue "df" from the console, it should at least contain a line with "/media/usb0". When the red LED is off again and you issue "df", the line with "/media/usb0" should be gone. This indicates that the usb-stick was unmounted correctly. Now go to the hdd-folder and have a look if all the data from the stick was correctly backuped.
If this is the case you are done here and have a very nice small backup-device powered by raspberry pi!
pi_cron.txt
Step 12: Outlook...
I
wished I could have used the Raspberry pi zero or the model A for that
project, but unfortunately they only have one accessible USB-Port. And
the zero only provides a micro-USB connector.
I could have used a USB-Hub for it, but didn't like the idea of powering an external HDD and a memory-stick with only one port.
In the end I think it turned out quite nice. The Raspberry Pi housing is nearly as big as the external HDD-housing. The cables are not too long and everything could be stored in a small box.
And the housing provides a good protection against unintended shorts or ESD.
In hotels, where there is only a wired connection: I normally unplug the hdd and use the raspberry pi as a simple access point to provide wifi for my phone. That even saves me additional wifi-access point hardware!
This is the backup of a ~150MB file from SD-Card to the external hdd, from boot to shutdown!
I could have used a USB-Hub for it, but didn't like the idea of powering an external HDD and a memory-stick with only one port.
In the end I think it turned out quite nice. The Raspberry Pi housing is nearly as big as the external HDD-housing. The cables are not too long and everything could be stored in a small box.
And the housing provides a good protection against unintended shorts or ESD.
In hotels, where there is only a wired connection: I normally unplug the hdd and use the raspberry pi as a simple access point to provide wifi for my phone. That even saves me additional wifi-access point hardware!
This is the backup of a ~150MB file from SD-Card to the external hdd, from boot to shutdown!
No comments:
Post a Comment