Wednesday, 21 September 2016

This is simply a step by step guide showing how to use a NAS (network available storage) service (in this case WD MyCloud which is simply a debian family Linux NAS server) with Linux (in this case Linux Mint 18 based on Ubuntu). In my case I wanted NAS with a set of users, each having a private space as well as a shared area where we can all share music, videos, documents and images/photos. I'm assuming you have a local network with at least one PC with Linux including a Web browser (I used Firefox) and terminal app.
  1. Connect the NAS server to the network and power it up. Go to your router administration page for your home router and go to DHCP reservation. Reserve the ip address for the NAS server (It’s called MyCloud and will be obvious in the list). Take note of the ip address MyCloud is using.
  2. In your browser follow the steps to initialise your NAS server that came with the device. With MyCloud note each user added must have a unique email address so if administration is also a user you need 2 email addresses one for admin and one for the user.
  3. In order to go into the administration service for the MyCloud device type just the MyCloud ip address in your browser address field and log in using the admin account details you just created.
  4. Once in the administration page go to Shares set up a share for each user and give them read/write access. To be able to access your share in Linux switch on the NFS access. You can also restrict access to only one ip if you need it or leave it as * which means any workstation can mount that share.
  5. Create a shared share for common data and give all users access as required. Switch on NFS access with host as *
  6. Go to settings->Network and switch on ssh (take note of your ssh password). MyCloud will warn you about using ssh to make significant changes to the server - ignore that - we're just going to use ssh to shutdown the server - there's no other way to do it in Linux. For reasons only they understand Wester Digital have removed the shutdown option from the administration page
  7. Now on each Linux workstation install nfs-common if not already installed: sudo apt-get install nfs-common
  8. Use terminal to check available mount points (shares) on the server: sudo showmount -e 192.168.0.36 [obviously replace this ip address with your NAS server's ip address!]
  9. Create two mount points on your PC /mnt/NAS/shared and /mnt/NAS/user. It's important that the mount points are outside the user’s home directory so that when ever we want to back up the home directory we don't have to concern ourselves with backing up data that's already on the NAS server! We'll create symbolic links to the mounted NAS folders in our home folder later.
  10. As root (sudo xed) edit /etc/fstab and add these lines replacing the ip address with your NAS server’s address and changing the server directory paths as needed based on the showmount output:

192.168.0.36:/mnt/HD/HD_a2/shared /mnt/NAS/shared nfs rw,udp,users,vers=3,soft,intr,rsize=8192,wsize=8192 0 0
192.168.0.36:/mnt/HD/HD_a2/matthew /mnt/NAS/user nfs rw,udp,users,vers=3,soft,intr,rsize=8192,wsize=8192 0 0

  1. In terminal execute: sudo mount -a
  2. You should now have mounted the NAS folders on your PC. Create folders: Music, Images, Documents and Videos under /mnt/NAS/shared (because this folder is a mount point you are creating folders in the shared area on your server for everyone to use).
  3. In your home directory rename Music to MusicOld, Videos to VideosOld and Pictures to PicturesOld - unless there's nothing in them in which case just delete them!
  4. In terminal, in your home folder type: ln -s /mnt/NAS/shared/Music and then ln -s /mnt/NAS/shared/Videos and then ln -s /mnt/NAS/shared/Images Pictures
  5. Still in terminal: cd Documents and then ln -s /mnt/NAS/shared/Documents SharedDocuments
  6. You should now have symbolic links to those areas in your home folder now so that as you add and delete photos/music/videos they are being changed in the shared folder. There will also be a subfolder in Documents with shared documents.
  7. Add a symbolic link to your home folder for the backup routine: ln -s /mnt/NAS/user NASbackup
  8. Next you need some scripts for backing up data, mounting and unmounting manually and shutting down the NAS server.
  9. For backup I recommend rsync. Note that I don't use --delete option because I'm paranoid so if you delete stuff or move stuff you'll end up with a build up of data on your server unless you take duplicates/deleted data off manually or run rsync with --delete occasionally. This is my script (in $HOME/apps/utility/runSyncRoutine):
#!/bin/bash

zenity --question --text="<span font='16'>Backing up files to NAS.\n\nAre you sure you wish to proceed?" --width=400 --height=150

if [ "$?" = 0 ] ; then
rsync -rvul --progress $HOME/ $HOME/NASbackup/ |
awk -f $HOME/apps/utility/rsync.awk |
zenity --progress --title "Syncing data" \
--text="Scanning..." --percentage=0 --auto-kill
if [ "$?" = 0 ] ; then
msg="Backup completed: "$(date +"%T %d %b %Y")
zenity --notification --window-icon="info" --text="$msg"
exit 0
fi
zenity --error --text="<span font='16'>Backup failed.</span>" --width=400 --height=150
fi


In order to run this you will also need (in $HOME/apps/utility/rsync.awk):
{
if (index($0, "to-check=") > 0)
{
split($0, pieces, "to-check=")
split(pieces[2], term, ")");
split(term[1], division, "/");
print (1-(division[1]/division[2]))*100"%"
}

else
{
print "#"$0;
}
fflush();
}

The danger with running rsync with the --delete option is that if your PC drive starts to fail and you don’t notice and run an rsyunc routine with --delete you run the risk of having your backup routine removing the backup files it needs to restore the data on your PC’s drive!
  1. You need a script to unmount because if you have the server running and are still mounted to folders on the server when you want to shutdown your PC, if you don't first unmount your NAS mount points, your PC will take a long time to shutdown. NB: you must not be using any resources on these mount points or the umount will fail. This is my unmount script:
#!/bin/bash

umount /mnt/NAS/user /mnt/NAS/shared
if [ "$?" = 0 ] ; then
zenity --notification --window-icon="info" --text="NAS Drives unmounted successfully!"
exit 0
fi
zenity --error --text="<span font='16'>Error unmounting NAS drives.</span>" --width=400 --height=150

  1. This is my script to shutdown the NAS server remotely. When you add it as a menu item it needs to run in a terminal so that you can input your ssh password (NB not user or admin password but the password you set when you switched on ssh in the administration Web page):

#!/bin/bash

umount /mnt/NAS/NAS_USB /mnt/NAS/user /mnt/NAS/shared

if [ "$?" = 0 ] ; then
zenity --notification --window-icon="info" --text="NAS Drives unmounted successfully!"
ssh -t root@192.168.0.36 'sudo /usr/local/modules/script/shutdown.sh -h now'
exit 0
fi
zenity --error --text="<span font='16'>Error unmounting NAS drives.</span>" --width=400 --height=150


  1. In order to use mount, umount and ssh without the a password add these lines to your /etc/sudoers file on each Linux workstation (obviously changing the username to the username of the user on that workstation or look up how to use groups etc and change the path to the absolute path to your mount, umount and ssh files if they are different):
matthew ALL = (ALL) /bin/mount
matthew ALL = (ALL) /bin/umount
matthew ALL = (ALL) /usr/bin/ssh

Read up on the way to edit sudoers safely – it’s well documented on the Web and is important.
  1. Repeat steps 7-23 for each workstation except for 12, which you only do once.
  2. Now you have shared access to data as well as backups of data that is on Linux workstations attached to the network. I was concerned about the shared data since there was not a backup of that. The MyCloud NAS server has a USB port available for adding a drive. I bought a cheap USB drive (Western Digtital Elements – it’s not NAS) that matched the capacity of my NAS and plugged it into the USB port. It’s powered by USB and MyCloud NAS recognises it immediately and makes USB available as a share option in the shares section.
  3. I created a mount point folder for this drive (/mnt/NAS/NAS_USB) and added a line in my fstab file (after setting up a USB share and checking it’s mount point with sudo showmount -e 192.168.0.36):
192.168.0.36:/mnt/USB/USB1_b1 /mnt/NAS/NAS_USB nfs rw,udp,users,vers=3,soft,intr,rsize=8192,wsize=8192 0 0

  1. I made a symbolic link to this mount point as I did with the other mounts () and created a script to run backups. Note: MyCloud offers a backup service but I find it unreliable – it failed with an error message like ‘unable to run backup – check your system for errors (!) halfway through a backup). My script:
#!/bin/bash

zenity --question --text="<span font='16'>Backing up NAS to USB.\n\nAre you sure you wish to proceed?</span>" --width=400 --height=150
if [ "$?" = 0 ] ; then
rsync -rvul --progress /mnt/NAS/shared/ $HOME/NASMirror/AllMirror/shared/ |
awk -f $HOME/apps/utility/rsync.awk |
zenity --progress --title "Syncing data" \
--text="Scanning..." --percentage=0 --auto-kill
if [ "$?" = 0 ] ; then
msg="Backup completed: "$(date +"%T %d %b %Y")
zenity --notification --window-icon="info" --text="$msg"
exit 0
fi
zenity --error --text="<span font='16'>Backup failed.</span>" --width=400 --height=150
fi

NB: As I'm sure you know: the scripts won't execute unless the execute permission is authorised - you can use nemo or whatever file browser you are using to set that permission.

I hope that helps. PS: MyCLoud supports web access from an Android app but that's well documented so you should be able to follow their guidance and access your data on your phone.

If anyone knows this technology well and can see simpler, more elegant ways to do all this please let me know. Thx.