Backups, esta vez de verdad

Svet Ivantchev, eFaber

UniEE, 27 de marzo de 2007

Las copias de seguridad

• Necesidad

• Ya, pero es un rollo

• Los dos tipos de personas

¿Cómo nos quedará?

X X

¿Cómo nos quedará?

¿Cómo nos quedará?

¿Cómo nos quedará?

• cd /backup/snapshots/

• ls -l

drwxr-xr-x 14 svet admin 476 Mar 20 23:44 snapshot-2007-03-20drwxr-xr-x 14 svet admin 476 Mar 21 23:44 snapshot-2007-03-21drwxr-xr-x 14 svet admin 476 Mar 22 23:44 snapshot-2007-03-22drwxr-xr-x 14 svet admin 476 Mar 23 23:44 snapshot-2007-03-23drwxr-xr-x 14 svet admin 476 Mar 24 23:44 snapshot-2007-03-24drwxr-xr-x 14 svet admin 476 Mar 25 23:44 snapshot-2007-03-25

Foto

¿Cuanto ocupa?

• $ cd /backup/snapshots/$ du -s -h snapshot-2007-03-20470M snapshot-2007-03-20

$ du -s -h snapshot-2007-03-23688M snapshot-2007-03-23

$ du -s -h snapshot-2007-03-25689M snapshot-2007-03-25

$ du -s -h753M .

• 470M + 688M + 689M + ⋯ = 753M ???

¿Cuanto ocupa?

• $ cd /backup/snapshots/$ du -s -h snapshot-2007-03-20470M snapshot-2007-03-20

$ du -s -h snapshot-2007-03-23688M snapshot-2007-03-23

$ du -s -h snapshot-2007-03-25689M snapshot-2007-03-25

$ du -s -h753M .

• 470M + 688M + 689M + ⋯ = 753M ???

Nota: la programación necesaria son dos lineas (2 de verdad)

Componentes

• Software:

• rsync

• cp y hardlinks

• (opcionalmente) cron

• Hardware (opcional)

• Un NSLU2 (+ Unslung firmware)

• Disco duro externo USB2

rsync• $ rsync -ae ssh mi_portatil:/home/datos_importantes /backup/2007-03-27

Día 1

Día 2

Día 3

Evolución

No todo cambia

Nuevo

Nuevo

Nuevo

Enlaces simbólicos

• $ ls -ltotal 85052-rw-r--r-- 1 svet svet 86999860 2007-03-26 22:27 fichero1.dat

$ ln -s fichero1.dat fichero2.dat $ ls -ltotal 85052-rw-r--r-- 1 svet svet 86999860 2007-03-26 22:27 fichero1.datlrwxrwxrwx 1 svet svet 12 2007-03-26 22:27 fichero2.dat -> fichero1.dat

$ du -s 85056 .

Enlaces (hardlinks)• $ ls -l

total 85052-rw-r--r-- 1 svet svet 86999860 2007-03-26 22:27 fichero1.dat

$ ln fichero1.dat fichero2.dat $ ls -ltotal 170104-rw-r--r-- 2 svet svet 86999860 2007-03-26 22:27 fichero1.dat-rw-r--r-- 2 svet svet 86999860 2007-03-26 22:27 fichero2.dat

$ du -s 85056 .

$ ls -i fichero1.dat 3908701 fichero1.dat

$ ls -i fichero2.dat 3908701 fichero2.dat

cp -al origen destino

Todo junto

• rsync -v --delete -a servidor:/mis_datos /backup

cp -al /backup/mis_datos /snapshots/snapshot-`date -I`

X X

Harwdare

Cisco/Linksys NSLU2:

Intel XScale IXP420, 266 MHz,8MB flash, 32MB SDRAM,

Ethernet,2 puertos USB 2

Unslung• $ ssh -l root 192.168.1.78

Welcome to Unslung V2.3R63-uNSLUng-6.8-beta

---------- NOTE: THIS SYSTEM IS CURRENTLY UNSLUNG ----------

BusyBox v0.60.4 (2005.03.22-06:52+0000) Built-in shell (ash)Enter 'help' for a list of built-in commands.

# df

Filesystem 1k-blocks Used Available Use% Mounted onrootfs 240125792 75010680 162675564 32% //dev/sdb1 6528 6332 196 97% /initrd/dev/sdb1 240125792 75010680 162675564 32% //dev/sda1 307426936 223085068 81218592 73% /share/flash/data/dev/sdb1 240125792 75010680 162675564 32% /share/hdd/data/dev/sdb2 116661 4155 111302 4% /share/hdd/conf/dev/sdb2 116661 4155 111302 4% /share/flash/conf

Unslung: ipkg

• # ipkg list

abook - 0.5.6-1 - Abook is a text-based addressbook program designed ...adduser - 1.1.3-9 - a multi-call binary for login and user account administrationadns - 1.3-2 - Asynchronous resolver library and DNS resolver utilities.alac-decoder - 0.1.0-2 - A decoder for the apple lossless file format...asterisk - 1.2.14-3 - Open Source VoIP PBX System...rsync - 2.6.9-2 - fast remote file copy program (like rcp)...zoo - 2.10-1 - zoo - File archiving utility with compressionzsh - 4.2.6-3 - Zsh is a shell designed for interactive use.

Unslung: ipkg

• # ipkg info rsync

Package: rsyncVersion: 2.6.9-2Status: install user installedSection: netArchitecture: armebmaintainer: NSLU2 Linux <nslu2-linux@yahoogroups.com>MD5Sum: 4a7310cefe3d43ab5e6bd49577cdd057Size: 200677Filename: rsync_2.6.9-2_armeb.ipkSource: http://www.samba.org/ftp/rsync/rsync-2.6.9.tar.gzDescription: fast remote file copy program (like rcp)

# ipkg install rsync# ipkg install bash

Backups again

Backups again

Backups again

Backups again

Extra: Amazon S3

• import flickrimport BitBucketimport urllibme = flickr.people_findByUsername("svet")bucket = BitBucket.BitBucket("svet-flickr")page = 1total_f = found_f = 0while 1: try: photos = flickr.people_getPublicPhotos(me.id, 1000, page) for photo in photos: total_f = total_f + 1 if bucket.has_key("%s-%s" % (photo.title, photo.id)): pass # ok, ya la tenemos else: data = urllib.urlretrieve("http://static.flickr.com/%s/%s_%s_o.jpg" % \ (photo.server, photo.id, photo.secret), "flickr.jpg") bits = BitBucket.Bits(filename="flickr.jpg") bucket["%s-%s" % (photo.title, photo.id)] = bits print "saving %s" % photo.title found_f = found_f + 1 page = page + 1 except AttributeError: break # bucket vacíoprint "Encontradas %s fotos, guardadas %s nuevas" % (total_f, found_f)

Ejemplo de uso

Refrencias

• http://www.nslu2-linux.org/

• http://samba.anu.edu.au/rsync/

• http://aws.amazon.com

• http://www.mikerubel.org/computers/rsync_snapshots/

• http://www.postneo.com/2006/03/22/