Archiving
Manually creating archival copies
The most simple way is to tell the fileserver to create a file copy in an archival osd:
# fs archive ascii-file ascii-file done
An alternative to this is the fs fidarchive subcommand.
It is also possible to specify the desired archive:
# osd l
id name(loc) ---total space--- flag prior. own. server lun size range
1 local_disk wr rd (0kb-1mb)
13 ssu12 1781 gb 0.0 % up 55 64 ssu12.if 2 (1mb-64gb)
14 ssu13 1781 gb 0.0 % up 55 64 ssu13.if 2 (1mb-64gb)
15 z19mini 91 gb 2.3 % up hsm 64 64 zyklop19 0 (1mb-1gb)
16 z19maxi 458 gb 0.0 % up 55 64 zyklop19 2 (1mb-64gb)
17 arch1 458 gb 0.0 % up arch 55 64 zyklop20 0 (0kb-2tb)
18 arch2 458 gb 0.0 % up arch 55 64 zyklop20 1 (0kb-2tb)
19 z20online 391 gb 0.0 % up 55 64 zyklop20 2 (1mb-64gb)
# fs archive ascii-file -osd 18
ascii-file done
# fs osd archive-file
ascii-file has 284 bytes of osd metadata, v=3
On-line, 1 segm, flags=0x0
segment:
lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=15, stripe=0
Archive, dv=3, 2009-01-28 09:55:34, 1 segm, flags=0x0
segment:
lng=44537893, offs=0, stripes=1, strsize=0, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=18, stripe=0
metadata:
md5=a64535612df9ff989b4bd6d6a133fb5d as from 2009-01-28 09:56:36The archival copy is on osd 18 (zyklop20, vicepb = lun 1).
![/!\](/moin_static1911/sinorca4moin/img/alert.png "/!\"){kind=small}
There is no warning about existent copies. Repeat the command:
[iokaste] /afs/desytest/field_test # fs arch ascii-file -osd 18
ascii-file done
[iokaste] /afs/desytest/field_test # fs osd ascii-file
ascii-file has 284 bytes of osd metadata, v=3
On-line, 1 segm, flags=0x0
segment:
lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=15, stripe=0
Archive, dv=3, 2009-01-28 09:55:34, 1 segm, flags=0x0
segment:
lng=44537893, offs=0, stripes=1, strsize=0, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=18, stripe=0
metadata:
md5=a64535612df9ff989b4bd6d6a133fb5d as from 2009-01-28 09:56:36The call is silently ignored.
Change the data:
[iokaste] /afs/desytest/field_test # echo More data >> ascii-file
[iokaste] /afs/desytest/field_test # fs arch ascii-file -osd 18
ascii-file done
[iokaste] /afs/desytest/field_test # fs osd ascii-file
ascii-file has 284 bytes of osd metadata, v=3
On-line, 1 segm, flags=0x0
segment:
lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=15, stripe=0
Archive, dv=5, 2009-01-28 09:59:27, 1 segm, flags=0x0
segment:
lng=44537903, offs=0, stripes=1, strsize=0, cop=1, 1 objects
object:
obj=536871001.6.207.1, osd=18, stripe=0
metadata:
md5=1891d17de5eb67c0bd30d54952ea0325 as from 2009-01-28 09:59:43The copy is updated.
Another update:
[iokaste] /afs/desytest/field_test # echo More data yet >>| ascii-file
[iokaste] /afs/desytest/field_test # fs arch ascii-file
ascii-file done
[iokaste] /afs/desytest/field_test # fs osd ascii-file
ascii-file has 284 bytes of osd metadata, v=3
On-line, 1 segm, flags=0x0
segment:
lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=15, stripe=0
Archive, dv=7, 2009-01-28 10:00:46, 1 segm, flags=0x0
segment:
lng=44537917, offs=0, stripes=1, strsize=0, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=18, stripe=0
metadata:
md5=eec1410b6ae326e6c9173802122372c0 as from 2009-01-28 10:00:51Even without specifying the target osd, the system correctly assumes that the archive should be refreshed.
Without updates:
[iokaste] /afs/desytest/field_test # fs arch ascii-file
ascii-file done
[iokaste] /afs/desytest/field_test # fs osd ascii-file
ascii-file has 436 bytes of osd metadata, v=3
On-line, 1 segm, flags=0x0
segment:
lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=15, stripe=0
Archive, dv=7, 2009-01-28 10:00:46, 1 segm, flags=0x0
segment:
lng=44537917, offs=0, stripes=1, strsize=0, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=18, stripe=0
metadata:
md5=eec1410b6ae326e6c9173802122372c0 as from 2009-01-28 10:00:51
Archive, dv=7, 2009-01-28 10:00:51, 1 segm, flags=0x0
segment:
lng=44537917, offs=0, stripes=1, strsize=0, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=17, stripe=0
metadata:
md5=eec1410b6ae326e6c9173802122372c0 as from 2009-01-28 10:02:07Another archived copy was created, same data (as per md5data).
A last update:
[iokaste] /afs/desytest/field_test # echo And even more data >> ascii-file
[iokaste] /afs/desytest/field_test # fs arch ascii-file
ascii-file done
[iokaste] /afs/desytest/field_test # fs osd ascii-file
ascii-file has 284 bytes of osd metadata, v=3
On-line, 1 segm, flags=0x0
segment:
lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=15, stripe=0
Archive, dv=9, 2009-01-28 10:02:54, 1 segm, flags=0x0
segment:
lng=44537936, offs=0, stripes=1, strsize=0, cop=1, 1 objects
object:
obj=536871001.6.207.1, osd=17, stripe=0
metadata:
md5=150103c2d7bcfc367b707df35801cdfa as from 2009-01-28 10:02:59The copy in osd 17 is updated, the "old" archive in osd 18 is discarded.
Wiping files
If the data is archived, it can be wiped from its actual OSD:
[iokaste] /afs/desytest/field_test # fs wipe ascii-file
done
[iokaste] /afs/desytest/field_test # fs osd ascii-file
ascii-file has 160 bytes of osd metadata, v=3
Archive, dv=9, 2009-01-28 10:02:54, 1 segm, flags=0x2
segment:
lng=44537936, offs=0, stripes=1, strsize=0, cop=1, 1 objects
object:
obj=536871001.6.207.1, osd=17, stripe=0
metadata:
md5=150103c2d7bcfc367b707df35801cdfa as from 2009-01-28 10:02:59On an access, it is silently restored to an "normal" osd:
[iokaste] /afs/desytest/field_test # head ascii-file
[1177711200] LOG ROTATION: DAILY
[1177711200] LOG VERSION: 2.0
[1177711200] CURRENT HOST STATE: a;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0
.13 ms
[1177711200] CURRENT HOST STATE: amafs;UP;HARD;1;PING OK - Packet loss = 0%, RTA
= 0.14 ms
[1177711200] CURRENT HOST STATE: aphrodite;UP;HARD;1;PING OK - Packet loss = 0%,
RTA = 0.54 ms
[1177711200] CURRENT HOST STATE: aquila;UP;HARD;1;PING OK - Packet loss = 0%, RT
A = 0.25 ms
[1177711200] CURRENT HOST STATE: ares;UP;HARD;1;PING OK - Packet loss = 0%, RTA
= 0.59 ms
[1177711200] CURRENT HOST STATE: argus;UP;HARD;1;PING OK - Packet loss = 0%, RTA
= 0.16 ms
[1177711200] CURRENT HOST STATE: artax;UP;HARD;1;PING OK - Packet loss = 0%, RTA
= 1.08 ms
[1177711200] CURRENT HOST STATE: arwen;UP;HARD;1;PING OK - Packet loss = 0%, RTA
= 0.18 ms
[iokaste] /afs/desytest/field_test # fs osd ascii-file
ascii-file has 284 bytes of osd metadata, v=3
On-line, 1 segm, flags=0x0
segment:
lng=0, offs=0, stripes=1, strsize=0, cop=1, 1 objects
object:
obj=536871001.6.207.0, osd=15, stripe=0
Archive, dv=9, 2009-01-28 10:02:54, 1 fetches, last: 2009-01-28, 1 segm, flags=0
x2
segment:
lng=44537936, offs=0, stripes=1, strsize=0, cop=1, 1 objects
object:
obj=536871001.6.207.1, osd=17, stripe=0
metadata:
md5=150103c2d7bcfc367b707df35801cdfa as from 2009-01-28 10:02:59The osd is selected the same way as always - depending on priorities, usage perc entage, owner etc.
Automatic archiving
The archiver script is included in the OpenAFS+OSD sources. It runs infinitely and searches unarchived files in short intervals. It runs fs fidarchive on them.
The wiper script will traverse OSDs that have the "hsm" flag set using
# osd setosd <id> -wipeable
On those, files that have not been accessed for a while are wiped, so that only their latest archival copy remains in storage.
The scripts have yet to be ported to our specific requirements (as far as they differ from Garching's).