23741
Comment: updated cell layout
|
24591
updated building instructions, added quick and dirty rpm build
|
Deletions are marked like this. | Additions are marked like this. |
Line 31: | Line 31: |
In the ''openafs-1.4.6-osd'' directory: {{{ ./configure --enable-bitmap-later --enable-bos-restricted-mode --enable-bos-new-config --enable-fast-restart --enable-supergroups --enable-largefile-fileserver --enable-transarc-paths --with-krb5-conf --enable-object-storage }}} |
In the openafs-''version'' directory: {{{ mkdir `sys` cd `sys` ../configure --prefix=/usr --libdir=/usr/lib --bindir=/usr/bin --sbindir=/usr/sbin --with-krb5-conf --enable-bitmap-later --enable-bos-restricted-mode --enable-fast-restart --enable-bos-new-config --enable-supergroups --enable-transarc-paths --enable-object-storage --enable-osd-policies }}} == Quick RPM creation == Currently, this works only with the ''openafs.SLx.spec'' file, revision 70. Install any ''openafs-1.4.x.osd.*.src.rpm'' package to retrieve it. On the build machine: {{{ cd /usr/src/packages/SOURCES svn export https://svnsrv.desy.de/desy/openafs-osd/trunk/openafs openafs-1.4.x.osd.rXXX tar cjf openafs-1.4.x.osd.rXXX-src.tar.bz2 rm -r openafs-1.4.x.osd.rXXX vim ../SPECS/openafs.SLx.spec rpmbuild -ba ../SPECS/openafs.SLx.spec }}} * Change 1.4.x to the AFS version being built and rXXX to the revision number of the HEAD in the subversion repository (obtain using `svn info https://...`). * To build packages with policy support, substitute `trunk` in the URL by `branches/policies` and the version to 1.4.x.osd.pol.rXXX. |
Contents
Sub pages:
1. Important notes
On iokaste, /root/.zshenv now contains
PATH=/usr/heimdal/bin:/usr/afs/bin:$PATH
2. Status
Overview of installs ( this is how it's currently supposed to be / under way):
Machine |
service |
version |
iokaste |
fileserver |
manually from svn |
|
ubik servers |
manually from svn |
|
afsd |
manually from svn |
phobos |
fileserver |
manually from svn, for debugging only |
|
afsd |
manually from svn |
ssu12/13, zyklop19/20 |
fileserver |
OpenAFS+OSD RPM from svn (not zyklop20) |
|
rxosd |
OpenAFS+OSD RPM from svn |
|
afsd |
OpenAFS+OSD RPM from svn 1.4.7 or SL5 RPM |
deimos |
ubik servers |
OpenAFS+OSD RPM from svn |
3. Building OpenAFS-OSD
To build openafs, kernel headers are recommended to be installed (so configure can detect the local sysname). On SL5, as root do a
yum install kernel-devel
and you're set (for the time being). If there's still no /usr/src/linux:
ln -s /usr/src/kernels/2.6.18-92.1.10.el5-i686 /usr/src/linux
...or whatever directory the kernel source was installed to.
In the openafs-version directory:
mkdir `sys` cd `sys` ../configure --prefix=/usr --libdir=/usr/lib --bindir=/usr/bin --sbindir=/usr/sbin --with-krb5-conf --enable-bitmap-later --enable-bos-restricted-mode --enable-fast-restart --enable-bos-new-config --enable-supergroups --enable-transarc-paths --enable-object-storage --enable-osd-policies
3.1. Quick RPM creation
Currently, this works only with the openafs.SLx.spec file, revision 70. Install any openafs-1.4.x.osd.*.src.rpm package to retrieve it.
On the build machine:
cd /usr/src/packages/SOURCES svn export https://svnsrv.desy.de/desy/openafs-osd/trunk/openafs openafs-1.4.x.osd.rXXX tar cjf openafs-1.4.x.osd.rXXX-src.tar.bz2 rm -r openafs-1.4.x.osd.rXXX vim ../SPECS/openafs.SLx.spec rpmbuild -ba ../SPECS/openafs.SLx.spec
Change 1.4.x to the AFS version being built and rXXX to the revision number of the HEAD in the subversion repository (obtain using svn info https://...).
To build packages with policy support, substitute trunk in the URL by branches/policies and the version to 1.4.x.osd.pol.rXXX.
4. Enabeling the client
From the source directory copy
src/libafs/MODLOAD-2.6.18-8.1.15.el5-MP/libafs.ko
(where 2.6.18-8.1.15.el5-MP is the kernel version)
to
/lib/modules/2.6.18-8.1.10.el5/kernel/fs/openafs/libafs.ko
depmod
- reboot
launch afsd
5. Enabling OSD
[iokaste] ~ # vos setfields osdtestvol2 -osd 1 [iokaste] ~ # fs protocol -enable RXOSD
6. Issues
6.1. Missing token is not handled well
$ osd volumes io -lun 1 RX xdr error Cannot list volumes Request aborted. $ osd createv io 1 536870924 Createvolume failed: RXOSD_create_part ended with 13 Request aborted.
In general, missing privileges are an issue but there is no "friendly output" describing the cause of the problem. Some examples of unprivileged access attempts:
fs osd |
Error 13 reading length of metadata |
works with r privilege granted to system:anyuser |
osd read |
RX xdr error \\ Cannot read the object \\ rx_EndCall returns: 13 \\ Request aborted. |
requires token |
osd volumes |
RX xdr error \\ Cannot list volumes \\ Request aborted. |
requires token |
osd examine |
RXOSD_examine failed with code 13 \\ Request aborted. |
requires token |
Database queries osd listosds and osd servers seem to work unauthenticated.
7. Subcommands
7.1. fs createstripedfile
7.1.1. creating mirrored files
# fs creates /afs/desytest/osdtest/mirrored-file 1 14 -copies 2
where 1 is the number of stripes and 14 is log_2(stripesize in bytes).
- Note
- Allowed exponents are 12..19, allowing stripe sizes 4kB..512kB.
The above command created an unstriped but mirrored file.
7.1.2. creating striped files
This is how we created the first striped file:
# fs creates /afs/desytest/osdtest/striped_file_1 2 14 done
We stored a kernel image inside:
# cp /boot/vmlinuz-2.6.18-8.1.10.el5 striped_file_1 # mv striped_file_1 vmlinuz-2.6.18-8.1.10.el5 # ll total 7615 -rw-r--r-- 1 bin root 417032 Jan 14 14:54 libafs.ko -rw-r--r-- 1 bin root 5612872 Jan 28 15:31 mirrored._usr_afs.2007-12-17.tar.bz2 -rw-r--r-- 1 bin root 1765652 Feb 14 13:37 vmlinuz-2.6.18-8.1.10.el5 # fs osd vmlinuz-2.6.18-8.1.10.el5 vmlinuz-2.6.18-8.1.10.el5 has 172 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=2, strsize=16384, cop=1, 2 objects object: pid=536870924, oid=720578573194231814, osd=12, stripe=0 obj=536870924.6.167772773.0 object: pid=536870924, oid=3026421582407925766, osd=11, stripe=1 obj=536870924.6.704643685.0
As the stripes were put in OSDs 11 and 12 respectively:
# osd listo id name(loc) ---total space--- flag prior. own. server lun size range 1 local_disk wr rd (0kb-1mb) 8 io_a 279 gb 0.0 % up hsm 50 0 io.ifh.d 0 (1mb-64gb) 9 io_b 279 gb 0.0 % up hsm 50 0 io.ifh.d 1 (1mb-64gb) 10 io_c 279 gb 0.0 % up hsm 50 0 io.ifh.d 2 (1mb-64gb) 11 io_d 279 gb 0.0 % up hsm 50 0 io.ifh.d 3 (1mb-64gb) 12 io_e 278 gb 0.0 % up hsm 50 0 io.ifh.d 4 (1mb-64gb) # osd vol io 1 volumes found: 536870924 # osd obj io 536870924 3 536870924.4.33555043.0 fid 536870924.4.611 tag 0 not-striped lng 4209654 lc 1 536870924.6.704643685.0 fid 536870924.6.613 tag 0 1/2/16384 lng 880916 lc 1 536870924.4.704643683.1 fid 536870924.4.611 tag 1 1/2/16384 lng 0 lc 1 3 object(s) with totally 5090570 bytes for volume 536870924 found # osd obj io 536870924 4 536870924.4.167772771.0 fid 536870924.4.611 tag 0 0/2/16384 lng 0 lc 1 536870924.6.167772773.0 fid 536870924.6.613 tag 0 0/2/16384 lng 884736 lc 1 2 object(s) with totally 884736 bytes for volume 536870924 found
The first file on io/3 (vicepd) belongs to the mirrored file created above, not the striped file.
7.1.3. adding segments to an existing OSD file
# fs creates /afs/desytest/osdtest/mirrored._usr_afs.2007-12-17.tar.bz2 2 14 File /afs/desytest/osdtest/mirrored._usr_afs.2007-12-17.tar.bz2 already exists. Create new segment? (Y|N) y done # fs osd mirrored._usr_afs.2007-12-17.tar.bz2 mirrored._usr_afs.2007-12-17.tar.bz2 has 436 bytes of osd metadata, v=3 On-line, 3 segm, flags=0x0 segment: lng=5612872, offs=0, stripes=1, strsize=16384, cop=2, 2 objects object: pid=536870924, oid=144117812300873732, osd=10, stripe=0 obj=536870924.4.33555043.0 object: pid=536870924, oid=144117812300873732, osd=11, stripe=0 obj=536870924.4.33555043.0 segment: lng=5612872, offs=5612872, stripes=2, strsize=16384, cop=1, 2 objects object: pid=536870924, oid=720578564604297220, osd=12, stripe=0 obj=536870924.4.167772771.0 object: pid=536870924, oid=3026421573885100036, osd=11, stripe=1 obj=536870924.4.704643683.1 segment: lng=0, offs=5612872, stripes=2, strsize=16384, cop=1, 2 objects object: pid=536870924, oid=720578564671406084, osd=12, stripe=0 obj=536870924.4.167772771.1 object: pid=536870924, oid=3026421573952208900, osd=11, stripe=1 obj=536870924.4.704643683.2
7.2. fs osd
# fs osd /afs/desytest/osdtest/mirrored-file /afs/desytest/osdtest/mirrored-file has 172 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=16384, cop=2, 2 objects object: pid=536870924, oid=144117812300873732, osd=10, stripe=0 obj=536870924.4.33555043.0 object: pid=536870924, oid=144117812300873732, osd=11, stripe=0 obj=536870924.4.33555043.0
where osd=10 and osd=11 identify the storing OSD of each file copy. IDs are looked up using osd listo.
7.3. osd wipecandidates
# osd wipe io # osd wipe iokaste [ some waiting time ] RXOSD_wipe_candidates failed with code -1 Request aborted. # osd wipe /afs/desytest/osdtest/mirrored._usr_afs.2007-12-17.tar.bz2 (no error output) # osd wipe io 0 # osd wipe io 1 # osd wipe io 2 0 Jan 24 16:03 4209654 536870924.4.33555043.0 536870924.4.33555043 # osd wipe io 3 0 Jan 28 15:31 4209654 536870924.4.33555043.0 536870924.4.33555043 # osd wipe io 4 RXOSD_wipe_candidates failed with code 5 Request aborted.
- Note
- The last request propably failed because there was no /vicepe/AFSIDat/ directory on io at the time.
- Note
Using a filename as the only argument to osd wipe should have at least yielded an error similar to when using iokaste (which is not an OSD server).
- Note
- The entries for io/2 and io/3 are the two copies of the file created above.
7.4. fs wipe
# fs wipe mirrored._usr_afs.2007-12-17.tar.bz2 Could not wipe mirrored._usr_afs.2007-12-17.tar.bz2, error code was 22
- Note
This appears to be the default answer to a tried wipe of a file that has not yet been archived using fs archive.
7.5. osd createvolume
[iokaste] ~ # vos create iokaste a osdtestvol2 Volume 536870927 created on partition /vicepa of iokaste [iokaste] ~ # osd createvol io 536870927 0 Createvolume: volume 536870927 successfully created. [iokaste] ~ # osd volu io 0 2 volumes found: 536870924 536870927 [iokaste] ~ # osd volu io 1 1 volumes found: 536870924
7.6. osd volumes
# osd volumes io 1 volumes found: 536870924
That's the same VolID as in AFS:
# vos listvl VLDB entries for all servers osdtestvol RWrite: 536870924 number of sites -> 1 server iokaste.ifh.de partition /vicepa RW Site root.cell RWrite: 536870921 number of sites -> 1 server iokaste.ifh.de partition /vicepa RW Site Total entries: 2
- Note
This information seems to be persistent, even after removing all of a volume's files from Object Storage:
[iokaste] /afs/desytest/osdtest # osd volu io 0 1 volumes found: 536870924 [iokaste] /afs/desytest/osdtest # osd volu io 1 1 volumes found: 536870924 [iokaste] /afs/desytest/osdtest # osd volu io 2 1 volumes found: 536870924 [iokaste] /afs/desytest/osdtest # osd volu io 3 1 volumes found: 536870924 [iokaste] /afs/desytest/osdtest # osd volu io 4 1 volumes found: 536870924 [iokaste] /afs/desytest/osdtest # osd volu io 5 0 volumes found: [iokaste] /afs/desytest/osdtest # osd obj io 536870924 0 0 object(s) with totally 0 bytes for volume 536870924 found [iokaste] /afs/desytest/osdtest # osd obj io 536870924 1 0 object(s) with totally 0 bytes for volume 536870924 found [iokaste] /afs/desytest/osdtest # osd obj io 536870924 2 0 object(s) with totally 0 bytes for volume 536870924 found [iokaste] /afs/desytest/osdtest # osd obj io 536870924 0 object(s) with totally 0 bytes for volume 536870924 found [iokaste] /afs/desytest/osdtest # osd obj io 536870924 3 0 object(s) with totally 0 bytes for volume 536870924 found [iokaste] /afs/desytest/osdtest # osd obj io 536870924 4 0 object(s) with totally 0 bytes for volume 536870924 found
7.7. osd objects
Both volume and partition are needed:
# osd obj io 536870924 0 object(s) with totally 0 bytes for volume 536870924 found # osd obj io 536870924 2 536870924.4.33555043.0 fid 536870924.4.611 tag 0 not-striped lng 4209654 lc 1 1 object(s) with totally 4209654 bytes for volume 536870924 found
- Note
- Without specifying a lun, the default of 0 (vicepa) is used.
7.8. osd examine
This command also has issues with missing LUNs:
[iokaste] /afs/desytest/osdtest # osd exa io 536870924.6.621.0 RXOSD_examine failed with code 5 Request aborted. [iokaste] /afs/desytest/osdtest # osd exa io 536870924.6.621.0 -lun 4 536870924.6.621.0 fid 536870924.6.621 tag 0 not-striped lng 44537893 lc 1 Feb 21 13:53
7.9. osd read
This works for unstriped files, but suffers from the "missing LUN syndrome" as well:
[iokaste] /afs/desytest/osdtest # osd read io 536870924.6.621 RX xdr error Cannot read the object rx_EndCall returns: 5 Request aborted. [iokaste] /afs/desytest/osdtest # osd read io 536870924.6.621 0 512 -lun 4 sscanf failed at offset (0x0, 0x0) [1177711200] LOG ROTATION: DAILY [1177711200] LOG VERSION: 2.0 [1177711200] CURRENT HOST STATE: a;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.13 ms [1177711200] CURRENT HOST STATE: amafs;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.14 ms [1177711200] CURRENT HOST STATE: aphrodite;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.54 ms [1177711200] CURRENT HOST STATE: aquila;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.25 ms [1177711200] CURRENT HOST STATE: ares;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0. reading of 512 bytes took 0.003 sec. Total data rate = 180 Kbytes/sec. for read.
- Note
0 and 512 in example 2 are offset and bytes to read, resp.
7.10. fs replaceosd
First tries were very discouraging:
[iokaste] ~ # fs osd /afs/desytest/osdtest/nagios-04-29-2007-00.log /afs/desytest/osdtest/nagios-04-29-2007-00.log has 132 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: pid=536870924, oid=2667174690822, osd=12, stripe=0 obj=536870924.6.621.0 [iokaste] ~ # fs rep /afs/desytest/osdtest/nagios-04-29-2007-00.log 12 8 failed to replace osd 12 for /afs/desytest/osdtest/nagios-04-29-2007-00.log, error code is 5 [iokaste] ~ # fs rep /afs/desytest/osdtest/nagios-04-29-2007-00.log 12
The last command hung and could not be TERMed or KILLed. A subsequent
[iokaste] ~ # fs osd /afs/desytest/osdtest/nagios-04-29-2007-00.log
got stuck much the same.
8. Case study: Creating a long striped file
[iokaste] /afs/desytest/osdtest # fs creates striped-ascii-44m 2 12 done [iokaste] /afs/desytest/osdtest # cp /root/nagios-04-29-2007-00.log striped-ascii-44m [ took about 3 seconds ] [iokaste] /afs/desytest/osdtest # fs osd striped-ascii-44m striped-ascii-44m has 172 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=2, strsize=4096, cop=1, 2 objects object: pid=536870924, oid=576463393708310532, osd=12, stripe=0 obj=536870924.4.134218343.0 object: pid=536870924, oid=2882306402922004484, osd=11, stripe=1 obj=536870924.4.671089255.0 [iokaste] /afs/desytest/osdtest # osd obj io 536870924 3 536870924.4.671089255.0 fid 536870924.4.615 tag 0 1/2/4096 lng 22267941 lc 1 1 object(s) with totally 22267941 bytes for volume 536870924 found [iokaste] /afs/desytest/osdtest # osd obj io 536870924 4 536870924.4.134218343.0 fid 536870924.4.615 tag 0 0/2/4096 lng 22269952 lc 1 1 object(s) with totally 22269952 bytes for volume 536870924 found
- Note
The lengths (lng) of both objects amount to the size of the stored file: 22267941 + 22269952 = 44537893.
9. Archiving
9.1. Manually creating archival copies
The most simple way is to tell the fileserver to create a file copy in an archival osd:
# fs archive ascii-file ascii-file done
An alternative to this is the fs fidarchive subcommand.
It is also possible to specify the desired archive:
# osd l id name(loc) ---total space--- flag prior. own. server lun size range 1 local_disk wr rd (0kb-1mb) 13 ssu12 1781 gb 0.0 % up 55 64 ssu12.if 2 (1mb-64gb) 14 ssu13 1781 gb 0.0 % up 55 64 ssu13.if 2 (1mb-64gb) 15 z19mini 91 gb 2.3 % up hsm 64 64 zyklop19 0 (1mb-1gb) 16 z19maxi 458 gb 0.0 % up 55 64 zyklop19 2 (1mb-64gb) 17 arch1 458 gb 0.0 % up arch 55 64 zyklop20 0 (0kb-2tb) 18 arch2 458 gb 0.0 % up arch 55 64 zyklop20 1 (0kb-2tb) 19 z20online 391 gb 0.0 % up 55 64 zyklop20 2 (1mb-64gb) # fs archive ascii-file -osd 18 ascii-file done # fs osd archive-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=3, 2009-01-28 09:55:34, 1 segm, flags=0x0 segment: lng=44537893, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=18, stripe=0 metadata: md5=a64535612df9ff989b4bd6d6a133fb5d as from 2009-01-28 09:56:36
The archival copy is on osd 18 (zyklop20, vicepb = lun 1).
There is no warning about existent copies. Repeat the command:
[iokaste] /afs/desytest/field_test # fs arch ascii-file -osd 18 ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=3, 2009-01-28 09:55:34, 1 segm, flags=0x0 segment: lng=44537893, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=18, stripe=0 metadata: md5=a64535612df9ff989b4bd6d6a133fb5d as from 2009-01-28 09:56:36
The call is silently ignored.
Change the data:
[iokaste] /afs/desytest/field_test # echo More data >> ascii-file [iokaste] /afs/desytest/field_test # fs arch ascii-file -osd 18 ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=5, 2009-01-28 09:59:27, 1 segm, flags=0x0 segment: lng=44537903, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.1, osd=18, stripe=0 metadata: md5=1891d17de5eb67c0bd30d54952ea0325 as from 2009-01-28 09:59:43
The copy is updated.
Another update:
[iokaste] /afs/desytest/field_test # echo More data yet >>| ascii-file [iokaste] /afs/desytest/field_test # fs arch ascii-file ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=7, 2009-01-28 10:00:46, 1 segm, flags=0x0 segment: lng=44537917, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=18, stripe=0 metadata: md5=eec1410b6ae326e6c9173802122372c0 as from 2009-01-28 10:00:51
Even without specifying the target osd, the system correctly assumes that the archive should be refreshed.
Without updates:
[iokaste] /afs/desytest/field_test # fs arch ascii-file ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 436 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=7, 2009-01-28 10:00:46, 1 segm, flags=0x0 segment: lng=44537917, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=18, stripe=0 metadata: md5=eec1410b6ae326e6c9173802122372c0 as from 2009-01-28 10:00:51 Archive, dv=7, 2009-01-28 10:00:51, 1 segm, flags=0x0 segment: lng=44537917, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=17, stripe=0 metadata: md5=eec1410b6ae326e6c9173802122372c0 as from 2009-01-28 10:02:07
Another archived copy was created, same data (as per md5data).
A last update:
[iokaste] /afs/desytest/field_test # echo And even more data >> ascii-file [iokaste] /afs/desytest/field_test # fs arch ascii-file ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=4096, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=9, 2009-01-28 10:02:54, 1 segm, flags=0x0 segment: lng=44537936, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.1, osd=17, stripe=0 metadata: md5=150103c2d7bcfc367b707df35801cdfa as from 2009-01-28 10:02:59
The copy in osd 17 is updated, the "old" archive in osd 18 is discarded.
9.2. Wiping files
If the data is archived, it can be wiped from its actual OSD:
[iokaste] /afs/desytest/field_test # fs wipe ascii-file done [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 160 bytes of osd metadata, v=3 Archive, dv=9, 2009-01-28 10:02:54, 1 segm, flags=0x2 segment: lng=44537936, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.1, osd=17, stripe=0 metadata: md5=150103c2d7bcfc367b707df35801cdfa as from 2009-01-28 10:02:59
On an access, it is silently restored to an "normal" osd:
[iokaste] /afs/desytest/field_test # head ascii-file [1177711200] LOG ROTATION: DAILY [1177711200] LOG VERSION: 2.0 [1177711200] CURRENT HOST STATE: a;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.13 ms [1177711200] CURRENT HOST STATE: amafs;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.14 ms [1177711200] CURRENT HOST STATE: aphrodite;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.54 ms [1177711200] CURRENT HOST STATE: aquila;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.25 ms [1177711200] CURRENT HOST STATE: ares;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.59 ms [1177711200] CURRENT HOST STATE: argus;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.16 ms [1177711200] CURRENT HOST STATE: artax;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 1.08 ms [1177711200] CURRENT HOST STATE: arwen;UP;HARD;1;PING OK - Packet loss = 0%, RTA = 0.18 ms [iokaste] /afs/desytest/field_test # fs osd ascii-file ascii-file has 284 bytes of osd metadata, v=3 On-line, 1 segm, flags=0x0 segment: lng=0, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.0, osd=15, stripe=0 Archive, dv=9, 2009-01-28 10:02:54, 1 fetches, last: 2009-01-28, 1 segm, flags=0x2 segment: lng=44537936, offs=0, stripes=1, strsize=0, cop=1, 1 objects object: obj=536871001.6.207.1, osd=17, stripe=0 metadata: md5=150103c2d7bcfc367b707df35801cdfa as from 2009-01-28 10:02:59
The osd is selected the same way as always - depending on priorities, usage percentage, owner etc.
9.3. Automatic archiving
The archiver script is included in the OpenAFS+OSD sources. It runs infinitely and searches unarchived files in short intervals. It runs fs fidarchive on them.
The wiper script will traverse OSDs that have the "hsm" flag set using
# osd setosd <id> -wipeable
On those, files that have not been accessed for a while are wiped, so that only their latest archival copy remains in storage.
The scripts have yet to be ported to our specific requirements (as far as they differ from Garching's).