TableOfContents

Status and Availability

• Scientific Linux 5 for the i386 platform was released May 7, 2007, with x86_64 following May 15.
• Since May 8, 2007, 32-bit SL5 was available for early adopters at DESY Zeuthen, with 64-bit following May 16.
• A presentation in the technical seminar was given June 5, 2007. (Slides: attachment:SL5Z.odp / attachment:SL5Z.pdf)
• Requests to update systems to both 32-bit and 64-bit SL5 are welcome.

• Today, SL5 is the default and recommended Linux platform at DESY, Zeuthen. It is running on the majority of PCs, the majority of farm worker nodes, the majority of servers, and the Myrinet and InfiniBand Clusters.

• Public Login systems for both platforms are available (see below). Problem reports for both are welcome.
• An SL5 queue on the batch farm (64-bit only) is available (see [:Batch_System_Usage]). It now controls most of of the fastest cores in the farm, with (90% of) 2 GB RAM per core, representing the better part of the farm's total computing power. As of July 4, 2007, batch jobs are scheduled in any free Slot regardless of the operating system on the worker node. Jobs can still request to be scheduled on SL3 (or SL5) nodes only.

Public Preview Systems

Two DNS aliases have been set up:

• sl5.ifh.de (x86)

• sl5-64.ifh.de (x86_64)

They point to (virtual) systems suitable for getting familiar with SL5, finding remaining problems with it to report to uco, and to test or port user software. These systems are not for production use.

The public PCs in 2L01 are running SL5 as well.

Improvements over SL3

SL5 users will benefit directly from these:

• more recent versions of GNOME, KDE, TeX, and lots of other software

• new applications, like scribus

• better support for recent hardware
• better support for hotpluggable storage

• better interactive response during I/O (much better)

• faster startup time for large applications
• does not require more hardware resources than SL3 (except some more space in /)
  • especially on old PCs, SL5 is much more fun to work with than SL3
• one user reports that the new gcc-4.1.1 compiles his code much faster than 3.2.3 on SL3
• improved security (SELinux)

There are also many new features primarily of interest to administrators, like virtualization (Xen), configurable I/O schedulers, ionice(1), improved power management (really important for the farm these days), ...

General Points

AFS Sysname List

The AFS sysname list (the output of the fs sysname command) in Zeuthen is:

Login Shells

We made a serious effort to make bash a supported login shell, but it's impossible.

There is no way to reliably give bash users a working environment identical to that of zsh and tcsh users, due to bash's limited functionality w.r.t. startup files processing.

We recommend using zsh, zsh or zsh as the login shell. Tcsh is available for those who insist. Bash is not, sorry.

Notice this does not prevent users from writing or using bash scripts in any way.

Language Support, UTF8

It was initially planned to introduce use of UTF8 as the default with SL4. Alas, it was found that it's causing too much trouble, and makes things too incompatible to the rest of our environment. This hasn't changed with SL5. Also notice that a lot of software still has bugs when used under a UTF8 locale.

Hence the default for the LANG environment variable on SL5 systems is C, as on SL4, SL3 and DL5 before. Early SL4 systems had a default of en_US, because it makes certain GNOME applications behave more sensible (in particular gnome-terminal), but it was found to cause other problems (among them, changes in the date format and sorting order).

Any user may change the personal default by creating a file ~/.i18n:

# recommended and default:
#LANG=C

# alternative (but see the remarks concerning a2ps below):
#LANG=en_US

# I want to experiment with UTF-8
LANG=en_US.UTF-8
# But I really prefer A4 over Letter
LC_PAPER=de_DE

We do not install language support packages for other languages than US-english, with the sole exception of dictionaries for spellchecking. Languages other than english for the user interface are not supported.

But: Typing and displaying non ASCII characters should work very well under a UTF-8 locale in GNOME and KDE applications. Here's an example of how to achieve this even under the default environment:

export LANG=en_US.UTF-8
export LC_PAPER=de_DE
exec gnome-terminal \
  --hide-menubar --disable-factory --geometry=80x40 --window-with-profile=alpine \
  -x alpine -i

This script will open a new gnome-terminal window and execute the alpine mail client in it. The alpine profile in your gnome-terminal should of course have suitable settings (no scroll bar, maybe a certain font,...).

For KDE users, something like this is probably more suitable:

export LANG=en_US.UTF-8
export LC_PAPER=de_DE
exec konsole \
  --nohist --nomenubar --notabbar --noscrollbar --vt_sz 80x40 \
  -e alpine -i

It should be noted that both gnome-terminal and konsole do provide fairly good UTF-8 support, xterm will handle west european chars well, but does (by default?) not display russian, chinese etc. chars.

When switching to UTF-8 you might need further adjustments to the programs. We know that the following settings are needed:(please report more findings!)

• alpine (in the config screen): Posting Character Set = UTF-8
• less: set the environment variable LESSCHARSET to utf-8
• a2ps: setting the LANG environment variable to en_US.UTF-8 results in the default paper format letter, this is NOT what you want. Setting LC_PAPER=de_DE overrides LANG in this respect and sets the default paper format A4, without other side effects like changes in sorting order, number display, etc.

There are two methods for typing: Compose Character (see below) should work in most applications, and for most "european" characters. In addition, the SCIM (smart common input method) is available in GNOME and KDE applications, and can be used to type other character sets, like cyryllic, chinese, japanese, ...

Removable Media and Hotplug Storage Devices

• Mount points are now in /media. They are created at mount time, not after plugging the device. No fstab entries are used, that's why you cannot use the usual mount and umount commands. The names of the mount points are taken from the file system's volume label if applicable. They are removed when the device is removed.

• Manual Mounting If the automatic mounting of devices by GNOME/KDE is not used, gnome-mount can be called manually like this, using the appropriate device file like /dev/dvd or /dev/cdrom for a DVD and CD-ROM respectively, usually /dev/sda1 for USB disks and memory sticks:

  gnome-mount -d /dev/sda1
  gnome-umount -d /dev/sda1

• Hotplug should work better than on SL3 or SL4, thanks to the more modern kernel and hotplug scripts. It is known to not work perfectly in all cases though.

  • Multiple partitions on USB storage devices should work now, at least if all partitions have a supported filesystem, and the partition table format is legal.

  • The new hotplug system does not assign ownership of the device to the user. Hence you can't partition the device or create filesystems, unlike on SL3.

  • The new hotplug system will clean up properly if a mounted device if removed, and it no longer causes problems to log out while a hotplug device is mounted.

    It is still recommended to always umount all filesystems on a hotplug device before removing it physically, and before logging off from the console.

    Notice the GNOME desktop will automount filesystems on hotplug devices when they are

    connected. This feature can be turned off under System->Preferences->Removable Storage:BRattachment:g-v-p.png

• Notice there is no firewire support

Backward Compatibility

Desktop/User Environment

• As on SL4, the Display Manager is gdm, no longer kdm as on SL3 and before.

  • Because gdm is the default on SL.
  • Gdm will remember your preferred session without the need to

    move some configuration file, but it doesn't know about the rpeference set for kdm, hence the session type has to be chosen at least once.

• HEPiX11 was dropped (already with SL4)

  • fvwm2 is no longer available.

  • Configuration in ~/.hepix is no longer used.

  • The Windows-Key can no longer be used to type german Umlaute (äöÜ...).

  • Instead, the right Alt-Key now works as a Compose Character-Key: To input an ä, type [R-Alt], then ", then a (one after another, not simultaneously). This is slightly less convenient, but much more general: This method works for characters like ç ñ ø ô ë ...

• The old HEPiX profiles were completely replaced

  • The replacement is HEPiX-like where not incompatible with today's defaults on vanilla linux systems.

• KDE, GNOME, IceWM, WindowMaker are available as on SL3.

  • IceWM is the recommended window manager for older desktops with 256 or even 128 MB RAM

    • an enhanced default configuration for IceWM is provided

  • WindowMaker is causing quite a few problems, and slated for removal.

• No special support for running X sessions from Windows Desktops is provided, nopr is xdmcp. We recommend running the following command through an ssh connection forwarding X11 to your Exceed/Xwin32/whatever server:

  /etc/X11/xinit Xsession xterm

Make sure you have a font server configured. One option is fontsrv.ifh.de . If this is not sufficient, you may consider using a linux desktop. Excellent support for running a Windows session from linux desktops is provided by means of the Windows terminal servers (see winrdp below).

Printing

Print Service has been changed from LPRng to [:Printing_with_Cups: Cups] because LPRng is obsolete and Cups is better integrated in KDE, Gnome, OpenOffice and other tools.

Binaries

SL5 should be binary backward compatible with SL4. This means that executables built on and for SL4 should work on SL5. It does not mean that any executable that works on SL4 will work on SL5 as well: If it worked only due to legacy support before, it may no longer work on SL5.

In particular, binaries that depend on setting the environment variable $LD_ASSUME_KERNEL to a value lower than 2.6.9 won't work on SL5.

C++ ABI

There was no major change to the C++ ABI from the default compiler on SL3 to the one on SL5, only a few "ABI fixes". Most binaries built on SL3 should work fine with the shared libaries built on and for SL5. Example: Building the tests for ROOT-5.14 on SL4 and running them on SL5 works fine. One of the executables behaves slightly differently - but in this one, glibc detects a memory handling bug (see below).

In fact, it usually even works to build and run an executable on SL5 against libraries built on and for SL4, and sometimes even SL3.

Missing Shared Libraries

If your executable fails with an error message like this

error while loading shared libraries: libldap.so.2: cannot open shared object file: No such file or directory

this indicates a missing shared library. If such a a shared library is available on SL3/4, we'll try to make it available on SL5 as well, so please report these cases. To find out whether a libray is available on the older systems, log into an SL3/4 system and use rpm:

[sl3] ~ % rpm -q --whatprovides libldap.so.2
openldap-2.0.27-22.i386
[sl4] ~ % rpm -q --whatprovides libldap.so.2
compat-openldap-2.1.30-7.4E.i386

In this case, we created a compatibility package for SL5 from the SL4 one:

[sl5] ~ % rpm -q --whatprovides libldap.so.2 
compat-sl3-openldap-2.1.30-7.4E.i386

This will be possible in almost all cases.

Using Shared Libraries from the SL3 AFS Installation

Many /opt/products packages are installed in AFS space, to make it possible to reference them with a symbolic link at least on systems with smaller disks. In many cases, this allows using this software on SL5 even if it's not installed into /opt/products on this platform. For example, to use the shared libraries from the root64-5.12.00 build from SL3, one can export LD_LIBRARY_PATH=/afs/ifh.de/amd64_rhel30/products/root64/5.12.00/lib64 . The corresponding path for the SL3/32bit installation is afs/ifh.de/i586_rhel30/products/.

Executables failing with *** glibc detected *** error messages

If your executable does not work, but instead fails like this

 ~ % voms-proxy-init
Cannot find file or dir: /afs/ifh.de/user/w/wiesand/.glite/vomses
Your identity: /O=GermanGrid/OU=DESY/CN=Stephan Wiesand
Enter GRID pass phrase:
Creating proxy ............................................................ Done
*** glibc detected *** voms-proxy-init: munmap_chunk(): invalid pointer: 0xbf97bd02 ***
======= Backtrace: =========
/lib/libc.so.6(cfree+0x1bb)[0x47c216db]

and so on, that's a bug in the application's memory management. Starting with SL3, glibc began detecting such bugs and warning about them. Since SL4, processes exhibiting such bugs are terminated by default. With SL5, glibc detects more of these problems, hence this may affect applications that worked on older releases.

As a workaround, you can set the environment variable MALLOC_CHECK_ to 1, to keep glibc from terminating such processes: {{{~ % MALLOC_CHECK_=1 voms-proxy-init malloc: using debugging hooks Cannot find file or dir: /afs/ifh.de/user/w/wiesand/.glite/vomses Your identity: /O=GermanGrid/OU=DESY/CN=Stephan Wiesand Enter GRID pass phrase: Creating proxy ................................................................................. Done *** glibc detected *** voms-proxy-init: free(): invalid pointer: 0xbf900d01 *** Your proxy is valid until Wed May 9 03:49:39 2007 }}}

Notice the 'malloc: using debugging hooks' message before each command. Since this also costs performance, do not add this environment variable to your profile.

Executables failing with "cannot restore segment prot after reloc"

SL5 is the first SL release exposing end users to SELinux - which was present on SL4, but much more permissive by default, except for the targeted daemons. For example, processes normally have permission to either execute a memory location or write to it, but not both (wherever possible and practical, anyway).

This restriction makes life hard for attackers trying to exploit bugs like buffer overflows in the software. It also prevents a few existing applications from being executed. Here's an example for a certain release of GRID User Interface Software:

.../gui-2.3/lcg/bin/lcg-cp: error while loading shared libraries: .../gui-2.3/lcg/lib/libgfal_pthr.so: cannot restore segment prot after reloc: Permission denied

There is some information on SELinux memory protection available in http://people.redhat.com/~drepper/selinux-mem.html , and more about text relocations in http://people.redhat.com/~drepper/textrelocs.html . In short, the author of these documents explains that executables (including shared libraries) exhibiting this error are incorrectly built:

• "A text relocation is the result of a reference to an object with a variable address at runtime using an absolute addressing mode. The instruction encoding itself contains the address and therefore the executable text of the binary must be changed to contain the correct address when taking the actual load addresses at runtime into account.

  The result of a text relocation is that the binary text is written to. This means this page of the binary cannot be physically shared with other processes on the system (this is the goal of DSOs, aka shared libraries). It also means that the binary must have permission to change the access permissions for the memory page to include writing and then back to executing. This is a privileged operation when SELinux is enabled."

The solution is usually to recompile the source with -fPIC .

If you can't do that, the workaround is to apply a security label to the binary which will allow it to perform text relocation: {{{[sl5] % chcon -t textrel_shlib_t .../gui-2.3/lcg/lib/libgfal_pthr.so }}} This only works on the local disk. Not in NFS/AFS/panfs.

Source Code

• GCC4 is much stricter than previous versions, some C and C++ code may need to be adapted.

  • GCC4 hints for fixing the typical new compiler errors: http://dev.gentoo.org/~vanquirius/gcc4-porting-guide.html

• The FORTRAN frontend of GCC4 is gfortran, no longer g77, and some code may need to be adapted

  • the corresponding runtime library is libgfortran, no longer libg2c

Building Software

In general, it is more strictly necessary than before to compile anything that will be used in a shared object with -fPIC. In particular, if one receives this error from the linker: {{{/usr/bin/ld: xyz.o: relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC }}} one should really do as told.

Applications

• ghostscript options sometimes have a different syntax. This may bite you if you have configured special options in frontends like gv in the past. For example, -scale 1 is no longer accepted and this has to be changed to -scale=1.

Software

Browser

Firefox ist the recommended web browser. We provide the java, flash and realplayer plugins.

Mail Client

Pine is the recommended e-mail client. It will however soon be superseded by alpine, and you may want to give that a try instead.

Both pine and alpine do consult /etc/mailcap to display attachments. Some users complained that ps and pdf files are displayed using evince. This can be changed to acroread and gv respectively by appending the two lines

application/pdf; acroread %s
application/postscript; gv %s

to your ~/.mailcap file.

Compilers

GCC

• The default compiler suite is GCC 4.1.1.
• The compatibility release 3.4.6 (default on SL4, available on SL3) is installed as well.

  • Invoke as gcc34, g++34, g77

  g77 is the FORTRAN frontend from the old compiler. The new frontend is gfortran .

Intel

Version 9.1 (or later) of the C, C++, and FORTRAN compilers are available. The 32-bit runtime environment is installed on 64-bit Systems as well.

• Invoke as icc, icpc, ifort

Portland Group

Version 7.0 (or later) of the PGI compiler is installed. The 32-bit runtime system is available on 64-bit hosts.

• Invoke as cc, CC, f77, f90 after ini pgi

Java

Version 1.5 is the default and installed locally. It's 32-bit even on 64-bit systems, to make the browser plugin work.

Versions 1.4.2 and 1.6 are available in /opt/products, as well as a 64-bit Version of 1.5.

ROOT

We provide version 5.14.00 (now patch release f) built with GCC4. Older versions for older compilers could be provided on request. This will stay the default version (with root, root-config etc. in the user's PATH), but more recent versions will be provided as well (as of November 2007: 5.16.00).

As on SL3/4, the 64-bit build is installed in /opt/products/root64, the 32-bit one in /opt/products/root.

Known Problems

evo seems not to work

There are actually two problems involved here:

1. Firefox doesn't know how to deal with JNLP ("Java Web Start") Files. We're working on fixing this. For the time being, please instruct Firefox to open them with /etc/alternatives/javaws. Update: this should be ok since December 2007.

2. Error loading koala.jar . This seems to be a problem on the evo side. As a workaround, please start evo from http://evo.vrvs.org and not https://evo.vrvs.org

vi misbehaves in konsole & gnome-terminal if $TERM=vt100

If the environment variable TERM is set to vt100, some escape sequences produced by vi (or vim) are not processed correctly by konsole and gnome-terminal. Please do not set TERM in your dot files.

ps2pdf displacing graphics

There is one report of ps2pdf (from the gs package) shifting graphics in the document to the wrong posisiton on the page. Using epstopdf (from the tetex package) instead solved this problem.

firefox crashes

There are a number of reports that firefox crashes often. While this software does have bugs, and since the Mosaic aera browsers have always crashed occasionally, there are indeed indications that firefox is unusually unstable on SL5 in the default KDE environment.

Possible workarounds:

• Use GNOME instead of KDE. Firefox is reasonably stable for us in this desktop environment.

• Changing your KDE theme to Plastik or Keramik may also help

Feedback welcome.

firefox doesn't allow changing the helper application for postscript and PDF files

This is not specific to SL5. As of January 17, 2008, this procedure should help:

1. stop firefox

2. rm ~/.mozilla/firefox/*/mimeTypes.rdf

3. restart firefox

Firefox should then get a working copy of this configuration file we now provide. You should be able to choose helper applications for the most relevant file types under Edit -> Preferences -> Downloads -> View & Edit Actions

Software known not to work on SL5

• Grid Software (LCG/Glite)

  • the older the release, the more problems to use it
  • some executables need the MALLOC_CHECK_ workaround and/or relabelling of shared libs as described above
  • as of October 30, 2007, the UI components of the LCG middleware should generally work, although the MALLOC_CHECK_ workaround is still required.

    The workaround is simple and explained in this page. See above

• Mathematica 5.0

  • The graphical user interface is unusable. The command interface and the kernel seem to work, though.

Software that is not yet available, but will be eventually

Software we do not foresee to make available

• xemacs
  • This software is dead and not provided with the distribution. While it still has a lot of fans, it's really time to get used to GNU emacs.
• ICA/Wincenter

  • providing it would be no problem, but there's a very viable alternative: rdesktop

  • in particular, try the winrdp command

    • you'll find /usr1 and your AFS home directory in \\tsclient on the terminal server

    • winrdp is a simple script; create your personal copy and modify it to export other filesystems or change other options

  • in fact, we haven't received any request to provide ICA from anyone who tried rdesktop/winrdp

PC Upgrades

Requirements

• A supported PC (najade class or newer), 128 MB RAM, 20 GB disk

• A root filesystem of at least 6 GB ( 8 GB for 64-bit)

  • example: this is sufficient for a 32-bit installation: {{{[host] ~ % df -H /

Filesystem Size Used Avail Use% Mounted on /dev/hda7 6.3G 4.9G 1.1G 82% / }}}

• If your / is too small, the disk has to be repartitioned. This should only affect a few of the oldest najade class Systems.

What about my data in /usr1/scratch ?

See above: If the old root filesystem is large enough that we don't have to repartition the disk, /usr1 can be preserved, and the user decides whether or not this should be done. Please note that your PC's local disk is a very unsafe place for your data anyway: We haven't ever lost any data in /usr1 during many hundreds of PC upgrades, but it's not uncommon that these disks break during normal daily operation, usually causing 100% data loss. Your group has AFS space on reliable, redundant hardware, with or without backup. Please talk to your group admin and store your valuable data there. All this being said, preserving /usr1 is the normal case.

Procedure

1. The owner or group admin of the PC contacts uco by e-mail with a subject of SL5 upgrade, providing the following information:

   • the PC's hostname
   • whether or not the /usr1 partition should be preserved
   • date of the upgrade
   • whether the PC should be rebooted by DV or by the user
2. DV prepares the upgrade on the required date.

3. The next reboot starts the upgrade installation. This must happen the same day.

Checklist

To prepare the PC for the upgrade, please

• make sure the PC and the monitor are switched on

• unplug all USB devices (except a USB mouse)
• remove all CDs, DVDs, Floppies

Don't Panic

If you watch the installation (there's no need to stay around, and we actually recommend doing something else), some observations may be a bit disturbing, but are completeley harmless:

• If your PC spends several minutes displaying "retrieving installation information" in the early installation phase, with a spinning cursor and the progress bar indicating that it should be finished, that's perfectly normal.

• Lots of cryptic failure messages starting "audit:", in bunches - just ignore them.
• Error messages about removing packages failed because they are not installed anyway. Again, please ignore.
• Depending on the video hardware, the screen may go blank and stay like this for several minutes even if you press a key like Shift or Ctrl. This happens because the video card is confused after probing it.

Only if you see a clear, persistent error message on the screen indicating that installation was aborted and you should press "ok" to reboot, something went really wrong. Otherwise, please be patient. The one and only way to make an installation fail very reliably is to reboot the PC while it's in progress.

Installation Times

Time to upgrade an SL3/4 system to SL5 (begin of reboot to gdm ready for login):

Installing the 64bit flavour takes 10% longer.

Desktop Upgrade Record, Success/Failure Rates

This table covers only the upgrades of existing PCs in production use. New PCs delivered to the user with SL5 don't show up here. Nor do the test installations/upgrades we run frequently (several times a week) to make sure the process works flawlessly.

This table is no longer updated since the last entry in July 2007. We continue updating systems every now and then, and at the time of writing (October 30), we still haven't had any failures.

Fixed Problems