The software that has been packaged for Debian is available in one of several directory trees on each Debian mirror site through FTP or HTTP.

The following directories can be found on each Debian mirror site under the debian directory:

dists/:

This directory contains the "distributions", and this used to be the canonical way to access the currently available packages in Debian releases and pre-releases. Some old packages, the Contents-*.gz and Packages.gz files are still in here.

pool/:

The new physical location for all packages of Debian releases and pre-releases.

tools/:

DOS utilities for creating boot disks, partitioning your disk drive, compressing/decompressing files, and booting Linux.

doc/:

The basic Debian documentation, such as the FAQ, the bug reporting system instructions, etc.

indices/:

The Maintainers file and the override files.

project/:

mostly developer-only materials, such as:

project/experimental/:

This directory contains packages and tools which are still being developed, and are still in the alpha testing stage. Users shouldn't be using packages from here, because they can be dangerous and harmful even for the most experienced.

project/orphaned/:

Packages that have been orphaned by their old maintainers, and withdrawn from the distribution.

Normally there are three Debian distributions in the dists directory. They are named the stable distribution, the testing distribution, and the unstable distribution. Sometimes there was also a frozen distribution (currently it is just a development stage of the testing distribution). Each distribution is defined as a symlink to the actual directory with a codename in the dists directory.

Package entries for the stable distribution, Debian Etch (4.0), are recorded into the stable (symlink to etch/) directory:

• stable/main/: This directory contains the package versions belonging to the most recent official release of the Debian system.

  These packages are all free; that is, they all comply with the Debian Free Software Guidelines (DFSG) (also available as file:///usr/share/doc/debian/social-contract.txt installed by debian-doc).

• stable/non-free/: This directory contains packages that fail to qualify as free according to the DFSG.

  For example, some packages have licenses that prohibit commercial distribution. Others can be redistributed but are shareware.

• stable/contrib/: Each package in this directory is itself DFSG-free but somehow Depends on a package that is not DFSG-free.

Now, in addition to the above locations, nowadays physical packages are located under the pool directory (The pool directory, Section 2.1.10).

The current status of stable distribution bugs is reported on the Stable Problems web page.

Package entries for the testing distribution, Debian Lenny, are recorded into the testing (symlink to lenny/) directory after they have undergone some degree of testing in unstable. Nowadays physical packages are located under the pool directory (The pool directory, Section 2.1.10). There are also main, contrib, and non-free subdirectories in testing/, which serve the same functions as in stable/.

These packages must be in sync on all architectures where they have been built and must be installable; they must also have fewer release-critical bugs than the versions currently in unstable. This way, we hope that testing is always close to being a release candidate. More details of the testing mechanism are at http://www.debian.org/devel/testing.

The latest status of the testing distribution is reported at these sites:

• update excuses

• testing problems

• release-critical bugs

• base system bugs

• bugs in standard and task packages

• other bugs and bug-squashing party notes

Package entries for the unstable distribution, always codenamed "Sid", are recorded into the unstable (symlink to sid/) directory after they are uploaded to the Debian archive and stay here until they are moved to testing/. Nowadays physical packages are located under the pool directory (The pool directory, Section 2.1.10). There are also main, contrib, and non-free subdirectories in unstable/, which serve the same functions as in stable/.

The unstable distribution contains a snapshot of the most current development system. Users are welcome to use and test these packages, but are warned about their state of readiness. The advantage of using the unstable distribution is that you are always up-to-date with the latest in the Debian software project—but if it breaks, you get to keep both parts. :-)

The current status of unstable distribution bugs is reported on the Unstable Problems web page.

When the testing distribution is mature enough, it becomes frozen, meaning no new code is accepted anymore, just bugfixes, if necessary. Also, a new testing tree is created in the dists directory, assigned a new codename. The frozen distribution passes through a few months of testing, with intermittent updates and deep freezes called "test cycles".

We keep a record of bugs in the frozen distribution that can delay a package from being released or bugs that can hold back the whole release. Once that bug count lowers to maximum acceptable values, the frozen distribution becomes stable, it is released, and the previous stable distribution becomes obsolete (and moves to the archive).

Physical directory names in the dists directory, such as etch/ and lenny/, are just "codenames". When a Debian distribution is in the development stage, it has no version number, but a codename instead. The purpose of these codenames is to make the mirroring of the Debian distributions easier. (If unstable would be a real directory and it's name would suddenly change to stable/, a lot of stuff would have to be needlessly downloaded again).

Currently, stable/ is a symbolic link to etch/, and testing/ is a symbolic link to lenny/. This means that Etch is the current stable distribution and Lenny is the current testing distribution.

unstable/ is a permanent symbolic link to sid/, as Sid is always the unstable distribution.

Codenames that have already been used are: "Buzz" for release 1.1, "Rex" for release 1.2, "Bo" for releases 1.3.x, "Hamm" for release 2.0, "Slink" for release 2.1, "Potato" for release 2.2, "Woody" for release 3.0, and "Sarge" for release 3.1.

So far they have been characters taken from the movie Toy Story by Pixar.

• Buzz (Buzz Lightyear) was the spaceman,

• Rex was the tyrannosaurus,

• Bo (Bo Peep) was the girl who took care of the sheep,

• Hamm was the piggy bank,

• Slink (Slinky Dog) was the toy dog,

• Potato was, of course, Mr. Potato Head,

• Woody was the cowboy,

• Sarge was a leader of the Green Plastic Army Men,

• Etch (Etch-a-Sketch) was the blackboard,

• Sid was a boy next door who destroyed toys.

Historically, packages were kept in the subdirectory of dists corresponding to the distribution that contained them. This turned out to cause various problems, such as large bandwidth consumption on mirrors when major changes were made.

Packages are now kept in a large "pool", structured according to the name of the source package. To make this manageable, the pool is subdivided by section (main, contrib, and non-free) and by the first letter of the source package name. These directories contain several files: the binary packages for each architecture, and the source packages from which the binary packages were generated.

You can find out where each package is placed by executing a command like apt-cache showsrc mypackagename and looking at the "Directory:" line. For example, the apache packages are stored in pool/main/a/apache/. Since there are so many lib* packages, these are treated specially: for instance, libpaper packages are stored in pool/main/libp/libpaper/.

The dists directories are still used for the index files used by programs like apt.

Normally, you won't have to worry about any of this, as new apt and probably older dpkg-ftp will handle it seamlessly. If you want more information, see the RFC: implementation of package pools.

When the present-day Sid did not exist, the Debian archive site organization had one major flaw: there was an assumption that when an architecture was created in the current unstable/, it would be released when that distribution became the new stable. For many architectures that wasn't the case, with the result that those directories had to be moved at release time. This was impractical because the move would chew up lots of bandwidth.

The archive administrators worked around this problem for several years by placing binaries for unreleased architectures in a special directory called sid. When an architecture was released the first time there was a link from the current stable/ to sid/, and from then on they were created inside the unstable/ tree as usual. This layout was somewhat confusing to users.

With the advent of package pools (see The pool directory, Section 2.1.10) during the Woody distribution development, binary packages began to be stored in a canonical location in the pool, regardless of the distribution, so releasing a distribution no longer causes large bandwidth consumption on the mirrors (there is, however, a lot of gradual bandwidth consumption throughout the development process).

Uploaded packages are first located at http://incoming.debian.org/ after being checked to insure that they really come from a Debian developer (and are put in the DELAYED subdirectory in the case of a Non-Maintainer Upload (NMU)). Once a day, they are moved out of incoming/ to unstable/.

In an emergency, you may want to install packages from incoming/ before they reach unstable/.

While the most recent Debian distributions are kept under the debian directory on each Debian mirror site, archives for older Debian distributions such as Slink are kept on http://archive.debian.org/ or under the debian-archive directory on each Debian mirror site.

Older testing and unstable packages can be located at http://snapshot.debian.net/.

Within each of the major directory trees (dists/stable/main, dists/stable/contrib, dists/stable/non-free, dists/unstable/main, etc.), the binary package entries reside in subdirectories whose names indicate the chip architecture for which they were compiled.

• binary-all/, for packages which are architecture-independent. These include, for example, Perl scripts, or pure documentation.

• binary-platform/, for packages which execute on a particular binary platform.

Please note that the actual binary packages no longer reside in these directories, but in the top-level pool directory. The index files (Packages and Packages.gz) have been kept, though, for backwards compatibility.

For the actual binary architectures supported, see the Release Notes for each distribution. They can be located at the Release Notes sites for stable and testing.

Source code is included for everything in the Debian system. Moreover, the license terms of most programs in the system require that source code be distributed along with the programs, or that an offer to provide the source code accompany the programs.

Normally the source code is distributed in the source directories, which are parallel to all the architecture-specific binary directories, or more recently in the pool directory (see The pool directory, Section 2.1.10). To retrieve the source code without having to be familiar with the structure of the Debian archive, try a command like apt-get source mypackagename.

Some packages, notably pine, are only available in a source package due to their licensing limitations. (Recently the pine-tracker package has been provided to facilitate Pine installation.) The procedures described in Port a package to the stable system, Section 6.4.10 and Packaging, Section 13.10 provide ways to build a package manually.

Source code may or may not be available for packages in the contrib and non-free directories, which are not formally part of the Debian system.

Packages generally contain all of the files necessary to implement a set of related commands or features. There are two types of Debian packages:

• Binary packages, which contain executables, configuration files, man/info pages, copyright information, and other documentation. These packages are distributed in a Debian-specific archive format (see Debian package format, Section 2.2.2); they are usually distinguished by having a .deb file extension. Binary packages can be unpacked using the Debian utility dpkg; details are given in its manual page.

• Source packages, which consist of a .dsc file describing the source package (including the names of the following files), a .orig.tar.gz file that contains the original unmodified source in gzip-compressed tar format, and usually a .diff.gz file that contains the Debian-specific changes to the original source. The utility dpkg-source packs and unpacks Debian source archives; details are provided in its manual page.

Installation of software by the package system uses "dependencies" which are declared by the package maintainers. These dependencies are documented in the control file associated with each package. For example, the package containing the GNU C compiler (gcc) Depends on the package binutils which includes the linker and assembler. If a user attempts to install gcc without having first installed binutils, the package management system (dpkg) will print an error message that it also needs binutils, and stop installing gcc. (However, this facility can be overridden by the insistent user; see dpkg(8).) For additional details, see Package dependencies, Section 2.2.8 below.

Debian's packaging tools can be used to:

• manipulate and manage packages or parts of packages,

• aid the user in the splitting of packages that must be transmitted through a limited-size medium such as floppy disks,

• aid developers in the construction of package archives, and

• aid users in the installation of packages which reside on a remote Debian archive site.

A Debian "package", or a Debian archive file, contains the executable files, libraries, and documentation associated with a particular program suite or set of related programs. Normally, a Debian archive file has a filename that ends in .deb. [1]

The internals of this Debian binary package format are described in the deb(5) manual page. Because this internal format is subject to change (between major releases of Debian), always use dpkg-deb(1) for manipulating .deb files.

Through at least the Sarge distribution, all Debian archive files have been manipulable by the standard Unix commands ar and tar, even when dpkg commands are not available.

The Debian package filenames conform to the following convention:

     foo_ver-rev_arch.deb

where, usually, foo is the package name, ver is the upstream version number, rev is the Debian revision number, and arch is the target architecture. Files are easily renamed, of course. You can find out what package is really contained in any given file of name filename by running the following command:

     dpkg --info filename

The Debian revision number is specified by the Debian developer or by whoever built the package. A change in revision number usually indicates that some aspect of the packaging has changed.

Files that are intended to be changeable by the local administrator are kept in /etc/. Debian policy dictates that all changes to locally configurable files be preserved across package upgrades.

If a default version of a locally configurable file is shipped in the package itself then the file is listed as a "conffile". The package management system does not upgrade conffiles that have been changed by the administrator since the package was last installed without getting the administrator's permission. On the other hand, if the conffile has not been changed by the administrator then the conffile will be upgraded along with the rest of the package. This is almost always desirable and so it is advantageous to minimize changes to conffiles.

To list the conffiles belonging to a package run the following command:

     dpkg --status package

The list follows the "Conffiles:" line.

For more information about conffiles you can read the section of the Debian Policy Manual entitled "Configuration files". (See References, Section 15.1).

Debian maintenance scripts are executable scripts which are automatically run before or after a package is installed. Along with a file named control, all of these files are part of the "control" section of a Debian archive file.

The individual files are:

preinst

This script executes before its package is unpacked from its Debian archive (.deb) file. Many "preinst" scripts stop services for packages which are being upgraded until their installation or upgrade is completed (following the successful execution of the "postinst" script).

postinst

This script typically completes any required configuration of a package once it has been unpacked from its Debian archive (.deb) file. Often, "postinst" scripts ask the user for input, and/or warn the user that if he accepts default values, he should remember to go back and reconfigure the package as the situation warrants. Many "postinst" scripts then execute any commands necessary to start or restart a service once a new package has been installed or upgraded.

prerm

This script typically stops any daemons which are associated with a package. It is executed before the removal of files associated with the package.

postrm

This script typically modifies links or other files associated with a package, and/or removes files created by it. (Also see Virtual packages, Section 2.2.7.)

Currently all of the control files can be found in the directory /var/lib/dpkg/info. The files relevant to package foo begin with the name "foo" and have file extensions of "preinst", "postinst", etc., as appropriate. The file foo.list in that directory lists all of the files that were installed with the package foo. (Note that the location of these files is a dpkg internal, and may be subject to change.)

Each Debian package is assigned a priority by the distribution maintainers, as an aid to the package management system. The priorities are:

• Required packages are necessary for the proper functioning of the system.

  This includes all tools that are necessary to repair system defects. You must not remove these packages or your system may become totally broken and you may not even be able to use dpkg to restore things. Systems with only the Required packages are probably inadequate for most purposes, but they do have enough functionality to allow the sysadmin to boot and install more software.

• Important packages should be found on any Unix-like system.

  Other packages without which the system will not run well or be usable will carry this priority. This does not include Emacs or X11 or TeX or any other large applications. These packages only constitute the bare infrastructure.

• Standard packages are standard on any Linux system, including a reasonably small but not too limited character-mode system.

  This is what will install by default if users do not select anything else. "Standard" does not include many large applications, but it does include Emacs (this is more a piece of infrastructure than an application) and a reasonable subset of TeX and LaTeX (if this turns out to be possible without X).

• Optional packages include all those that you might reasonably want to install even if you are unfamiliar with them, and if you don't have specialized requirements.

  This includes X11, a full TeX distribution, and lots of applications.

• Extra packages either conflict with others with higher priorities, have little use to users who are unfamiliar with them, or have specialized requirements that make them unsuitable for "Optional".

Please note the differences among "Priority: required", "Section: base" and "Essential: yes" in the package description. "Section: base" means that this package is installed before everything else on a new system. Most of the packages in "Section: base" have the "Priority: required" or at least "Priority: important", and many of them are tagged with "Essential: yes". "Essential: yes" means that this package requires to specify an extra force option to the package management system such as dpkg when removing from the system. For example, libc6, mawk, and makedev are "Priority: required" and "Section: base" but are not "Essential: yes".

A virtual package is a generic name that applies to any one of a group of packages, all of which provide similar basic functionality. For example, both the tin and trn programs are news readers, and either one should therefore satisfy the need of a program that requires a news reader on the system in order to be useful. They are therefore both said to Provide the "virtual package" called news-reader.

Similarly, many packages such as exim, exim4, sendmail, and postfix, provide the functionality of a mail transport agent. They are therefore said to Provide the virtual package mail-transport-agent. If either one is installed, then any program that Depends on the installation of a mail transport agent will be satisfied by the existence of this virtual package.

Debian has a mechanism such that, if more than one package which Provides the same virtual package is installed on a system, the system administrator can set one as the preferred package. The relevant command is update-alternatives, and is described further in Alternative commands, Section 6.5.3.

The Debian packaging system handles dependency declarations which are used to express the fact that one package requires another package to be installed in order to work or in order to work better.

• Package A Depends on Package B if B absolutely must be installed in order to use A. In some cases, A Depends not only on B, but on a specific version of B. In this case, the version dependency is usually a lower limit, in the sense that A Depends on any version of B more recent than some specified version.

• Package A Recommends Package B if the package maintainer judges that most users would not want A without also having the functionality provided by B.

• Package A Suggests Package B if B contains files that are related to and enhance the functionality of A. The same relationship is expressed by declaring that Package B Enhances Package A.

• Package A Conflicts with Package B when A will not operate properly if B is installed on the system. "Conflicts" status is often combined with "Replaces".

• Package A Replaces Package B when files installed by B are removed or overwritten by files in A.

• Package A Provides Package B when all of the files and functionality of B are incorporated into A.

More detailed information on the use of each these terms can be found in the Packaging Manual and the Policy Manual.

Note that aptitude and dselect have more fine-grained control over packages specified by Recommends and Suggests than apt-get, which simply pulls all the packages specified by Depends and leaves all the packages specified by Recommends and Suggests. Both programs in modern form use APT as their back end.

dpkg always configures a package upon which another package Depends before it configures the package that Depends on it. However, dpkg normally unpacks archive files in arbitrary order, independently of dependencies. (Unpacking consists of extracting files from the archive file and putting them in the right place.) If, however, a package Pre-Depends on another then the other package is unpacked and configured before the one that Pre-Depends is even unpacked. [2] The use of this dependency is kept to a minimum.

Package status can be "unknown", "install", "remove", "purge", or "hold". These "want" flags indicate what the user wanted to do with a package (either by making choices in the "Select" section of dselect, or by directly invoking dpkg).

Their meanings are:

• unknown - the user has never indicated whether he wants the package.

• install - the user wants the package installed or upgraded.

• remove - the user wants the package removed, but does not want to remove any existing configuration files.

• purge - the user wants the package to be removed completely, including its configuration files.

• hold - the user wants this package not to be processed, i.e., he wants to keep the current version with the current status, whatever that is.

There are two mechanisms for holding back packages from an upgrade, through dpkg, or, beginning with Woody, through APT.

With dpkg, first export the list of package selections:

     dpkg --get-selections \* > selections.txt

Then edit the resulting file selections.txt, changing the line containing the package you wish to hold, e.g. libc6, from this:

     libc6                       install

to this:

     libc6                       hold

Save the file, and reload it into dpkg database with:

     dpkg --set-selections < selections.txt

Or, if you know the package name to hold, simply run:

     echo libc6 hold | dpkg --set-selections

This procedure holds packages at the install process of each package file.

The same effect can be obtained through dselect. Simply enter the [S]elect screen, find the package you wish to hold in its present state, and press the `=' key (or `H'). The changes will take effect immediately after you exit the [S]elect screen.

The APT system in the Woody distribution has a new alternative mechanism for holding packages during the archive retrieval process using Pin-Priority. See the manual page apt_preferences(5), along with http://www.debian.org/doc/manuals/apt-howto/ or the apt-howto package.

Source packages are distributed in a directory called source, and you can either download them manually, or use

     apt-get source foo

to fetch them (see the apt-get(8) manual page on how to set up APT for doing that).

For a package foo, you will need all of foo_*.dsc, foo_*.tar.gz, and foo_*.diff.gz to compile the source (note: there is no .diff.gz for a Debian native package).

Once you have them, if you have the dpkg-dev package installed, the command

     $ dpkg-source -x foo_version-revision.dsc

will extract the package into a directory called foo-version.

Issue the following command to build the binary package:

     $ cd foo-version
     $ su -c "apt-get update ; apt-get install fakeroot"
     $ dpkg-buildpackage -rfakeroot -us -uc

Then,

     # su -c "dpkg -i ../foo_version-revision_arch.deb"

to install the newly built package. See Port a package to the stable system, Section 6.4.10.

For detailed information on creating new packages, read the New Maintainers' Guide, available in the maint-guide package, or at http://www.debian.org/doc/manuals/maint-guide/.

This is the main program for manipulating package files; read dpkg(8) for a full description.

dpkg comes with several primitive supplemental programs.

• dpkg-deb: Manipulate .deb files. dpkg-deb(1)

• dpkg-ftp: An older package file retrieval command. dpkg-ftp(1)

• dpkg-mountable: An older package file retrieval command. dpkg-mountable(1)

• dpkg-split: Splits a large package into smaller files. dpkg-split(1)

dpkg-ftp and dpkg-mountable have been superseded by the introduction of the APT system.

APT (the Advanced Packaging Tool) is an advanced interface to the Debian packaging system consisting of several programs whose names typically begin with "apt-". apt-get, apt-cache, and apt-cdrom are the command-line tools for handling packages. These also function as the user's "back end" programs to other tools, such as dselect and aptitude. Currently aptitude is the preferred tool for system maintenance.

For more information, install packages apt and aptitude and read aptitude(8), apt-get(8), apt-cache(8), apt-cdrom(8), apt.conf(5), sources.list(5), and apt_preferences(5).

An alternative source of information is the APT HOWTO. This can be installed by apt-howto at file:///usr/share/doc/Debian/apt-howto/.

apt-get upgrade and apt-get dist-upgrade pull only the packages listed under "Depends:" and overlook all the packages listed under "Recommends:" and "Suggests:". To avoid this, use dselect.

This program is a menu-driven user interface to the Debian package management system. It is particularly useful for first-time installations and large-scale upgrades. See dselect, Section 6.2.4.

For more information, read dselect Documentation for Beginners.

The kernel (filesystem) in Debian systems supports replacing files even while they're being used. When packages are upgraded any services provided by those packages are restarted if they are configured to run in the current runlevel. The Debian system does not require use of the single-user mode to upgrade a running system.

If you have manually downloaded package files to your disk (which is not absolutely necessary, see above for the description of dpkg-ftp or APT), then after you have installed the packages, you can remove the .deb files from your system.

If APT is used, these files are cached in the /var/cache/apt/archives directory. You may erase them after installation (apt-get clean) or copy them to another machine's /var/cache/apt/archives directory to save downloading during subsequent installations.

dpkg keeps a record of the packages that have been unpacked, configured, removed, and/or purged, but does not (currently) keep a log of terminal activity that occurred while a package was being so manipulated.

The simplest way to work around this is to run your dpkg, dselect, apt-get, etc., sessions within the script(1) program.

Like all Unixes, Debian boots up by executing the program init. The configuration file for init (which is /etc/inittab) specifies that the first script to be executed should be /etc/init.d/rcS.

What happens next depends on whether the sysv-rc package or the file-rc package is installed. The following assumes that the sysv-rc package is installed. (file-rc contains its own /etc/init.d/rcS script and uses a file instead of symlinks in rc directories to control which services are started in which runlevels.)

The /etc/init.d/rcS file from the sysv-rc package runs all of the scripts in /etc/rcS.d/ in order to perform initialization such as checking and mounting file systems, loading modules, starting the network services, setting the clock, and so on. Then, for compatibility, it also runs all the files (except those with a `.' in the filename) in /etc/rc.boot/. The latter directory is reserved for system administrator use, and using it is deprecated. See System initialization, Section 9.1 and System run levels and init.d scripts in the Debian Policy Manual for more info.

Debian does not use a BSD-style rc.local directory.

After completing the boot process, init starts all services that are configured to run in the default runlevel. The default runlevel is given by the entry for id in /etc/inittab. Debian ships with id=2.

Debian uses the following runlevels:

• 1 (single-user mode),

• 2 through 5 (multiuser modes), and

• 0 (halt the system),

• 6 (reboot the system).

Runlevels 7, 8, and 9 can also be used but their rc directories are not populated when packages are installed.

Switch runlevels using the telinit command.

When entering a runlevel all scripts in /etc/rcrunlevel.d/ are executed. The first letter in the name of the script determines the way in which the script is run: scripts whose names begin with K are run with the argument stop. Scripts beginning with S are run with the argument start. The scripts are run in the alphabetical order of their names; thus "stop" scripts are run before "start" scripts and the two-digit numbers following the K or S determine the order in which the scripts are run.

The scripts in /etc/rcrunlevel.d are in fact just symbolic links back to scripts in /etc/init.d/. These scripts also accept "restart" and "force-reload" as argument; the latter methods can be used after a system has been booted in order to restart services or force them to reload their configuration files.

For example:

     # /etc/init.d/exim4 force-reload

Customizing runlevels is an advanced system administration task. The following advice holds for most services.

To enable service service in runlevel R create the symbolic link /etc/rcR.d/Sxyservice with target ../init.d/service. The sequence number xy should be the sequence number that was assigned to the service when the package was installed.

To disable the service, rename the symbolic link so that its name begins with a K instead of with an S and its sequence number is 100 minus xy.

It is convenient to use a runlevel editor such as sysv-rc-conf or ksysv for these purposes.

It is possible to delete the S symlink for a service in a particular runlevel directory instead of renaming it. This does not disable the service but leaves it in a "floating" state as far as the sysv-rc init system is concerned: on runlevel changes the service will be neither started nor stopped but will be left as it was, whether running or not running. Note, however, that a service left in such a floating state will be started if its package is upgraded whether or not it was running before the upgrade. This is a known shortcoming of the current Debian system. Note also that you should retain a service's K symlinks in runlevels 0 and 6. If you delete all the symlinks for a service then on upgrade the service's package will restore the symlinks to their factory default state.

It is not advisable to make any changes to symlinks in /etc/rcS.d/.

One has to understand the Debian policy with respect to headers.

The Debian C libraries are built with the most recent stable releases of the kernel headers.

For example, the Debian-1.2 release used version 5.4.13 of the headers. This practice contrasts with the Linux kernel source packages distributed at all Linux FTP archive sites, which use even more recent versions of the headers. The kernel headers distributed with the kernel source are located in /usr/include/linux/include/.

If you need to compile a program with kernel headers that are newer than those provided by libc6-dev, then you must add -I/usr/src/linux/include/ to your command line when compiling. This came up at one point, for example, with the packaging of the automounter daemon (amd). When new kernels changed some internals dealing with NFS, amd needed to know about them. This required the inclusion of the latest kernel headers.

Users who wish to (or must) build a custom kernel are encouraged to download the package kernel-package. This package contains the script to build the kernel package, and provides the capability to create a Debian kernel-image package just by running the command

     # make-kpkg kernel_image

in the top-level kernel source directory. Help is available by executing the command

     # make-kpkg --help

and through the manual page make-kpkg(1) and The Linux kernel under Debian, Chapter 7.

Users must separately download the source code for the most recent kernel (or the kernel of their choice) from their favorite Linux archive site, unless a kernel-source-version package is available (where version stands for the kernel version). The Debian initrd boot script requires a special kernel patch called initrd; see http://bugs.debian.org/149236.

Detailed instructions for using the kernel-package package are given in the file /usr/share/doc/kernel-package/README.gz.

Debian's modconf package provides a shell script (/usr/sbin/modconf) which can be used to customize the configuration of modules. This script presents a menu-based interface, prompting the user for particulars on the loadable device drivers in his system. The responses are used to customize the file /etc/modules.conf (which lists aliases, and other arguments that must be used in conjunction with various modules) through files in /etc/modutils/, and /etc/modules (which lists the modules that must be loaded at boot time).

Like the (new) Configure.help files that are now available to support the construction of custom kernels, the modconf package comes with a series of help files (in /usr/share/modconf/) which provide detailed information on appropriate arguments for each of the modules.

The kernel-image-NNN.prerm script checks to see whether the kernel you are currently running is the same as the kernel you are trying to de-install. Therefore you can safely remove unwanted kernel image packages using this command:

     # dpkg --purge --force-remove-essential kernel-image-NNN

(Replace NNN with your kernel version and revision number, of course.)

Linux is compatible with most PC hardware and can be installed to almost any system. For me it was as easy as installing Windows 95/98/Me. The hardware compatibility list just seems to keep growing.

If you have a laptop PC, check Linux on Laptops for installation pointers by brand and model.

My recommendation for desktop PC hardware is "Just be conservative":

• SCSI rather than IDE for work, IDE/ATAPI HD for private use.

• IDE/ATAPI CD-ROM (or CD-RW).

• PCI rather than ISA, especially for the network card (NIC).

• Use a cheap NIC. Tulip for PCI, NE2000 for ISA are good.

• Avoid PCMCIA (notebook) as your first Linux install.

• No USB keyboard, mouse, ... unless you want a challenge.

If you have a slow machine, yanking out the hard drive and plugging it into another faster machine for installation is a good idea.

During installation, one will be asked to identify the hardware or chip set of the PC. Sometimes that information may not seem easy to find. Here is one method:

• Open your PC's case and look inside.
• Record the product ID codes on the large chips on the graphics card, network card, chip near serial ports, chip near IDE ports.
• Record card names printed on the back of the PCI and ISA cards.

The following commands on a Linux system should give some idea of actual hardware and configuration.

     $ pager /proc/pci
     $ pager /proc/interrupts
     $ pager /proc/ioports
     $ pager /proc/bus/usb/devices

These commands can be run during the install process from the console screen by pressing Alt-F2.

After the initial installation, with the installation of optional packages such as pciutils, usbutils, and lshw, you can obtain more extensive system information.

     $ lspci -v |pager
     $ lsusb -v |pager
     # lshw  |pager

Typical uses of interrupts:

• IRQ0: timer output (8254)

• IRQ1: keyboard controller

• IRQ2: cascade to IRQ8–IRQ15 on PC-AT

• IRQ3: secondary serial port (io-port=0x2F8) (/dev/ttyS1)

• IRQ4: primary serial port (io-port=0x3F8) (/dev/ttyS0)

• IRQ5: free [sound card (SB16: io-port=0x220, DMA-low=1, DMA-high=5)]

• IRQ6: floppy disk controller (io-port=0x3F0) (/dev/fd0, /dev/fd1)

• IRQ7: parport (io-port=0x378) (/dev/lp0)

• IRQ8: rtc

• IRQ9: software interrupt (int 0x0A), redirect to IRQ2

• IRQ10: free [network interface card (NE2000: io-port=0x300)]

• IRQ11: free [(SB16-SCSI: io-port=0x340, SB16-IDE: io-port=0x1E8,0x3EE)]

• IRQ12: PS/2 Mouse

• IRQ13: free (was 80287 math coprocessor)

• IRQ14: primary IDE controller (/dev/hda, /dev/hdb)

• IRQ15: secondary IDE controller (/dev/hdc, /dev/hdd)

For old non-PnP ISA cards, you may want to set IRQ5, IRQ10, and IRQ11 as non-PnP from the BIOS.

For USB devices, device classes are listed in /proc/bus/usb/devices as Cls=nn:

• Cls=00 : Unused

• Cls=01 : Audio (speaker etc.)

• Cls=02 : Communication (MODEM, NIC, ...)

• Cls=03 : HID (Human Interface Device: KB, mouse, joystick)

• Cls=07 : Printer

• Cls=08 : Mass storage (FDD, CD/DVD drive, HDD, Flash, ...)

• Cls=09 : Hub (USB hub)

• Cls=255 : Vendor specific

If the device class of a device is not 255, Linux supports the device.

Hardware information can also be obtained from other OSs:

Install another commercial Linux distribution. Hardware detection on those tends to be better than on Debian as of now. (This situation should even out once debian-installer is introduced with Sarge.)

Install Windows. Hardware configuration can be obtained by right-clicking "My Computer" to get to Properties / Device Manager. Record all resource information such as IRQ, I/O port address, and DMA. Some old ISA cards may need to be configured under DOS and used accordingly.

"Lilo is limited to 1024 cylinders." Wrong!

The newer lilo used after Debian Potato has lba32 support. If the BIOS of your motherboard is recent enough to support lba32, lilo should be able to load beyond the old 1024-cylinder limitation.

Just make sure to add a line reading "lba32" somewhere near the beginning of your lilo.conf file if you have kept an old lilo.conf. See file:///usr/share/doc/lilo/Manual.txt.gz.

The new boot loader grub from the GNU Hurd project can be installed on a Debian Woody system:

     # apt-get update
     # apt-get install grub-doc
     # mc file:///usr/share/doc/grub-doc/html/
     ... read contents
     # apt-get install grub
     # pager file:///usr/share/doc/grub/README.Debian.gz
     ... read it :)

To edit the GRUB menu, edit /boot/grub/menu.lst. See Setting GRUB boot parameters, Section 8.1.6 for how to set boot parameters during the boot process since it is slightly different from lilo configuration.

For Potato, I liked the IDEPCI disk set for normal install to a desktop. For Woody, I like the bf2.4 boot disk set. They both use a version of boot-floppies to create boot floppies.

If you have a PCMCIA network card, you need to use the standard boot disk set (largest number of floppies but all driver modules available) and configure the NIC in the PCMCIA setup; do not try to set up an NIC card in the standard network setup dialog.

For special systems, you may need to create a custom rescue disk. This can be done by replacing the kernel image named "linux" on the Debian rescue disk by overwriting it with another compressed kernel image compiled off-site for the machine. Details are documented in readme.txt on the rescue disk. The rescue floppy uses the MS-DOS filesystem, so you can use any system to read and edit it. This should make life easier for people with a special network card, etc.

For Sarge, debian-installer and/or pgi is expected to be used for creating boot floppies.

Follow the official instructions found in http://www.debian.org/releases/stable/installmanual or http://www.debian.org/releases/testing/installmanual (work in progress, sometimes this may not exist).

If you are installing a system using boot-floppies in the testing distribution, you may need to open a console terminal during the install process by pressing Alt-F2 and manually edit /etc/apt/sources.list entries, changing "stable" to "testing" to adjust APT sources.

I tend to install lilo into places like /dev/hda3, while installing mbr into /dev/hda. This minimizes the risk of overwriting boot information.

Here is what I choose during the install process.

• MD5 passwords "yes"

• shadow passwords "yes"

• Install "advanced" (dselect **) and select

  • Exclude emacs (if selected), nvi, tex, telnet, talk(d);

  • Include mc, vim, either one of nano-tiny or elvis-tiny.

    See dselect, Section 6.2.4. Even if you are an Emacs fan, avoid it now and be content with nano during install. Also avoid installing other large packages such as TeX (Potato used to do this) at this stage. See Rescue editors, Section 11.2 for the reason for installing nano-tiny or elvis-tiny here.

• All configuration questions = "y" (replace current) during each package install dialog.

• exim: select 2 for machine since I send mail through my ISP's SMTP server.

For more information on dselect, see dselect, Section 6.2.4.

Example of LAN configuration (C subnet: 192.168.1.0/24):

     Internet
        |
        +--- External ISP provides POP service (accessed by fetchmail)
        | 
       Access point ISP provides DHCP service and SMTP relay service
        |                     :
       Cable modem         (Dialup)
        |                     :
     LAN Gateway machine external port: eth0 (IP given by ISP's DHCP)
      use old notebook PC (IBM Thinkpad, 486 DX2 50MHz, 20MB RAM)
      run Linux 2.4 kernel with ext3 filesystem.
      run "ipmasq" package (with stronger patch, NAT, and firewall)
      run "dhcp-client" package configured for eth0 (override DNS setting)
      run "dhcp" package configured for eth1
      run "exim" as the smarthost (mode 2)
      run "fetchmail" with a long interval (fallback)
      run "bind" as the cache nameserver for Internet from LAN
                 as authoritative nameserver for LAN domain from LAN
      run "ssh" on port 22 and 8080 (connect from anywhere)
      run "squid" as the cache server for the Debian archive (for APT)
     LAN Gateway machine internal port: eth1 (IP = 192.168.1.1, fixed)
                              |
              +--- LAN Switch (100base T) ---+
              |                              |
     Some fixed IP clients on LAN     Some DHCP clients on LAN
     (IP = 192.168.1.2-127, fixed)    (IP = 192.168.1.128-200, dynamic)

See Network configuration, Chapter 10 for the details of configuring the network. See Building a gateway router, Section 10.12 for the details of configuring the LAN gateway server.

In order to have a consistent feel across machines, the first few accounts are always the same in my system.

I always create a first user account with a name like "admin" (uid=1000). I forward all root email there. This account is given membership in the adm group (see "Why GNU su does not support the wheel group", Section 9.2.2), which can be given a good amount of root privilege through su using PAM or the sudo command. See Add a user account, Section 4.1.3 for details.

Following are rough guidelines for DRAM.

       4MB:  Bare minimum for Linux kernel to function.
      16MB:  Minimum for reasonable console system.
      32MB:  Minimum for simple X system.
      64MB:  Minimum for X system with GNOME/KDE.
     128MB:  Comfortable for X system with GNOME/KDE.
     256MB (or more): Why not if you can afford it?  DRAM is cheap.

Using the boot option mem=4m (or lilo append="mem=4m") will show how the system would perform with 4MB of memory installed. A lilo boot parameter is needed for a system containing more than 64MB of memory with an old BIOS.

I use the following guidelines for swap space:

• Each swap partition is < 128MB (if using an old 2.0 kernel), < 2GB (with recent kernels)

• Total = either (1 to 2 times installed RAM) or (128MB to 2GB) as a guideline

• Spread them on different drives and mount all of them with sw,pri=1 options in /etc/fstab. This ensures that the kernel does a striping RAID of the swap partitions and offers the maximum swap performance.

• Use a central portion of the hard disk when possible.

Even if you never need it, some swap space (128MB) is desirable so the system will slow down before it crashes hard with a program which leaks memory.

Make sure you have all required kernel functions activated through kernel compile time configuration or modules:

• Under "Input core support":

  • "Input core support" (CONFIG_INPUT, input.o),

  • "Mouse support" (CONFIG_INPUT_MOUSEDEV, mousedev.o),

• Under "USB support":

  • "Support for USB" (CONFIG_USB, usbcore.o),

  • "Preliminary USB device filesystem" (CONFIG_USB_DEVICEFS),

  • "UHCI" or "OHCI" (CONFIG_USB_UHCI || CONFIG_USB_UHCI_ALT || CONFIG_USB_OHCI, usb-uhci.o || uhci.o || usb-ohci.o),

  • "USB Human Interface Device (full HID) support" (CONFIG_USB_HID, hid.o), and

  • "HID input layer support" (CONFIG_USB_HIDINPUT)

Here, lower case names are module names.

If you're not using devfs, create a device node /dev/input/mice with major 13 and minor 63 as follows:

     # cd /dev
     # mkdir input
     # mknod input/mice c 13 63

For typical scroll USB mice, configuration combinations should be:

     /etc/gpm.conf            | /etc/X11/XF86Config-4
     =========================+======================================
     device=/dev/input/mice   | Section "InputDevice"
     responsiveness=          |  Identifier "Generic Mouse"
     repeat_type=raw          |  Driver     "mouse"
     type=autops2             |  Option     "SendCoreEvents" "true"
     append=""                |  Option     "Device"   "/dev/input/mice"
     sample_rate=             |  Option     "Protocol" "IMPS/2"
                              |  Option     "Buttons" "5"
                              |  Option  "ZAxisMapping" "4 5"
                              | EndSection

See the Linux USB Project for more information.

Although the touchpad on a laptop computer emulates a 2-button PS/2 mouse as the default behavior, the tpconfig package enables full control of the device. For example, setting OPTIONS="--tapmode=0" in /etc/default/tpconfig will disable pesky "click by tap" behavior. Set /etc/gpm.conf as follows to use both touchpad and USB external mouse on the console:

     device=/dev/psaux
     responsiveness=
     repeat_type=ms3
     type=autops2
     append="-M -m /dev/input/mice -t autops2"
     sample_rate=

For the lpr/lpd type spoolers (lpr, lprng, and gnulpr), set up /etc/printcap as follows if they are connected to a PostScript or text-only printer (the basics):

     lp|alias:\
             :sd=/var/spool/lpd/lp:\
             :mx#0:\
             :sh:\
             :lp=/dev/lp0:

Meaning of the above lines:

• Head line: lp – name of spool, alias = alias

• mx#0 – max file size unlimited

• sh – suppress printing of burst page header

• lp=/dev/lp0 – local printer device, or port@host for remote

This is a good configuration if you are connected to a PostScript printer. Also, when printing from a Windows machine through Samba, this is a good configuration for any Windows-supported printer (no bidirectional communication is supported). You have to select the corresponding printer configuration on the Windows machine.

If you do not have a PostScript printer, you need to set up a filtering system using gs. There are many autoconfiguration tools provided for setting up /etc/printcap. Any of these combinations is an option:

• gnulpr, (lpr-ppd) and printtool—I use this.

• lpr and apsfilter

• lpr and magicfilter

• lprng and lprngtool

• lprng and apsfilter

• lprng and magicfilter

In order to run GUI configuration tools such as printtool, see Getting root in X, Section 9.4.12 to gain root privilege. Printer spools created with printtool use gs and act like PostScript printers. So when accessing them, use PostScript printer drivers. On the Windows side, "Apple LaserWriter" is the standard one.

The Common UNIX Printing System (or CUPS™) is installed by using aptitude and installing all packages under "Tasks" -> "Servers" -> "Print Server". (Sarge) For the best result, you should set aptitude with "F10" -> "Options" -> "Dependency handling" -> "[X] Install Recommended packages automatically".

KDE and Gnome Desktop Environments provide easy printer configuration. Alternatively, you can configure the system using any web browser if swat is installed:

     $ mybrowser http://localhost:631

For example, to add your printer on some port to the list of accessible printers:

• click "Printers" from the main page, and then "Add Printer",

• enter "root" for the username and its password,

• proceed to add the printer following the prompts,

• go back to the "Printers" page and click "Configure Printer", and

• proceed to configure the paper size, resolution, and other parameters.

See more information at http://localhost:631/documentation.html and http://www.cups.org/cups-help.html.

Once you have made it this far, you have a small but functioning Debian system. It is a good time to install bigger packages.

Here the first thing you may want to do is select your favorite editor and any programs you need with aptitude. You can install many Emacs variants at the same time. See Popular editors, Section 11.1.

Then, again with aptitude, select (almost) all packages under "Tasks" --> "End-user" to obtain very complete end-user oriented system.

I usually edit /etc/inittab for easy shutdown.

     ...
     # What to do when CTRL-ALT-DEL is pressed.
     ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -h now

Modules for the device drivers are configured during the initial installation. modconf provides menu-driven module configuration afterward. This program is quite useful when some modules were left out during the initial installation or a new kernel was installed after the initial installation.

All preloading module names need to be listed in /etc/modules. I also use lsmod and depmod to control them manually.

Also make sure to add a few lines in /etc/modules to handle IP masquerading (FTP, etc.) if you did not enable it.

For IDE connected CD-RW drive with 2.4 kernel, edit the following files:

     /etc/lilo.conf  (add append="hdc=ide-scsi ignore=hdc", 
                      run lilo to activate)
     /dev/cdrom      (symlink # cd /dev; ln -sf scd0 cdrom)
     /etc/modules    (add "ide-scsi" and "sg". If needed "sr" after this.)

See CD writers, Section 9.3 for details.

Edit /etc/lilo.conf as follows to set boot-prompt parameters for large memory (for 2.2 kernels) and auto power-off (for APM):

     append="mem=128M apm=on apm=power-off noapic"

Run lilo to install these settings. apm=power-off is needed for a SMP kernel and noapic is needed to avoid problems for my buggy SMP hardware. The same can be done directly by entering options at the boot prompt. See Other boot tricks with the boot prompt, Section 8.1.5.

If APM is compiled as a module, as in Debian default 2.4 kernels, run insmod apm power_off=1 after boot or set /etc/modules by:

     # echo "apm power_off=1" >>/etc/modules

Alternatively, compiling ACPI support achieves the same goal with newer kernels and seems to be more SMP-friendly (this requires a newer motherboard). The 2.4 kernel on newer motherboards should detect large memory correctly.

     CONFIG_PM=y
     CONFIG_ACPI=y
     ...
     CONFIG_ACPI_BUSMGR=m
     CONFIG_ACPI_SYS=m

and add the following lines in /etc/modules in this order:

     ospm_busmgr
     ospm_system

Or recompile the kernel with all of the kernel options above set to "y". In any case, none of the boot-prompt parameters are needed with ACPI.

Recent Linux kernels enable ECN by default, which may cause access problems with some websites on bad routers. To check ECN status:

     # cat /proc/sys/net/ipv4/tcp_ecn
      ... or
     # sysctl net.ipv4.tcp_ecn

To turn it off, use:

     # echo "0" > /proc/sys/net/ipv4/tcp_ecn
      ... or
     # sysctl -w net.ipv4.tcp_ecn=0

To disable TCP ECN on every boot, edit /etc/sysctl.conf and add:

     net.ipv4.tcp_ecn = 0

Install the pppconfig package to set up dialup PPP access.

     # apt-get install pppconfig
     # pppconfig
      ... follow the directions to configure dialup PPP
     # adduser user_name dip
      ... allow user_name to access dialup PPP

Dialup PPP access can be initiated by the user (user_name):

     $ pon ISP_name  # start PPP access to your ISP
      ... enjoy the Internet
     $ poff ISP_name # stop PPP access, ISP_name optional

See Configuring a PPP interface, Section 10.2.4 for more details.

You may want to add an /etc/cron.deny file, missing from the standard Debian install (you can copy /etc/at.deny).

Upon rebooting the system, you will be presented either the graphical login screen or the character based login screen depending on your initial selection of packages. For the sake of simplicity, if you are presented with the graphical login screen, press Ctrl-Alt-F1 [3] to gain the character based login screen.

Suppose your hostname is foo, the login prompt looks like:

     foo login:

Type root , press the Enter-key and type the password which you selected during the install process. In the Debian system, following the Unix tradition, the password is case sensitive. Then the system starts with the greeting message and presents you with the root command prompt waiting for your input. [4]

     foo login: root
     Password: 
     Last login: Sun Oct 26 19:04:09 2003 on tty3
     Linux foo 2.4.22-1-686 #6 Sat Oct 4 14:09:08 EST 2003 i686 GNU/Linux
     
     Most of the programs included with the Debian GNU/Linux system are
     freely redistributable; the exact distribution terms for each program
     are described in the individual files in /usr/share/doc/*/copyright
     
     Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
     permitted by applicable law.
     
     root@foo:root#

You are ready to perform the system administration from this root command prompt. This root account is also called superuser or privileged user. From this account, you can do anything:

• read, write, and remove any files on the system irrespective of their file permissions

• set file ownership and permission of any files on the system

• set the password of any non-privileged users on the system

• login to any accounts without their passwords

It is extremely bad idea to share the access to the root account by sharing the password. Use of program such as sudo(8) is the good way to share the administrative privileges.

Please note that it is considered a good Unix habit to login to the non-privileged user account first even when you plan to perform administrative activities. Use commands sudo, super, or su -c to gain the limited root privileged when needed. See Working more safely – sudo, Section 9.2.4. [5]

I think learning a computer system is like learning a new foreign language. Although tutorial books are helpful, you have to practice it with helper tools. In this context, I think it is a good idea to install few additional packages such as mc, vim, lynx, doc-linux-text, and debian-policy. [6]

     # apt-get update
      ...
     # apt-get install mc vim lynx doc-linux-text debian-policy 
      ...

If you already had these packages installed, nothing will be installed.

During the installation, you usually created a non-privileged user account who receives e-mails sent to the root account. [7] Since you do not want to use this special user account for the following training activities either, you should create another new user account.

Suppose you wish this new username to be penguin, type:

     root@foo:root# adduser penguin
     ... answer all the questions

will create it. [8] Before going further, let's learn few things first.

In the default Debian system, there are six independent pseudo-terminals available, i.e., you can use the PC's VGA character console screen as 6 switchable VT-100 terminals. Switch from one to another by pressing the Left-Alt-key and one of the F1–F6 keys simultaneously. Each pseudo-terminal allows independent login to accounts. The multiuser environment is a great Unix feature, and very addictive.

If you accidentally typed Alt-F7 on a system running the X Window System and the console screen displays graphic screen, regain the access to the character console by pressing Ctrl-Alt-F1. Just try to move to different console and come back to the original one to get used to this.

Just like any other modern OSs where the file operation involves caching data in the memory, the Debian system needs the proper shutdown procedure before power can safely be turned off to maintain the integrity of files. Use the following command from the root command prompt to shutdown the system:

     # shutdown -h now

This is for the normal multiuser mode. If you are in the single-user mode, use following from the root command prompt:

     # poweroff -i -f

Alternatively, you may type Ctrl-Alt-Delete to shutdown. [9]

Wait until the system displays "System halted" then shut off power. If the APM or ACPI function has been turned on by the BIOS and Linux properly, the system will power down by itself. See Large memory and auto power-off, Section 3.8.4 for the detail.

Now you are ready to play with the Debian system without risks as long as you use this non-privileged user account penguin. [10]

Let's login to the penguin. If you are at root shell prompt, type Ctrl-D [11] at the root command prompt to close the root shell activity and return to the login prompt. Enter your newly created username penguin and the corresponding password. [12] You will be presented with the following command prompt.

     penguin@foo:penguin$

From here on, the example given will use simplified command prompt for the sake of simplicity. I will use:

• # : root shell prompt

• $ : non-privileged user shell prompt

We will start learning the Debian system first with the easy way Midnight Commander (MC), Section 4.2 and later with the proper way Unix-like work environment, Section 4.3.

In order to make MC to change working directory upon exit, you need to modify ~/.bashrc (or /etc/bash.bashrc, called from .bashrc), as detailed in its manual page, mc(1), under the -P option. [13]

MC takes care of all file operations through its menu, requiring minimal user effort. Just press F1 to get the help screen. You can play with MC just by pressing cursor-keys and function-keys. [14]

The default is two directory panels containing file lists. Another useful mode is to set the right window to "information" to see file access privilege information, etc. Following are some essential keystrokes. With the gpm daemon running, one can use a mouse, too. (Make sure to press the shift-key to obtain the normal behavior of cut and paste in MC.)

• F1: Help menu

• F3: Internal file viewer

• F4: Internal editor

• F9: Activate pulldown menu

• F10: Exit Midnight Commander

• Tab: Move between two windows

• Insert: Mark file for a multiple-file operation such as copy

• Del: Delete file (be careful—set MC to safe delete mode)

• Cursor keys: Self-explanatory

• Any cd command will change the directory shown on the selected screen.

• Ctrl-Enter or Alt-Enter will copy a filename to the command line. Use this with the cp or mv command together with command-line editing.

• Alt-Tab will show shell filename expansion choices.

• One can specify the starting directory for both windows as arguments to MC; for example, mc /etc /root.

• Esc + numberkey == Fn (i.e., Esc + `1' = F1, etc.; Esc + `0' = F10)

• Esc-key == Alt-key (= Meta, M-); i.e., type Esc + `c' for Alt-C.

The internal editor has an interesting cut-and-paste scheme. Pressing F3 marks the start of a selection, a second F3 marks the end of selection and highlights the selection. Then you can move your cursor. If you press F6, the selected area will be moved to the cursor location. If you press F5, the selected area will be copied and inserted at the cursor location. F2 will save the file. F10 will get you out. Most cursor keys work intuitively.

This editor can be directly started on a file:

     $ mc -e filename_to_edit
     $ mcedit filename_to_edit

This is not a multi-window editor, but one can use multiple Linux consoles to achieve the same effect. To copy between windows, use Alt-Fn keys to switch virtual consoles and use "File->Insert file" or "File->Copy to file" to move a portion of a file to another file.

This internal editor can be replaced with any external editor of choice.

Also, many programs use environment variables EDITOR or VISUAL to decide which editor to use. If you are uncomfortable with vim, set these to mcedit by adding these lines to ~/.bashrc:

     ...
     export EDITOR=mcedit
     export VISUAL=mcedit
     ...

I do recommend setting these to vim if possible. Getting used to vim commands is the right thing to do, since Vi-editor is always there in the Linux/Unix world. [15]

Very smart viewer. This is a great tool for searching words in documents. I always use this for files in the /usr/share/doc directory. This is the fastest way to browse through masses of Linux information. This viewer can be directly started like so:

     $ mc -v filename_to_view

Press Enter on a file, and the appropriate program will handle the content of the file. This is a very convenient MC feature.

     executable file:   Execute command
     man, html file:    Pipe content to viewer software
     tar.gz, deb file:  Browse its contents as if subdirectory

In order to allow these viewer and virtual file features to function, viewable files should not be set as executable. Change their status using the chmod command or via the MC file menu.

MC can be used to access files over the Internet using FTP. Go to the menu by pressing F9, then type `p' to activate the FTP virtual filesystem. Enter a URL in the form username:passwd@hostname.domainname, which will retrieve a remote directory that appears like a local one.

Try http.us.debian.org/debian as URL and browse Debian file archive. See The Debian archives, Section 2.1 for how these are organized.

In the Unix-like environment, there are few key strokes which have special meanings. [17]

• Ctrl-U: Erase line before cursor.

• Ctrl-H: Erase a character before cursor.

• Ctrl-D: Terminate input. (exit shell if you are using shell)

• Ctrl-C: Terminate a running program.

• Ctrl-Z: Temporarily stop program. (put it to the background job, see command &, Section 4.3.10.1)

• Ctrl-S: Halt output to screen. [18]

• Ctrl-Q: Reactivate output to screen.

The default shell, bash, has history-editing and tab-completion capabilities to aide the interactive use.

• up-arrow: Start command history search.

• Ctrl-R: Start incremental command history search.

• TAB: Complete input of the filename to the command line.

• Ctrl-V TAB: Input TAB without expansion to the command line.

Other important keystrokes to remember:

• Ctrl-Alt-Del: Reboot/halt the system, see Install a few more packages after initial install, Section 3.8.1.

• Left-click-and-drag mouse: Select and copy to the clipboard.

• Click middle mouse button: Paste clipboard at the cursor.

• Meta-key (Emacs terminology) is assigned traditionally to Left-Alt-key. Some system may be configured to use Windows-key for Meta-key.

Here, in order to use a mouse in the Linux character console, you need to have gpm running as daemon. [19] See Mouse configuration, Section 3.3.

Let's learn the basic Unix commands. [20] Try all the following commands from the non-privileged user account penguin :

• pwd

  • Display name of current/working directory.

• whoami

  • Display current user name.

• file foo

  • Display a type of file for the file foo.

• type -p commandname

  • Display a file location of command commandname.

  • which commandname does the same. [21]

• type commandname

  • Display information on command commandname.

• apropos key-word

  • Find commands related to key-word.

  • man -k key-word does the same.

• whatis commandname

  • Display one line explanation on command commandname.

• man -a commandname

  • Display explanation on command commandname. (Unix style)

• info commandname

  • Display rather long explanation on command commandname. (GNU style)

• ls

  • List contents of directory. (non-dot files and directories) [22]

• ls -a

  • List contents of directory. (all files and directories)

• ls -A

  • List contents of directory. (almost all files and directories, i.e., skip ".." and ".")

• ls -la

  • List all contents of directory with detail information. See The filesystem concept in Debian, Section 4.5.2.

• ls -d

  • List all directories under the current directory.

• lsof foo

  • List open status of file foo.

• mkdir foo

  • Make a new directory foo in the current directory.

• rmdir foo

  • Remove a directory foo in the current directory.

• cd foo

  • Change directory to the directory foo in the current directory or in the directory listed in the variable CDPATH. See cd command in builtins(7).

• cd /

  • Change directory to the root directory.

• cd

  • Change directory to the current user's home directory.

• cd /foo

  • Change directory to the absolute path directory /foo.

• cd ..

  • Change directory to the parent directory.

• cd ~foo

  • Change directory to the home directory of the user foo.

• cd -

  • Change directory to the previous directory.

• </etc/motd pager

  • Display contents of /etc/motd using the default pager. See command < foo, Section 4.3.10.9. [23]

• touch junkfile

  • Create a empty file junkfile.

• cp foo bar

  • Copy a existing file foo to a new file bar.

• rm junkfile

  • Remove a file junkfile.

• mv foo bar

  • Rename an existing file foo to a new name bar.

• mv foo bar/baz

  • Move an existing file foo to a new location with a new name bar/baz. The directory bar must exist.

• chmod 600 foo

  • Make an existing file foo to be non-readable and non-writable by the other people. (non-executable for all)

• chmod 644 foo

  • Make an existing file foo to be readable but non-writable by the other people. (non-executable for all)

• chmod 755 foo

  • Make an existing file foo to be readable but non-writable by the other people. (executable for all)

• top

  • Display process information using full screen. Type "q" to quit.

• ps aux | pager

  • Display information on all the running processes using BSD style output. See command1 | command2, Section 4.3.10.2.

• ps -ef | pager

  • Display information on all the running processes using Unix system-V style output.

• ps aux | grep -e "[e]xim4*"

  • Display all processes running exim or exim4. Learn the regular expression from grep(1) manual page by typing man grep. [24]

• ps axf | pager

  • Display information on all the running processes with ASCII art output.

• kill 1234

  • Kill a process identified by the process ID: 1234. See Kill a process, Section 8.5.1.

• grep -e "pattern" *.html

  • Find a "pattern" in all of the files ending with .html in current directory and display them all.

• gzip foo

  • Compress foo to create foo.gz using the Lempel-Ziv coding (LZ77).

• gunzip foo.gz

  • Decompress foo.gz to create foo.

• bzip2 foo

  • Compress foo to create foo.bz2 using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. (Better compression than gzip)

• bunzip2 foo.bz2

  • Decompress foo.bz2 to create foo.

• tar -xvvf foo.tar

  • Extract files from foo.tar archive.

• tar -xvvzf foo.tar.gz

  • Extract files from gzipped foo.tar.gz archive.

• tar -xvvf --bzip2 foo.tar.bz2

  • Extract files from foo.tar.bz2 archive. [25]

• tar -cvvf foo.tar bar/

  • Archive contents of folder bar/ in foo.tar archive.

• tar -cvvzf foo.tar.gz bar/

  • Archive contents of folder bar/ in compressed foo.tar.gz archive.

• tar -cvvf --bzip2 foo.tar.bz2 bar/

  • Archive contents of folder bar/ in foo.tar.bz2 archive. [26]

• zcat README.gz | pager

  • Display contents of compressed README.gz using the default pager.

• zcat README.gz > foo

  • Create a file foo with the decompressed content of README.gz.

• zcat README.gz >> foo

  • Append the decompressed content of README.gz to the end of the file foo. (If it does not exist, create it first.)

• find . -name pattern

  • find matching filenames using shell pattern. (slower)

• locate -d . pattern

  • find matching filenames using shell pattern. (quicker using regularly generated database)

Please traverse directories and peek into the system using above commands as a training. If you have questions on any of the console commands, please make sure to read the manual page. For example, these commands are the good start:

     $ man man
     $ man bash
     $ man ls

Also this is a good timing to start vim and press F1-key. You should at least read the first 35 lines. Then do the online training course by moving cursor to |tutor| and pressing Ctrl-]. See Editors, Chapter 11 to learn more about editors.

Please note that many Unix-like commands including ones from GNU and BSD will display brief help information if you invoke them in one of the following ways (or without any arguments in some cases):

     $ commandname --help
     $ commandname -h

Try also examples in Debian tips, Chapter 8 as your self training.

Now you have some feel on how to use the Debian system. Let's look deep into the mechanism of the command execution in the Debian system. [27]

A simple command is a sequence of

• variable assignments (optional)
  
• command name arguments (optional)
  
• redirections (optional: > , >> , < , << , etc.)
  
• control operator (optional: && , || ; <newline> , ; , & , ( , ) )
  

For more complex commands with quotations and substitutions, see Command-line processing, Section 13.2.6.

Typical command execution uses a shell line sequence like the following: [28]

     $ date
     Sun Oct 26 08:17:20 CET 2003
     $ LC_ALL=fr_FR date
     dim oct 26 08:17:39 CET 2003

Here, the program date is executed in the foreground job. The environment variable LC_ALL is:

• unset (system default, same as C) for the first command

• set to fr_FR (French locale) for the second command

Most command executions usually do not have preceding environment variable definition. For the above example, you can alternatively execute:

     $ LC_ALL=fr_FR
     $ date
     dim oct 26 08:17:39 CET 2003

As you can see here, the output of command is affected by the environment variable to produce French output. If you want the environment variable to be inherited to the subprocesses (e.g., when calling shell script), you need to "export" it instead by using:

     $ export LC_ALL

When you type a command into the shell, the shell searches the command in the list of directories contained in the PATH environment variable. The value of the PATH environment variable is also called the shell's search path.

In the default Debian installation, the PATH environment variable of user accounts may not include /sbin/. So if you want to run any commands such as ifconfig from /sbin/, you must change the PATH environment variable to include it. The PATH environment variable is usually set by the initialization file ~/.bash_profile, see Bash configuration, Section 3.2.

Some commands take arguments. The arguments starting with - or -- are called options and control the behavior of the command.

     $ date
     Mon Oct 27 23:02:09 CET 2003
     $ date -R
     Mon, 27 Oct 2003 23:02:40 +0100

Here the command-line argument -R changes the date command behavior to output RFC-2822 compliant date string.

Often you want a command to work with a group of files without typing all of them. The filename expansion pattern using the shell wildcards facilitate this needs.

• *

  • This matches any group of 0 or more characters.

  • This does not match a filename started with ".".

• ?

  • This matches exactly one character.

• [...]

  • This matches exactly one character with any character enclosed in brackets

• [a-z]

  • This matches exactly one character with any character between a and z.

• [^...]

  • This matches exactly one character other than any character enclosed in brackets (excluding "^").

For example, try the following and think yourself:

     $ mkdir junk; cd junk; touch 1.txt 2.txt 3.c 4.h .5.txt
     $ echo *.txt
     1.txt 2.txt
     $ echo *
     1.txt 2.txt 3.c 4.h
     $ echo *.[hc]
     3.c 4.h
     $ echo .*
     . .. .5.txt
     $ echo .[^.]*
     .5.txt
     $ echo [^1-3]*
     4.h
     $ cd ..; rm -rf junk

Each command returns its exit status as the return value.

• return value = 0 if the command executes successfully.

• return value = non-zero if the command exits with error.

This return value can be accessed by the $? shell variable immediately after the execution.

     $ [ 1 = 1 ] ; echo $?
     0
     $ [ 1 = 2 ] ; echo $?
     1

Please note that, when the return value is used in the logical context for the shell, success is treated as the logical TRUE. This is somewhat non-intuitive since success bears value zero.

See Shell conditionals, Section 13.2.5.

You can set an alias for the frequently used command. For example:

     $ alias la='ls -la'

Now, la works as a short hand for ls -la which lists all files in the long listing format.

You can identity exact path or identity of the command using type command. For example:

     $ type ls
     ls is hashed (/bin/ls)
     $ type la
     la is aliased to `ls -la'
     $ type echo
     echo is a shell builtin
     $ type file
     file is /usr/bin/file

Here ls was recently searched while file was not, thus ls is "hashed", i.e., the shell has an internal record for the quick access to the location of the ls command.

Regular expressions are used in many text processing tools. They are analogous to the shell wildcards (see Shell wildcards, Section 4.3.8), but they are both more complicated and more powerful.

The regular expression describes the matching pattern and is made up of text characters and metacharacters. The metacharacter is just a character with a special meaning. There are 2 major styles, BRE and ERE, depending on the text tools as described in Unix-like text processing, Section 4.4.

For the EREs, the metacharacters include "\ . [ ] ^ $ * + ? ( ) { } |". The regular expression means:

• c

  • This matches the non-metacharacter "c".

• \c

  • This matches the literal character "c".

• .

  • This matches any character including newline.

• ^

  • This matches the beginning of a string.

• $

  • This matches the end of a string.

• \<

  • This matches the beginning of a word.

• \>

  • This matches the end of a word.

• [abc...]

  • This character list matches any of the characters "abc...".

• [^abc...]

  • This negated character list matches any of the characters except "abc...".

• r*

  • This matches zero or more regular expressions identified by "r".

• r+

  • This matches one or more regular expressions identified by "r".

• r?

  • This matches zero or one regular expressions identified by "r".

• r1|r2

  • This matches one of the regular expressions identified by "r1" or "r2".

• (r1|r2)

  • This matches one of the regular expressions identified by "r1" or "r2" and treats it as a bracketed regular expression.

In BREs the metacharacters "+ ? ( ) { } |" lose their special meaning; instead use the backslashed versions "\+ \? \( \) \{ \} \|". Thus the grouping construct (r1|r2) needs to be quoted as \(r1|r2\) in BREs. Since emacs, although being basically BRE, treats "+ ?" as the metacharacters. Thus there are no needs to quote them. See Replacement expressions, Section 4.4.2 for how the grouping construct is used.

For example, grep can be used to perform the text search using the regular expression:

     $ egrep 'GNU.*LICENSE|Yoyodyne' /usr/share/common-licenses/GPL
                         GNU GENERAL PUBLIC LICENSE
                         GNU GENERAL PUBLIC LICENSE
       Yoyodyne, Inc., hereby disclaims all copyright interest in the program

For the replacement expression, following characters have special meanings:

• &

  • This represents what the regular expression matched. (use \& in emacs)

• \n

  • This represents what the n-th bracketed regular expression matched.

For Perl replacement string, $n is used instead of \n and & has no special meaning.

For example:

     $ echo zzz1abc2efg3hij4 | \
       sed -e 's/\(1[a-z]*\)[0-9]*\(.*\)$/=&=/'
     zzz=1abc2efg3hij4=
     $ echo zzz1abc2efg3hij4 | \
       sed -e 's/\(1[a-z]*\)[0-9]*\(.*\)$/\2===\1/'
     zzzefg3hij4===1abc
     $ echo zzz1abc2efg3hij4 | \
       perl -pe 's/(1[a-z]*)[0-9]*(.*)$/$2===$1/'
     zzzefg3hij4===1abc
     $ echo zzz1abc2efg3hij4 | \
       perl -pe 's/(1[a-z]*)[0-9]*(.*)$/=&=/'
     zzz=&=

Here please pay extra attention to the style of the bracketed regular expression and how the matched strings are used in the text replacement process on different tools.

These regular expressions can be used for the cursor movements and the text replacement actions in the editors too.

Please read all the related manual pages to learn these commands.

Here are the basics:

• Filenames are case sensitive. That is, MYFILE and MyFile are different files.

• The root directory is referred to as simply /. Don't confuse this "root" with the root user. See Login to a shell prompt as root, Section 4.1.1.

• Every directory has a name which can contain any letters or symbols except /. [31] The root directory is an exception; its name is / (pronounced "slash" or "the root directory") and it cannot be renamed.

• Each file or directory is designated by a fully-qualified filename, absolute filename, or path, giving the sequence of directories which must be passed through to reach it. The three terms are synonymous. All absolute filenames begin with the / directory, and there's a / between each directory or file in the filename. The first / is the name of a directory, but the others are simply separators to distinguish the parts of the filename.

  The words used here can be confusing. Take the following example:

       /usr/share/keytables/us.map.gz
  

  This is a fully-qualified filename; some people call it a path. However, people will also refer to us.map.gz alone as a filename. [32]

• The root directory has a number of branches, such as /etc/ and /usr/. These subdirectories in turn branch into still more subdirectories, such as /etc/init.d/ and /usr/local/. The whole thing together is called the directory tree.

  You can think of an absolute filename as a route from the base of the tree (/) to the end of some branch (a file). You'll also hear people talk about the directory tree as if it were a family tree: thus subdirectories have parents, and a path shows the complete ancestry of a file.

  There are also relative paths that begin somewhere other than the root directory. You should remember that the directory ../ refers to the parent directory.

• There's no directory that corresponds to a physical device, such as your hard disk. This differs from CP/M, DOS, and Windows, where all paths begin with a device name such as C:\. See The filesystem concept in Debian, Section 4.5.2.

The detailed best practices for the file hierarchy are described in the Filesystem Hierarchy Standard. You should remember the following facts as the starter:

• /

  • A simple / represents the root directory.

• /etc/

  • This is the place for the system wide configuration files.

• /var/log/

  • This is the place for the system log files.

• /home/

  • This is the directory which contains all the home directories for all non-privileged users.

Following the Unix tradition, the Debian system provides the filesystem under which physical data on harddisks and other storage devices, and the interaction with the hardware devices such as console screens and remote serial consoles are represented in an unified manner.

Each file, directory, named pipe, or physical device on a Debian system has a data structure called an inode which describes its associated attributes such as the user who owns it (owner), the group that it belongs to, the time last accessed, etc. See /usr/include/linux/fs.h for the exact definition of struct inode in the Debian GNU/Linux system.

This unified representation of physical entities is very powerful since this allows us to use the same command for the same kind of operation on many totally different devices.

All your files could be on one disk --- or you could have 20 disks, some of them connected to a different computer elsewhere on the network. You can't tell just by looking at the directory tree, and nearly all commands work just the same way no matter what physical device(s) your files are really on.

File and directory access permissions are defined separately for the following three categories of affected users:

• the user who owns the file (u),

• other users in the group which the file belongs to (g), and

• all other users (o).

For a file, each corresponding permission allows:

• read (r): to examine contents of the file,

• write (w): to modify the file, and

• execute (x): to run the file as a command.

For a directory, each corresponding permission allows:

• read (r): to list contents of the directory,

• write (w): to add or remove files in the directory, and

• execute (x): to access files in the directory.

Here, execute permission on the directory means not only to allow reading of files in its directory but also to allow viewing their attributes, such as the size and the modification time.

To display permission information (and more) for files and directories, ls is used. See ls(1). When ls invoked with the -l option, it displays the following information in the order given:

• the type of file (first character)

  • -: normal file

  • d: directory

  • l: symlink

  • c: character device node

  • b: block device node

  • p: named pipe

  • s: socket

• the file's access permissions (the next nine characters, consisting of three characters each for user, group, and other in this order)

• the number of hard links to the file

• the name of the user who owns the file

• the name of the group which the file belongs to

• the size of the file in characters (bytes)

• the date and time of the file (mtime)

• the name of the file.

To change the owner of the file, chown is used from the root account. To change the group of the file, chgrp is used from the file's owner or root account. To change file and directory access permissions, chmod is used from the file's owner or root account. Basic syntax to manipulate foo file is:

     # chown newowner foo
     # chgrp newgroup foo 
     # chmod  [ugoa][+-=][rwx][,...] foo

See chown(1), chgrp(1), and chmod(1) for the detail.

For example, in order to make a directory tree to be owned by a user foo and shared by a group bar, issue the following commands from the root account:

     # cd /some/location/
     # chown -R foo:bar .
     # chmod -R ug+rwX,o=rX .

There are three more special permission bits:

• set user ID (s or S instead of user's x),

• set group ID (s or S instead of group's x), and

• sticky bit (t or T instead of other's x).

Here the output of ls -l for these bits is capitalized if execution bits hidden by these outputs are unset.

Setting set user ID on an executable file allows a user to execute the executable file with the owner ID of the file (for example root). Similarly, setting set group ID on an executable file allows a user to execute the executable file with the group ID of the file (for example root). Because these settings can cause security risks, enabling them requires extra caution.

Setting set group ID on a directory enables the BSD-like file creation scheme where all files created in the directory belong to the group of the directory.

Setting the sticky bit on a directory prevents a file in the directory from being removed by a user who is not the owner of the file. In order to secure the contents of a file in world-writable directories such as /tmp or in group-writable directories, one must not only set write permission off for the file but also set the sticky bit on the directory. Otherwise, the file can be removed and a new file can be created with the same name by any user who has write access to the directory.

Here are a few interesting examples of the file permissions.

     $ ls -l /etc/passwd /etc/shadow /dev/ppp /usr/sbin/pppd
     crw-rw----    1 root     dip      108,   0 Jan 18 13:32 /dev/ppp
     -rw-r--r--    1 root     root         1051 Jan 26 08:29 /etc/passwd
     -rw-r-----    1 root     shadow        746 Jan 26 08:29 /etc/shadow
     -rwsr-xr--    1 root     dip        234504 Nov 24 03:58 /usr/sbin/pppd
     $ ls -ld /tmp /var/tmp /usr/local /var/mail /usr/src
     drwxrwxrwt    4 root     root         4096 Feb  9 16:35 /tmp
     drwxrwsr-x   10 root     staff        4096 Jan 18 13:31 /usr/local
     drwxrwsr-x    3 root     src          4096 Jan 19 08:36 /usr/src
     drwxrwsr-x    2 root     mail         4096 Feb  2 22:19 /var/mail
     drwxrwxrwt    3 root     root         4096 Jan 25 02:48 /var/tmp

There is an alternative numeric mode to describe file permissions in chmod(1) commands. This numeric mode uses 3 to 4 digit wide octal (radix=8) numbers. Each digit corresponds to:

• 1st optional digit: sum of set user ID (=4), set group ID (=2), and sticky bit (=1)

• 2nd digit: sum of read (=4), write (=2), and execute (=1) permissions for user

• 3rd digit: ditto for group

• 4th digit: ditto for other

This sounds complicated but it is actually quite simple. If you look at the first few (2-10) columns from ls -l command output and read it as a binary (radix=2) representation of file permissions ("-" being "0" and "rwx" being "1"), this numeric mode value should make sense as an octal (radix=8) representation of file permissions to you. [33] For example, try:

     $ touch foo bar
     $ chmod u=rw,go=r foo
     $ chmod 644 bar
     $ ls -l foo bar
     -rw-r--r--    1 penguin  penguin  0 Nov  3 23:30  foo
     -rw-r--r--    1 penguin  penguin  0 Nov  3 23:30  bar

The default file permission mask can be set by using the umask shell built-in command. See builtins(7).

There are three types of timestamps for a GNU/Linux file:

• mtime: the modification time (ls -l),

• ctime: the status change time (ls -lc), and

• atime: the last access time (ls -lu).

Note that ctime is not file creation time.

• Overwriting a file will change all of mtime, ctime, and atime of the file.

• Changing permission or owner of a file will change ctime and atime of the file.

• Reading a file will change atime of the file.

Note that even simply reading a file on the Debian system will normally cause a file write operation to update atime information in the inode. Mounting a filesystem with the noatime option will let the system skip this operation and will result in faster file access for the read. See mount(8).

Use touch(1) command to change timestamps of existing files.

There are two methods of associating a file foo with a different filename bar.

• a hard link is a duplicate name for an existing file (ln foo bar),

• a symbolic link, or "symlink", is a special file that points to another file by name (ln -s foo bar).

See the following example for the changes in link counts and the subtle differences in the result of the rm command.

     $ echo "Original Content" > foo
     $ ls -l foo
     -rw-r--r--    1 osamu    osamu           4 Feb  9 22:26 foo
     $ ln foo bar     # hard link
     $ ln -s foo baz  # symlink
     $ ls -l foo bar baz
     -rw-r--r--    2 osamu    osamu           4 Feb  9 22:26 bar
     lrwxrwxrwx    1 osamu    osamu           3 Feb  9 22:28 baz -> foo
     -rw-r--r--    2 osamu    osamu           4 Feb  9 22:26 foo
     $ rm foo
     $ echo "New Content" > foo
     $ cat bar
     Original Content
     $ cat baz
     New Content

The symlink always has nominal file access permissions of "rwxrwxrwx", as shown in the above example, with the effective access permissions dictated by the permissions of the file that it points to.

The . directory links to the directory that it appears in, thus the link count of any new directory starts at 2. The .. directory links to the parent directory, thus the link count of the directory increases with the addition of new subdirectories.

A named pipe is a file that acts like a pipe. You put something into the file, and it comes out the other end. Thus it's called a FIFO, or First-In-First-Out: the first thing you put in the pipe is the first thing to come out the other end.

If you write to a named pipe, the process which is writing to the pipe doesn't terminate until the information being written is read from the pipe. If you read from a named pipe, the reading process waits until there's something to read before terminating. The size of the pipe is always zero --- it doesn't store data, it just links two processes like the shell |. However, since this pipe has a name, the two processes don't have to be on the same command line or even be run by the same user.

You can try it by doing the following:

     $ cd; mkfifo mypipe
     $ echo "hello" >mypipe & # put into background
     [1] 5952
     $ ls -l mypipe
     prw-r--r--    1 penguin penguin  0 2003-11-06 23:18 mypipe
     $ cat mypipe
     hello
     [1]+  Done                    echo hello >mypipe
     $ ls mypipe
     prw-r--r--    1 penguin penguin  0 2003-11-06 23:20 mypipe
     $ rm mypipe

The socket is similar to the named pipe (FIFO) and allows processes to exchange information. For the socket, those processes do not need to be running at the same time nor need to be the children of the same ancestor process. This is the endpoint for the inter process communication. The exchange of information may occur over the network between different hosts.

The /proc filesystem is a pseudo-filesystem and contains information about the system and running processes.

People frequently panic when they notice one file in particular - /proc/kcore - which is generally huge. This is (more or less) a copy of the contents of your computer's memory. It's used to debug the kernel. It doesn't actually exist anywhere, so don't worry about its size.

See Tuning the kernel through the proc filesystem, Section 7.2 and proc(5).

The X Window System can be started automatically with xdm-like graphical login daemon or type following from the console.

     $ exec startx

Since X environment can accommodate many window managers, their user interfaces vary quite a bit. Please remember that right-clicking the root window will bring up menu selections. This is always available.

• To gain the shell command prompt, start Xterm from menu:

  • "XShells" --> "XTerm".

• For graphical browsing of the web pages, start Mozilla from menu:

  • "Apps" --> "Net" --> "Mozilla Navigator".

• For graphical browsing of the PDF files, start Xpdf from menu:

  • "Apps" --> "Viewers" --> "Xpdf".

If you do not find menu entry, install the pertinent packages. See Beginning Debian package management, Section 6.2.

Followings are the important keystrokes to remember when running the X Window System.

• Ctrl-Alt-F1 through F6: Switch to other pseudo-terminals (from an X window, DOSEMU, etc.)

• Alt-F7: Switch back to X window

• Ctrl-Alt-minus: Change screen resolution in X window (minus refers to the keys on the numeric keypad)

• Ctrl-Alt-plus: Change screen resolution opposite way in X window (plus refers to the keys on the numeric keypad)

• Ctrl-Alt-Backspace: Terminate the X Server program

• Alt-X, Alt-C, Alt-V: Usual Windows/Mac Cut, Copy, Paste keys combinations with Ctrl- keys are replaced by these Alt- keys in some programs such as Netscape Composer.

If a system has many packages which include -dev packages, etc., the following method using dselect is recommended for fine-grained package control.

     # dselect update  # always do this before upgrade
     # dselect select  # select additional packages

All your current packages will be selected when dselect starts. dselect may prompt you with additional packages based on Depends, Suggests, and Recommends. If you do not want to add any packages, just type Q to exit dselect again.

     # dselect install

You will have to answer some package configuration questions during this part of the process, so have your notes ready and allow some time for this part. See dselect, Section 6.2.4.

Use dselect. It always works :)

     # apt-get update
     # apt-get -t stable upgrade
     # apt-get -t stable dist-upgrade
     # apt-get -t testing upgrade
     # apt-get -t testing dist-upgrade
     # apt-get -t unstable upgrade
     # apt-get -t unstable dist-upgrade

Once your system has reached Sarge it is advisable to use aptitude instead of apt-get. (aptitude accepts many of the options that apt-get accepts, including those above.)

To upgrade and stay with current dselect settings:

     # apt-get dselect-upgrade

See Package dependencies, Section 2.2.8.

     dpkg      – Debian package file installer
     apt-get   – Command line front end for APT
     aptitude  – Advanced text and command line front end for APT
     synaptic  – Gtk GUI front end for APT
     dselect   – Menu-driven package manager
     tasksel   – Task installer

These tools aren't all alternatives to one another. For example, dselect uses both APT and dpkg.

APT uses /var/lib/apt/lists/* for tracking available packages while dpkg uses /var/lib/dpkg/available. If you have installed packages using aptitude or other APT front ends and you want to use dselect to install packages then the first thing you should do is update /var/lib/dpkg/available by selecting [U]pdate from dselect's menu (or by running "dselect update").

apt-get automatically installs all packages upon which a requested package Depends. It does not install the packages that a requested package merely Recommends or Suggests.

aptitude, in contrast, can be configured to install packages that a requested package Recommends or Suggests.

dselect presents the user with a list of packages that a selected package Recommends or Suggests and allows these to be selected or deselected individually. See Package dependencies, Section 2.2.8.

     dpkg-reconfigure  - reconfigure an already installed package
                         (if it uses debconf)
     dpkg-source       - manage source package file
     dpkg-buildpackage - automate the building of a package file
     apt-cache         - check package archive in local cache

Set up sources.list as described in Preparing for upgrade, Section 5.2. [34] Also refer to Debian System installation hints, Chapter 3, Upgrading a distribution to stable, testing, or unstable, Chapter 5, and Rescue editors, Section 11.2.

You can install sets of packages typically required in order to put a Debian system to a certain use. These sets of packages are called "tasks".

The simplest way to install tasks at the time of initial installation is to use tasksel. Note that you must run

     dselect update

before using it.

aptitude can also install tasks and is the tool recommended for this purpose. It enables you to deselect individual packages within tasks before proceeding to the installation step.

aptitude is a new menu-driven package installer similar to dselect but built from scratch on top of APT. It can be used as an alternative to apt-get for most commands. See aptitude(1) and file:///usr/share/doc/aptitude/README.

Once you start using aptitude it is best to continue using it rather than alternative methods of installing packages; otherwise you lose the advantage of aptitude keeping track of which packages you have deliberately installed.

aptitude in full screen mode accepts single-key commands which are usually lowercase. Notable key strokes are:

     Keystroke   Action
     F10         Menu
     ?           Help for keystroke (complete listing)
     u           Update package archive information
     +           Mark the package to be upgraded or newly installed
     -           Mark the package to be removed (keep config)
     _           Mark the package to be purged (remove config)
     =           Place the package on hold
     U           Mark all upgradable packages to be upgraded
     g           Download and install selected packages
     q           Quit current screen and save changes
     x           Quit current screen and discard changes
     Enter       View information about a package
     C           View a package's changelog
     l           Change the limit for the displayed packages
     /           Search for the first match
     \           Repeat the last search

Like apt-get, aptitude installs packages upon which a selected package Depends. aptitude also offers the option to pull in packages that a to-be-installed package Recommends or Suggests. You can change the default behavior by choosing F10 -> Options -> Dependency handling in its menu.

Other advantages of aptitude are:

• aptitude offers access to all versions of a package.

• aptitude logs its actions in /var/log/aptitude.

• aptitude makes it easy to keep track of obsolete software by listing under "Obsolete and Locally Created Packages".

• aptitude includes a fairly powerful system for searching particular packages and limiting the package display. Users familiar with mutt will pick up quickly, as mutt was the inspiration for the expression syntax. See "SEARCHING, LIMITING, AND EXPRESSIONS" in file:///usr/share/doc/aptitude/README.

• aptitude in full screen mode has su functionality embedded and can be run from normal user until you really need administrative privileges.

In stable releases up to and including Potato, dselect was the principal package maintenance tool. For Sarge, you should consider using aptitude instead.

When started, dselect automatically selects all "Required", "Important", and "Standard" packages.

dselect has a somewhat strange user interface. Most people get used to it, however. It has four commands (Capital means CAPITAL!):

     Key-stroke  Action
     Q           Quit. Confirm current selection and quit anyway. 
                 (override dependencies)
     R           Revert! I did not mean it.
     D           Damn it! I do not care what dselect thinks.  Just Do it!
     U           Set all to sUggested state

With D and Q, you can select conflicting selections at your own risk. Handle these commands with care.

Add a line containing the option "expert" in /etc/dpkg/dselect.cfg to reduce noise.

If your machine runs dselect slowly then you might consider running dselect on another (faster) machine in order to determine the packages you want to install, then use apt-get install on the slow machine to install them.

To track the testing distribution as it changes, make your /etc/apt/preferences file look like this:

     Package: *
     Pin: release a=testing
     Pin-Priority: 800
     
     Package: *
     Pin: release a=stable
     Pin-Priority: 600

Note that tracking the testing distribution can have the side effect of delaying the installation of packages containing security fixes. Such packages are uploaded to unstable and migrate to testing only after a delay.

See apt_preferences(5) for more complicated examples which will allow you, for example, to track testing while installing selected packages from unstable.

Examples which lock particular packages at particular versions while tracking other packages as they are released are available in the examples subdirectory as preferences.testing and preferences.unstable.

If you mix distributions, e.g., testing with stable or unstable with stable, you will eventually pull in core packages such as libc6 from testing or unstable and there is no guarantee that these will not contain bugs. You have been warned.

Another example, preferences.stable, forces all packages to be downgraded to stable.

Downgrading from a later release of a package to an earlier one is not officially supported in Debian. However, you may find that you have to downgrade a specific package in order to re-install a version of a package that works when a new version malfunctions. You may find these previous package files locally in /var/cache/apt/archives/ or remotely at http://snapshot.debian.net/. See also Rescue using dpkg, Section 6.3.3.

Downgrading from a later release of a distribution to an earlier one is not officially supported either and is very likely to cause problems. However, this may be worth trying as a last resort if you are desperate.

While tracking testing as described in the above example you can manage the system by using the following commands:

• aptitude update (or apt-get update)

  These update the list of available packages at the repositories.

• aptitude upgrade (or apt-get upgrade or aptitude dist-upgrade or apt-get dist-upgrade)

  These track the testing distribution — they upgrade each package on the system, after installing versions of packages upon which it Depends, from the testing distribution. [35]

• apt-get dselect-upgrade

  This tracks the testing distribution — it upgrades each package on the system according to the selections of dselect.

• aptitude install package/unstable

  This installs package from the unstable distribution while installing its dependencies from the testing distribution.

• aptitude install -t unstable package

  This installs package from the unstable distribution while installing its dependencies also from the unstable distribution by setting the Pin-Priority of unstable to 990.

• apt-cache policy foo bar ...

  This checks the status of packages foo bar ....

• aptitude show foo bar ... | less (or apt-cache show foo bar ... | less)

  This checks the information for packages foo bar ....

• aptitude install foo=2.2.4-1

  This installs the particular version 2.2.4-1 of the foo package.

• aptitude install foo bar-

  This installs the foo package and removes the bar package

• aptitude remove bar

  This removes the bar package but not its configuration files.

• aptitude purge bar

  This removes the bar package together with all its configuration files.

In the above examples, giving apt-get the -u option causes it to print a list of all packages that are to be upgraded and to prompt the user before taking action. aptitude does this by default. The following makes apt-get always do this:

     $ cat >> /etc/apt/apt.conf << .
     // Always show packages to be upgraded (-u)
     APT::Get::Show-Upgraded "true";
     .

Use the --no-act option to simulate actions without actually installing, removing, etc., any packages.

If you are experiencing problems with a specific package, make sure to check out these sites first before you seek help or file a bug report. (lynx, links, and w3m work equally well):

     $ lynx http://bugs.debian.org/
     $ lynx http://bugs.debian.org/package-name  # if you know package name
     $ lynx http://bugs.debian.org/bugnumber     # if you know bug number

Search Google (www.google.com) with search words including "site:debian.org".

When in doubt, read the fine manual. Set CDPATH as follows:

     export CDPATH=.:/usr/local:/usr/share/doc

and type

     $ cd packagename
     $ pager README.Debian # if this exists
     $ mc

More support resources are listed at Support for Debian, Chapter 15.

Package dependency problems may occur when upgrading in unstable or testing as described in Upgrading, Section 5.3. Most of the time this is because a package that will be upgraded Depends on a package that is not yet available. These problems are fixed by using

     # aptitude dist-upgrade

If this does not work, then repeat one of the following until the problem resolves itself:

     # aptitude -f upgrade        # continue upgrade even after error
     ... or
     # aptitude -f dist-upgrade   # continue dist-upgrade even after error

Some really broken upgrade scripts may cause persistent trouble. It is usually better to resolve this type of situation by inspecting the /var/lib/dpkg/info/packagename.{post,pre}{inst,rm} scripts of the offending package and then running:

     # dpkg --configure -a    # configures all partially installed packages

If a script complains about a missing configuration file, look in /etc/ for the corresponding configuration file. If one exists with an extension of .dpkg-new (or something similar), mv it to remove the suffix.

Package dependency problems may occur when installing in unstable or testing. There are ways to circumvent dependencies.

     # aptitude -f install package # override broken dependencies

An alternative method to fix these situations is to use the equivs package. See file:///usr/share/doc/equivs/README.Debian and The equivs package, Section 6.5.2.

If you reach a dead end using APT you can download package files from Debian mirrors and install them using dpkg. If you do have not access to the network you can look for cached copies of package files in /var/cache/apt/archives/.

     # dpkg -i fetchmail_6.2.5-4_i386.deb

If attempting to install a package this way fails due to dependency violations and you really need to install the package then you can override dependency checks using dpkg's --ignore-depends, --force-depends and other options. See dpkg(8) for details.

If /var/lib/dpkg/status becomes corrupt for any reason, the Debian system loses package selection data and suffers severely. Look for the old /var/lib/dpkg/status file at /var/lib/dpkg/status-old or /var/backups/dpkg.status.*.

Keeping /var/backups/ in a separate partition may be a good idea since this directory contains lots of important system data.

If no old /var/lib/dpkg/status file is available, you can still recover information from directories in /usr/share/doc/.

     # ls /usr/share/doc | \
       grep -v [A-Z] | \
       grep -v '^texmf$' | \
       grep -v '^debian$' | \
       awk '{print $1 " install"}' | \
       dpkg --set-selections
     # dselect --expert # reinstall system, de-select as needed

Since the /var directory contains regularly updated data such as mail, it is more susceptible of corruption than, e.g., /usr/. Putting /var/ on a separate partition reduces risks. If disaster happens, you may have to rebuild the /var directory to rescue your Debian system.

Obtain the skeleton content of the /var directory from a minimum working Debian system based on the same or older Debian version, for example var.tar.gz, and place it in the root directory of the broken system. Then

     # cd /
     # mv var var-old      # if any useful contents are left
     # tar xvzf var.tar.gz # use Woody skeleton file
     # aptitude            # or dselect

This should provide a working system. You can expedite the recovery of package selections by using the technique described in Recover package selection data, Section 6.3.4. ([FIXME]: This procedure needs more experiments to verify.)

Boot into Linux using a Debian rescue floppy/CD or an alternative partition in a multiboot Linux system. See Booting the system, Section 8.1. Mount the unbootable system on /target and use the chroot install mode of dpkg.

     # dpkg --root /target -i packagefile.deb

Then configure and fix problems.

By the way, if a broken lilo is all that prevents booting, you can boot using a standard Debian rescue disk. At boot prompt, assuming the root partition of your Linux installation is in /dev/hda12 and you want runlevel 3, enter:

     boot: rescue root=/dev/hda12 3

Then you are booted into an almost fully functional system with the kernel on floppy disk. (There may be minor glitches due to lack of kernel features or modules.)

A broken dpkg may make it impossible to install any .deb files. A procedure like the following will help you recover from this situation. (In the first line, you can replace "links" with your favorite browser command.)

     $ links http://http.us.debian.org/debian/pool/main/d/dpkg/
       ... download the good dpkg_version_arch.deb
     $ su
     password: *****
     # ar x dpkg_version_arch.deb
     # mv data.tar.gz /data.tar.gz
     # cd /
     # tar xzfv data.tar.gz

For i386, http://packages.debian.org/dpkg may also be used as the URL.

To find the package to which a particular filename pattern belongs in the installed packages:

     $ dpkg {-S|--search} pattern

Or to find the similar in the Debian archive:

     $ wget http://ftp.us.debian.org/debian/dists/sarge/Contents-i386.gz
     $ zgrep -e pattern Contents-i386.gz

Or use specialized package commands:

     # aptitude install dlocate  
     $ dlocate filename         # fast alternative to dpkg -L and dpkg -S
     ...
     # aptitude install auto-apt # on-demand package installation tool
     # auto-apt update          # create db file for auto-apt
     $ auto-apt search pattern  
                     # search for pattern in all packages, installed or not

Search and display information from package archives. Make sure to point APT to the proper archive(s) by editing /etc/apt/sources.list. If you want to see how packages in testing/unstable do against the currently installed one, use apt-cache policy—quite nice.

     # apt-get   check           # update cache and check for broken packages
     $ apt-cache search  pattern # search package from text description
     $ apt-cache policy  package # package priority/dists information
     $ apt-cache show -a package # show description of package in all dists
     $ apt-cache showsrc package # show description of matching source package
     $ apt-cache showpkg package # package information for debugging
     # dpkg  --audit|-C          # search for partially installed packages
     $ dpkg {-s|--status} package ... # description of installed package
     $ dpkg -l package ...       # status of installed package (1 line each)
     $ dpkg -L package ...       # list filenames installed by the package

apt-cache showsrc is not documented as of the Woody release but works :)

You can also find package information in (I use mc to browse these):

     /var/lib/apt/lists/*
     /var/lib/dpkg/available

The comparison of the following files provides information on what exactly has happened in the last few install sessions.

     /var/lib/dpkg/status
     /var/backups/dpkg.status*

For an unattended installation, add the following line in /etc/apt/apt.conf:

     Dpkg::Options {"--force-confold";}

This equivalent to running aptitude -y install packagename or apt-get -q -y install packagename. Because this automatically answers "yes" to all prompts, it may cause problems, so use this trick with care. See apt.conf(5) and dpkg(1).

You can configure any particular packages later by following Reconfigure installed packages, Section 6.4.4.

Use the following to reconfigure any already-installed package.

     # dpkg-reconfigure --priority=medium package [...]
     # dpkg-reconfigure --all   # reconfigure all packages
     # dpkg-reconfigure locales # generate any extra locales
     # dpkg-reconfigure --p=low xserver-xfree86 # reconfigure X server

Do this for debconf if you need to change the debconf dialog mode permanently.

Some programs come with special configuration scripts. [36]

     apt-setup     - create /etc/apt/sources.list
     install-mbr   - install a Master Boot Record manager
     tzconfig      - set the local time zone
     gpmconfig     - set gpm mouse daemon
     eximconfig    - configure Exim (MTA)
     texconfig     - configure teTeX
     apacheconfig  - configure Apache (httpd)
     cvsconfig     - configure CVS
     sndconfig     - configure sound system
     ...
     update-alternatives - set default command, e.g., vim as vi
     update-rc.d         - System-V init script management
     update-menus        - Debian menu system

Remove a package while maintaining its configuration:

     # aptitude remove package ...
     # dpkg   --remove package ...

Remove a package and all configuration:

     # aptitude purge  package ...
     # dpkg   --purge  package ...

For example, holding of libc6 and libc6-dev for dselect and aptitude install package can be done as follows:

     # echo -e "libc6 hold\nlibc6-dev hold" | dpkg --set-selections

aptitude install package will not be hindered by this "hold". To hold a package through forcing automatic downgrade for aptitude upgrade package or aptitude dist-upgrade, add the following to /etc/apt/preferences:

     Package: libc6
     Pin: release a=stable
     Pin-Priority: 2000

Here the "Package:" entry cannot use entries such as "libc6*". If you need to keep all binary packages related to the glibc source package in a synchronized version, you need to list them explicitly.

The following will list packages on hold:

     dpkg --get-selections "*"|grep -e "hold$"

apt-show-versions can list available package versions by distribution.

     $ apt-show-versions | fgrep /testing | wc
     ... how many packages you have from testing
     $ apt-show-versions -u
     ... list of upgradeable packages
     $ aptitude install `apt-show-versions -u -b | fgrep /unstable`
     ... upgrade all unstable packages to their newest versions

Package installation with APT leaves cached package files in /var/cache/apt/archives/ and these need to be cleaned.

     # aptitude autoclean # removes only useless package files
     # aptitude clean     # removes all cached package files

To make a local copy of the package selection states:

     # dpkg --get-selections "*" >myselections   # or use \*
     # debconf-get-selections > debconfsel.txt

"*" makes myselections include package entries for "purge" too.

You can transfer this file to another computer, and install it there with:

     # dselect update
     # debconf-set-selections < debconfsel.txt
     # dpkg --set-selections <myselections
     # apt-get -u dselect-upgrade    # or dselect install

For partial upgrades of the stable system, rebuilding a package within its environment using the source package is desirable. This avoids massive package upgrades due to their dependencies. First, add the following entries to /etc/apt/sources.list:

     deb-src http://http.us.debian.org/debian testing \
      main contrib non-free
     deb-src http://http.us.debian.org/debian unstable \
      main contrib non-free

Here each entry for deb-src is broken into two lines because of printing constraints, but the actual entry in sources.list should consist of a single line.

Then get the source and make a local package:

     $ apt-get update  # update the source package search list
     $ apt-get source package
     $ dpkg-source -x package.dsc
     $ cd package-version
       ... inspect required packages (Build-Depends in .dsc file) and
           install them too.  You need the "fakeroot" package also.
     
     $ dpkg-buildpackage -rfakeroot 
     
       ...or (no sig)
     $ dpkg-buildpackage -rfakeroot -us -uc # use "debsign" later if needed
     
       ...Then to install
     $ su -c "dpkg -i packagefile.deb"

Usually, one needs to install a few packages with the "-dev" suffix to satisfy package dependencies. debsign is in the devscripts package. auto-apt may ease satisfying these dependencies. Use of fakeroot avoids unnecessary use of the root account.

In Woody, these dependency issues can be simplified. For example, to compile a source-only pine package:

     # apt-get build-dep pine
     # apt-get source -b pine

In order to create a local package archive which is compatible with APT and the dselect system, Packages needs to be created and package files need to be populated in a particular directory tree.

A local deb repository similar to an official Debian archive can be made in this way:

     # aptitude install dpkg-dev
     # cd /usr/local
     # install -d pool # physical packages are located here
     # install -d dists/unstable/main/binary-i386
     # ls -1 pool | sed 's/_.*$/ priority section/' | uniq > override
     # editor override # adjust priority and section
     # dpkg-scanpackages pool override /usr/local/ \
        > dists/unstable/main/binary-i386/Packages
     # cat > dists/unstable/main/Release << EOF
     Archive: unstable
     Version: 3.0
     Component: main
     Origin: Local
     Label: Local
     Architecture: i386
     EOF
     # echo "deb file:/usr/local unstable main" \
        >> /etc/apt/sources.list

Alternatively, a quick-and-dirty local deb repository can be made:

     # aptitude install dpkg-dev
     # mkdir /usr/local/debian
     # mv /some/where/package.deb /usr/local/debian
     # dpkg-scanpackages /usr/local/debian /dev/null | \
       gzip - > /usr/local/debian/Packages.gz
     #  echo "deb file:/usr/local/debian ./" >> /etc/apt/sources.list

These archives can be remotely accessed by providing access to these directories through either HTTP or FTP methods and changing entries in /etc/apt/sources.list accordingly.

alien enables the conversion of binary packages provided in Red Hat rpm, Stampede slp, Slackware tgz, and Solaris pkg file formats into a Debian deb package. If you want to use a package from another Linux distribution than the one you have installed on your system, you can use alien to convert it to your preferred package format and install it. alien also supports LSB packages.

auto-apt is an on-demand package installation tool.

     $ sudo auto-apt update
      ... update database
     $ auto-apt -x -y run
     Entering auto-apt mode: /bin/bash
     Exit the command to leave auto-apt mode.
     $ less /usr/share/doc/med-bio/copyright # access non-existing file
      ...  Install the package which provide this file.
      ... Also install dependencies

debsums enables verification of installed package files against MD5 checksums. Some packages do not have available MD5 checksums. A possible temporary fix for sysadmins:

     # cat >>/etc/apt/apt.conf.d/90debsums
     DPkg::Post-Install-Pkgs {"xargs /usr/bin/debsums -sg";};
     ^D

per Joerg Wendland joergland@debian.org (untested).

In short, fancy efforts to create an optimized sources.list did not produce a significant improvement for me from a location in the USA. I manually chose a nearby site using apt-setup.

apt-spy creates sources.list automatically, based on latency and bandwidth. netselect-apt creates a more complete sources.list, but uses an inferior method of choosing the best mirror (ping time comparison).

     # aptitude install apt-spy
     # cd /etc/apt ; mv sources.list sources.list.org
     # apt-spy -d testing -l sources.apt

File diversions are a way of forcing dpkg not to install a file into its default location, but to a diverted location. Diversions can be used through the Debian package scripts to move a file away when it causes a conflict. System administrators can also use a diversion to override a package's configuration file, or whenever some files (which aren't marked as conffiles) need to be preserved by dpkg, when installing a newer version of a package which contains those files (see Preservation of local configuration, Section 2.2.4).

     # dpkg-divert [--add]  filename # add "diversion"
     # dpkg-divert --remove filename # remove "diversion"

It's usually a good idea not to use dpkg-divert unless it is absolutely necessary.

If you compile a program from source, it is best to make it into a real local debianized package (*.deb). Use equivs as a last resort.

     Package: equivs
     Priority: extra
     Section: admin
     Description: Circumventing Debian package dependencies
      This is a dummy package which can be used to create Debian
      packages, which only contain dependency information.

To make the command vi run vim, use update-alternatives:

     # update-alternatives --display vi
     ...
     # update-alternatives --config vi
       Selection    Command
     -----------------------------------------------
           1        /usr/bin/elvis-tiny
           2        /usr/bin/vim
     *+    3        /usr/bin/nvi
     
     Enter to keep the default[*], or type selection number: 2

Items in the Debian alternatives system are kept in /etc/alternatives/ as symlinks.

To set your favorite X Window environment, apply update-alternatives to /usr/bin/x-session-manager and /usr/bin/x-window-manager. For details, see Custom X sessions, Section 9.4.5.1.

/bin/sh is a direct symlink to /bin/bash or /bin/dash. It's safer to use /bin/bash to be compatible with old Bashism-contaminated scripts but better discipline to use /bin/dash to enforce POSIX compliance. Upgrading to a 2.4 Linux kernel tends to set this to /bin/dash.

When installed, most Debian packages configure their services to run in runlevels 2 through 5. Thus, there are no differences between runlevels 2, 3, 4 and 5 on a Debian system that has not been customized; Debian leaves it up to the local administrator to customize runlevels as described in Customizing runlevels, Section 2.4.3. This differs from the way runlevels are used by some other popular GNU/Linux distributions. One change you may want to make is to disable xdm or gdm in runlevel 2 so that the X display manager is not started at the end of the boot sequence; you can then start it by switching to runlevel 3.

For more information about runlevels see Runlevels, Section 2.4.2.

Debian developers take system security seriously. Many daemon services are installed with the fewest services and features enabled.

Run ps aux or check the contents of /etc/init.d/* and /etc/inetd.conf, if you have any doubts (about Exim, DHCP, ...). Also check /etc/hosts.deny as in Restricting logins with PAM, Section 9.2.1. The pidof command is also useful (see pidof(8)).

X11 doesn't allow TCP/IP (remote) connections by default in recent versions of Debian. See Using X over TCP/IP, Section 9.4.6. X forwarding in SSH is also disabled. See Connecting to a remote X server – ssh, Section 9.4.8.

Most "normal" programs don't need kernel headers and in fact may break if you use them directly; instead they should be compiled against the headers with which glibc was built, which are the versions in /usr/include/linux and /usr/include/asm of the Debian system.

So do not put symlinks to the directories in /usr/src/linux from /usr/include/linux and /usr/include/asm, as suggested by some outdated documents.

If you need particular kernel headers for some kernel-specific application programs, alter the makefile(s) so that their include path points to dir-of-particular-kernel-headers/include/linux and dir-of-particular-kernel-headers/include/asm.

The Linux kernel may complain "Too many open files". This is due to the small default value (8096) for file-max. To fix this problem, run the following command as root:

     # echo "65536"  > /proc/sys/fs/file-max  # for 2.2 and 2.4 kernel
     # echo "131072" > /proc/sys/fs/inode-max # for 2.2 kernel only

or put the following into /etc/sysctl.conf for the permanent change:

     file-max=65536   # for 2.2 and 2.4 kernel
     inode-max=131072 # for 2.2 kernel only

You can change disk flush intervals through the proc filesystem. The following will shorten its interval from the default five seconds to one second.

     # echo "40 0 0 0 100 30000 60 0 0"  > /proc/sys/vm/bdflush

This may negatively impact file I/O performance a little bit. But this secures file contents except for the last one second which is shorter than the default five seconds. This is true even for journaling filesystems.

For some old low memory systems, it may still be useful to enable over-commit of memory through the proc filesystem:

     # echo 1 > /proc/sys/vm/overcommit_memory

The udev is a dynamic replacement for /dev/. Device names can be chosen to be very short ones. The devfs used in the 2.4 kernel is now obsolete.

Installing the new Debian 2.6 kernel provided by kernel-image-2.6.NN with udev package will enable this.

It is possible to boot a system and log on to the root account without knowing the root password as long as one has access to the console keyboard. (This assumes there are no password requests from the BIOS or from a boot loader such as lilo that would prevent one from booting the system.)

This is a procedure which requires no external boot disks and no change in BIOS boot settings. Here, "Linux" is the label for booting the Linux kernel in the default Debian install.

At the lilo boot screen, as soon as boot: appears (you must press a shift key at this point on some systems to prevent automatic booting and when lilo uses the framebuffer you have to press TAB to see the options you type), enter:

     boot: Linux init=/bin/sh

This causes the system to boot the kernel and run /bin/sh instead of its standard init. Now you have gained root privileges and a root shell. Since / is currently mounted read-only and many disk partitions have not been mounted yet, you must do the following to have a reasonably functioning system.

     init-2.03# mount -n -o remount,rw /
     init-2.03# mount -avt nonfs,noproc,nosmbfs
     init-2.03# cd /etc
     init-2.03# vi passwd
     init-2.03# vi shadow

(If the second data field in /etc/passwd is "x" for every username, your system uses shadow passwords, and you must edit /etc/shadow.) To disable the root password, edit the second data field in the password file so that it is empty. Now the system can be rebooted and you can log on as root without a password. When booting into runlevel 1, Debian (at least after Potato) requires a password, which some older distributions did not.

It is a good idea to have a minimal editor in /bin/ in case /usr/ is not accessible (see Rescue editors, Section 11.2).

Also consider installing the sash package. When the system becomes unbootable, execute:

     boot: Linux init=/bin/sash

sash serves as an interactive substitute for sh even when /bin/sh is unusable. It's statically linked, and includes many standard utilities as built-ins (type "help" at the prompt for a reference list).

Boot from any emergency boot/root disk set. If /dev/hda3 is the original root partition, the following will let one edit the password file just as easily as the above.

     # mkdir fixit
     # mount /dev/hda3 fixit
     # cd fixit/etc
     # vi shadow
     # vi passwd

The advantage of this approach over the previous method is one does not need to know the lilo password (if any). But to use it one must be able to access the BIOS setup to allow the system to boot from floppy disk or CD, if that is not already set.

No problem, even if you didn't bother to make a boot disk during install. If lilo is broken, grab the boot disk from the Debian installation set and boot your system from it. At the boot prompt, assuming the root partition of your Linux installation is on /dev/hda12 and you want runlevel 3, enter:

     boot: rescue root=/dev/hda12 3

Then you are booted into an almost fully functional system using the kernel on the floppy. (There may be minor glitches due to lack of kernel features or modules.)

See also Install a package into an unbootable system, Section 6.3.6 if you have a broken system.

If you need a custom boot floppy, follow readme.txt on the rescue disk.

Chasing unstable/sid is fun, but buggy xdm, gdm, kdm, or wdm started during the boot process can bite you bad.

First get the root shell by entering the following at the boot prompt:

     boot: Linux vga=normal s

Here, Linux is the label for the kernel image you are booting; "vga=normal" will make sure lilo runs in normal VGA screen, and "s" (or "S") is the parameter passed to init to invoke single-user mode. Enter the root password at the prompt.

There are few ways to disable all the X starting daemons:

• run update-rc.d -f ?dm remove ; update-rc.d ?dm stop 99 1 2 3 4 5 6 .

• insert "exit 0" at the start of all /etc/init.d/?dm files.

• rename all /etc/rc2.d/S99?dm files to /etc/rc2.d/K99?dm.

• remove all /etc/rc2.d/S99?dm files.

• run :>/etc/X11/default-display-manager

Here, number in rc2.d must correspond to the runlevel specified in the /etc/inittab. Also ?dm means that you need to run the command multiple times by substituting it with all of the xdm, gdm, kdm, and wdm.

Only the first one in the list is "the one true way" in Debian. The last one is easy but only works on Debian and requires you to set the display manager again later using dpkg-reconfigure. Others are generic methods to disable daemons.

You can still start X with the startx command from any console shell.

The system can be booted into a particular runlevel and configuration using the lilo boot prompt. Details are given in the BootPrompt-HOWTO (LDP).

If you want to boot the system into runlevel 4, use the following input at the lilo boot prompt.

     boot: Linux 4

If you want to boot the system into normally functioning single-user mode and you know the root password, one of the following examples at the lilo boot prompt will work.

     boot: Linux S
     boot: Linux 1
     boot: Linux -s

If you want to boot the system with less memory than system actually has (say 48MB for a system with 64MB), use this input at the lilo boot prompt:

     boot: Linux mem=48M

Make sure not to specify more than the actual memory size here, otherwise the kernel will crash. If one has more than 64MB of memory, e.g. 128MB, unless one executes mem=128M at the boot prompt or includes a similar append line in /etc/lilo.conf, old kernels and/or a motherboard with an old BIOS will not use memory beyond 64MB.

GRUB is a new boot manager from the GNU Hurd project and is much more flexible than Lilo but has slightly different handling of boot parameters.

     grub> find /vmlinuz
     grub> root (hd0,0)
     grub> kernel /vmlinuz root=/dev/hda1
     grub> initrd /initrd
     grub> boot

Here, you must be aware of the Hurd device names:

     the Hurd/GRUB       Linux               MS-DOS/Windows
      (fd0)               /dev/fd0            A:
      (hd0,0)             /dev/hda1           C: (usually)
      (hd0,3)             /dev/hda4           F: (usually)
      (hd1,3)             /dev/hdb4           ?

See file:///usr/share/doc/grub/README.Debian.gz and file:///usr/share/doc/grub-doc/html/ for details.

System administration involves much more elaborate tasks in a Unix environment than in an ordinary personal computer environment. Make sure to know the most basic means of configuration in case you need to recover from system trouble. X11-based GUI configuration tools look nice and convenient but are often unsuitable in these emergency situations.

In this context, recording shell activities is a good practice, especially as root.

Emacs: Use M-x shell to start recording into a buffer, and use C-x C-w to write the buffer to a file.

Shell: Use the screen command with "^A H" as described in Console switching with screen, Section 8.6.28; or use the script command.

     $ script
     Script started, file is typescript
      ... do whatever ...
      Ctrl-D
     $ col -bx <typescript >savefile
     $ vi savefile

The following can be used instead of script:

     $ bash -i 2>&1 | tee typescript

If you need to record the graphic image of an X application, including an xterm display, use gimp (GUI). It can capture each window or the whole screen. Alternatives are xwd (xbase-clients), import (imagemagick), and scrot (scrot).

If you need to rearrange file structure, move content including file links by:

     Standard method:
     # cp -a /source/directory /dest/directory # requires GNU cp
     # (cd /source/directory && tar cf - . ) | \
             (cd /dest/directory && tar xvfp - )
     If a hard link is involved, a pedantic method is needed:
     # cd /path/to/old/directory
     # find . -depth -print0 | afio -p -xv -0a /mount/point/of/new/directory
     If remote:
     # (cd /source/directory && tar cf - . ) | \
             ssh user@host.dom (cd /dest/directory && tar xvfp - )
     If there are no linked files:
     # scp -pr user1@host1.dom:/source/directory \
               user2@host2.dom:/dest/directory

The following comparative information on copying a whole subdirectory was presented by Manoj Srivastava srivasta@debian.org to debian-user@lists.debian.org.

Traditionally, cp was not really a candidate for this task since it did not dereference symbolic links, or preserve hard links either. Another thing to consider was sparse files (files with holes).

GNU cp has overcome these limitations; however, on a non-GNU system, cp could still have problems. Also, you can't generate small, portable archives using cp.

     % cp -a . newdir

Tar overcame some of the problems that cp had with symbolic links. However, although cpio handles special files, traditional tar doesn't.

tar's way of handling multiple hard links to a file places only one copy of the link on the tape, but the name attached to that copy is the only one you can use to retrieve the file; cpio's way puts one copy for every link, but you can retrieve it using any of the names.

The tar command changed its option for .bz2 files between Potato and Woody, so use --bzip2 in scripts instead of its short form -I (Potato) or -j (Woody).

The new, POSIX (IEEE Std 1003.2-1992, pages 380–388 (section 4.48) and pages 936–940 (section E.4.48)), all-singing, all-dancing, Portable Archive Interchange utility. pax will read, write, and list the members of an archive file, and will copy directory hierarchies. pax operation is independent of the specific archive format, and supports a wide variety of different archive formats.

pax implementations are still new and wet behind the ears.

     # apt-get install pax
     $ pax -rw -p e . newdir
      or
     $ find . -depth  | pax -rw -p e  newdir

cpio copies files into or out of a cpio or tar archive. The archive can be another file on the disk, a magnetic tape, or a pipe.

     $ find . -depth -print0 | cpio --null --sparse -pvd new-dir

afio is a better way of dealing with cpio-format archives. It is generally faster than cpio, provides more diverse magnetic tape options and deals somewhat gracefully with input data corruption. It supports multivolume archives during interactive operation. afio can make compressed archives that are much safer than compressed tar or cpio archives. afio is best used as an "archive engine" in a backup script.

     $ find . -depth -print0 | afio -px -0a new-dir

All my backups onto tape use afio.

rdiff-backup offers nice and simple backup with differential history for any types of files, including symlinks. To back up most of ~/ to /mnt/backup:

     $ rdiff-backup --include ~/tmp/keep --exclude ~/tmp  ~/ /mnt/backup

To restore three-day-old data from this archive to ~/old:

     $ rdiff-backup -r 3D /mnt/backup ~/old

See rdiff-backup(1).

pdumpfs is a simple daily backup system similar to Plan9's dumpfs which preserves every daily snapshot. You can access the past snapshots at any time for retrieving a certain day's file. Let's backup your home directory with pdumpfs and cron!

pdumpfs constructs the snapshot YYYY/MM/DD in the destination directory. All source files are copied to the snapshot directory when pdumpfs is run for the first time. On and after the second time, pdumpfs copies only updated or newly created files and stores unchanged files as hard links to the files of the previous day's snapshot in order to save disk space.

     $ pdumpfs src-dir dest-dir [dest-basename]

See pdumpfs(8).

Changetrack will record changes to the text-based configuration files in RCS archives regularly. See changetrack(1).

     # apt-get install changetrack
     # vi changetrack.conf

Run top to see what process is acting funny. Press `P' to sort by CPU usage, `M' to sort by memory, and `k' to kill a process. Alternatively, BSD-style ps aux | less or System-V-style ps -efH | less may be used. The System-V-style syntax displays parent process IDs (PPID) which can be used for killing zombie (defunct) children.

Use kill to kill (or send a signal to) a process by process ID, killall to do the same by process command name. Frequently used signals:

      1: HUP,  restart daemon

Insurance against system malfunction is provided by the kernel compile option "Magic SysRq key". Pressing Alt-SysRq on an i386, followed by one of the keys r 0 k e i s u b, does the magic.

Un`r'aw restores the keyboard after things like X crashes. Changing the console loglevel to `0' reduces error messages. sa`k' (system attention key) kills all processes on the current virtual console. t`e'rminate kills all processes on the current terminal except init. k`i'll kills all processes except init.

`S'ync, `u'mount, and re`b'oot are for getting out of really bad situations.

Detailed information is in /usr/share/doc/kernel-doc-version/Documentation/sysrq.txt.gz or /usr/src/kernel-version/Documentation/sysrq.txt.gz.

less is the default pager (file content browser). Hit `h' for help. It can do much more than more. less can be supercharged by executing eval $(lesspipe) or eval $(lessfile) in the shell startup script. See more in file:///usr/share/doc/less/LESSOPEN. The -R option allows raw character output and enables ANSI color escape sequences. See less(1).

w3m may be a useful alternative pager for some code systems (EUC).

free and top give good information on memory resources. Do not worry about the size of "used" in the "Mem:" line, but read the one under it (38792 in the example below).

     $ free -k # for 256MB machine
                  total       used       free     shared    buffers cached
     Mem:        257136     230456      26680      45736     116136 75528
     -/+ buffers/cache:      38792     218344
     Swap:       264996          0     264996

The exact amount of physical memory can be confirmed by grep '^Memory' /var/log/dmesg, which in this case gives "Memory: 256984k/262144k available (1652k kernel code, 412k reserved, 2944k data, 152k init)".

     Total         = 262144k = 256M (1k=1024, 1M=1024k)
     Free to dmesg = 256984k = Total - kernel - reserved - data - init
     Free to shell = 257136k = Total - kernel - reserved - data

About 5MB is not usable by the system because the kernel uses it.

     # date MMDDhhmmCCYY
     # hwclock --utc --systohc
     # hwclock --show

This will set system and hardware time to MM/DD hh:mm, CCYY. Times are displayed in local time but hardware time uses UTC.

If the hardware (BIOS) time is set to GMT, change the setting to UTC=yes in the /etc/default/rcS.

For disabling the screensaver, use following commands.

In the Linux console:

     # setterm -powersave off

Start the kon2 (kanji) console with:

     # kon -SaveTime 0

While running X:

     # xset s off
      or
     # xset -dpms
      or
     # xscreensaver-command -prefs

Read the corresponding manpages for controlling other console features. See also stty(1) for changing and printing terminal line settings.

Glibc offers getent(1) for searching entries from administrative databases, i.e., passwd, group, hosts, services, protocols, or networks.

     getent database [key ...]

One can always unplug the PC speaker. ;-) For the Bash shell:

     echo "set bell-style none">> ~/.inputrc

In order to quiet on-screen error messages, the first place to check is /etc/init.d/klogd. Set KLOGD="-c 3" in this script and run /etc/init.d/klogd restart. An alternative method is to run dmesg -n3.

Here error levels mean:

• 0: KERN_EMERG, system is unusable

• 1: KERN_ALERT, action must be taken immediately

• 2: KERN_CRIT, critical conditions

• 3: KERN_ERR, error conditions

• 4: KERN_WARNING, warning conditions

• 5: KERN_NOTICE, normal but significant condition

• 6: KERN_INFO, informational

• 7: KERN_DEBUG, debug-level messages

If one particular useless error message bothers you a lot, consider making a trivial kernel patch like shutup-abit-bp6 (available in the examples subdirectory).

Another place to look may be /etc/syslog.conf; check to see whether any messages are logged to a console device.

Console screens in Unix-like systems are usually accessed using (n)curses library routines. These give the user a terminal-independent method of updating character screens with reasonable optimization. See ncurses(3X) and terminfo(5).

On a Debian system, there are quite a lot of predefined entries:

     $ toe | less                  # all entries
     $ toe /etc/terminfo/ | less   # user reconfigurable entries

Export your selection as environment variable TERM.

If the terminfo entry for xterm doesn't work with a non-Debian xterm, change your terminal type from "xterm" to one of the feature-limited versions such as "xterm-r6" when you log in to a Debian system remotely. See file:///usr/share/doc/libncurses5/FAQ for more. "dumb" is the lowest common denominator for terminfo.

When the screen goes berserk after cat some-binary-file (you may not be able to see the command echoed as you type):

     $ reset

Convert a DOS text file (end-of-line = ^M^J) to a Unix text file (end-of-line = ^J).

     # apt-get install sysutils
     $ dos2unix dosfile

Following will convert text files between DOS, Mac, and Unix line ending styles:

     $ recode /cl../cr <dos.txt >mac.txt
     $ recode /cr.. <mac.txt >unix.txt
     $ recode ../cl <unix.txt >dos.txt

Free recode converts files between various character sets and surfaces with:

     $ recode charset1/surface1..charset2/surface2 \
       <input.txt >output.txt

Common character sets used are (see also Introduction to locales, Section 9.7.3) [37] :

• us — ASCII (7 bits)

• l1 — ISO Latin-1 (ISO-8859-1, Western Europe, 8 bits)

• EUCJP — EUC-JP for Japanese (Unix)

• SJIS — Shift-JIS for Japanese (Microsoft)

• ISO2022JP — Mail encoding for Japanese (7 bits)

• u2 — UCS-2 (Universal Character Set, 2 bytes)

• u8 — UTF-8 (Universal Transformation Format, 8 bits)

Common surfaces used are [38] :

• /cr — Carriage return as end of line (Mac text)

• /cl — Carriage return line feed as end of line (DOS text)

• / — Line feed as end of line (Unix text)

• /d1 — Human readable bytewise decimal dump

• /x1 — Human readable bytewise hexidecimal dump

• /64 — Base64 encoded text

• /QP — Quoted-Printable encoded text

For more, see pertinent description in the info recode.

There are also more specialized conversion tools:

• character set conversion:

  • iconv — locale encoding conversions

  • konwert — fancy encoding conversions

• binary file conversion:

  • uuencode and uudecode — for Unix.

  • mimencode — for the mail.

Replace all instances of FROM_REGEX with TO_TEXT in all of the files FILES ...:

     $ perl -i -p -e 's/FROM_REGEX/TO_TEXT/g;' FILES ...

-i is for "in-place editing", -p is for "implicit loop over FILES ...". If the substitution is complex, you can make recovery from errors easier by using the parameter -i.bak instead of -i; this will keep each original file, adding .bak as a file extension.

The following script will remove lines 5–10 and lines 16–20 in place.

     #!/bin/bash
     ed $1 <<EOF
     16,20d
     5,10d
     w
     q
     EOF

Here, ed commands are the same as vi command-mode commands. Editing from the back of file makes it easy for scripting.

Following one of these procedures will extract differences between two source files and create unified diff files file.patch0 or file.patch1 depending on the file location:

     $ diff -u file.old file.new > file.patch0
     $ diff -u old/file new/file > file.patch1

The diff file (alternatively called patch file) is used to send a program update. The receiving party will apply this update to another file by:

     $ patch -p0 file < file.patch0
     $ patch -p1 file < file.patch1

If you have three versions of source code, you can merge them more effectively using diff3:

     $ diff3 -m file.mine file.old file.yours > file

     $ split -b 650m file   # split file into 650MB chunks
     $ cat x* >largefile    # merge files into 1 large file

Let's consider a text file called DPL in which all previous Debian project leader's names and their initiation days are listed in a space-separated format.

     Ian     Murdock   August  1993
     Bruce   Perens    April   1996
     Ian     Jackson   January 1998
     Wichert Akkerman  January 1999
     Ben     Collins   April   2001
     Bdale   Garbee    April   2002
     Martin  Michlmayr March   2003

Awk is frequently used to extract data from these types of files.

     $ awk '{ print $3 }' <DPL                   # month started
     August
     April
     January
     January
     April
     April
     March
     $ awk '($1=="Ian") { print }' <DPL          # DPL called Ian
     Ian     Murdock   August  1993
     Ian     Jackson   January 1998
     $ awk '($2=="Perens") { print $3,$4 }' <DPL # When Perens started
     April 1996

Shells such as Bash can be also used to parse this kind of file:

     $ while read first last month year; do 
         echo $month
       done <DPL
     ... same output as the first Awk example

Here, read built-in command uses the characters in $IFS (internal field separators) to split lines into words.

If you change IFS to ":", you can parse /etc/passwd with shell nicely:

     $ oldIFS="$IFS"   # save old value
     $ IFS=":"
     $ while read user password uid gid rest_of_line; do
         if [ "$user" = "osamu" ]; then 
           echo "$user's ID is $uid"
         fi
       done < /etc/passwd
     osamu's ID is 1001
     $ IFS="$oldIFS"   # restore old value

(If Awk is used to do the equivalent, use FS=":" to set the field separator.)

IFS is also used by the shell to split results of parameter expansion, command substitution, and arithmetic expansion. These do not occur within double or single quoted words. The default value of IFS is <space>, <tab>, and <newline> combined.

Be careful about using this shell IFS tricks. Strange things may happen, when shell interprets some parts of the script as its input.

     $ IFS=":,"                        # use ":" and "," as IFS
     $ echo IFS=$IFS,   IFS="$IFS"     # echo is a Bash built-in
     IFS=  , IFS=:,
     $ date -R                         # just a command output
     Sat, 23 Aug 2003 08:30:15 +0200
     $ echo $(date -R)                 # sub shell --> input to main shell
     Sat  23 Aug 2003 08 30 36 +0200
     $ unset IFS                       # reset IFS to the default
     $ echo $(date -R)
     Sat, 23 Aug 2003 08:30:50 +0200

The following scripts will do nice things as a part of a pipe.

     find /usr | egrep -v "/usr/var|/usr/tmp|/usr/local"
                          # find all files in /usr excluding some files
     xargs -n 1 command   # run command for all items from stdin
     xargs -n 1 echo |    # split white-space-separated items into lines
     xargs echo      |    # merge all lines into a line
     grep -e pattern|     # extract lines containing pattern
     cut -d: -f3 -|
             # extract third field separated by : (passwd file etc.)
     awk '{ print $3 }' | # extract third field separated by whitespaces
     awk -F'\t' '{ print $3 }' |
             # extract third field separated by tab
     col -bx |            # remove backspace and expand tabs to spaces
     expand -|            # expand tabs
     sort -u|             # sort and remove duplicates
     
     tr '\n' ' '|         # concatenate lines into one line
     tr '\r' ''|          # remove CR
     tr 'A-Z' 'a-z'|      # convert uppercase to lowercase
     sed 's/^/# /'|       # make each line a comment
     sed 's/\.ext//g'|    # remove .ext
     sed  -n -e 2p|       # print the second line 
     head -n 2 -|         # print the first 2 lines
     tail -n 2 -|         # print the last 2 lines

The following ways of looping over each file matching *.ext ensures proper handling of funny file names such as ones with spaces and performs equivalent process:

• Shell loop (This example is multi line style with PS2=" ". To do the same in one line, you insert a semicolon for each line break.):

       for x in *.ext; do
         if test -f "$x"; then
           command "$x"
         fi
       done
  

• find and xargs combination:

       find . -type f -maxdepth 1 -name '*.ext' -print0 | \
        xargs -0 -n 1 command
  

• find with -exec option with a command:

       find . -type f -maxdepth 1 -name '*.ext' \
        -exec command '{}' \;
  

• find with -exec option with a short shell script:

       find . -type f -maxdepth 1 -name '*.ext' \
        -exec sh -c "command '{}' && echo 'successful'" \;
  

Although any Awk scripts can be automatically rewritten in Perl using a2p(1), one-liner Awk scripts are best converted to one-liner perl scripts manually. For example

     awk '($2=="1957") { print $3 }' |

is equivalent to any one of the following lines:

     perl -ne '@f=split; if ($f[1] eq "1957") { print "$f[2]\n"}' |
     perl -ne 'if ((@f=split)[1] eq "1957") { print "$f[2]\n"}' |
     perl -ne '@f=split; print $f[2] if ( $f[1]==1957 )' |
     perl -lane 'print $F[2] if $F[1] eq "1957"' |

Since all the whitespace in the arguments to perl in the line above can be removed, and taking advantage of the automatic conversions between numbers and strings in Perl:

     perl -lane 'print$F[2]if$F[1]eq+1957' |

See perlrun(1) for the command-line options. For more crazy Perl scripts, http://perlgolf.sourceforge.net may be interesting.

The following will read a web page into a text file. Very useful when copying configurations off the Web.

     $ lynx -dump http://www.remote-site.com/help-info.html >textfile

links and w3m can be used here, too, with slight differences in rendering.

If this is a mailing list archive, use munpack to obtain mime contents from text.

The following will print a web page into a PostScript file/printer.

     $ apt-get install html2ps
     $ html2ps URL | lpr

See lpr/lpd, Section 3.6.1. Also check a2ps and mpage packages for creating PostScript files.

The following will print a manual page into a PostScript file/printer.

     $ man -Tps some-manpage | lpr
     $ man -Tps some-manpage | mpage -2 | lpr

You can merge two PostScript or PDF files.

     $ gs -q -dNOPAUSE -dBATCH -sDEVICE=pswrite \
       -sOutputFile=bla.ps -f foo1.ps foo2.ps
     $ gs -q -dNOPAUSE -dBATCH -sDEVICE=pdfwrite \
       -sOutputFile=bla.pdf -f foo1.pdf foo2.pdf

Display time used by a process.

     # time some-command >/dev/null
     real    0m0.035s       # time on wall clock (elapsed real time)
     user    0m0.000s       # time in user mode
     sys     0m0.020s       # time in kernel mode

Use nice (from the GNU shellutils package) to set a command's nice value when starting. renice (bsdutils) and top can renice a process. A nice value of 19 represents the slowest (lowest priority) process; negative values are "not-nice", with -20 being a very fast (high priority) process. Only the superuser can set negative nice values.

     # nice  -19 top                                      # very nice
     # nice --20 wodim -v -eject speed=2 dev=0,0 disk.img # very fast

Sometimes an extreme nice value does more harm than good to the system. Use this command carefully.

Use cron and at to schedule tasks under Linux. See at(1), crontab(5), crontab(8).

Run the command crontab -e to create or edit a crontab file to set up regularly scheduled events. Example of a crontab file:

     # use /bin/sh to run commands, no matter what /etc/passwd says
     SHELL=/bin/sh
     # mail any output to `paul', no matter whose crontab this is
     MAILTO=paul
     # Min Hour DayOfMonth Month DayOfWeek command (Day... are OR'ed)
     # run at 00:05, every day
     5  0  *  * *   $HOME/bin/daily.job >> $HOME/tmp/out 2>&1
     # run at 14:15 on the first of every month -- output mailed to paul
     15 14 1  * *   $HOME/bin/monthly
     # run at 22:00 on weekdays(1-5), annoy Joe. % for newline, last % for cc:
     0 22 *   * 1-5 mail -s "It's 10pm" joe%Joe,%%Where are your kids?%.%%
     23 */2 1 2 *   echo "run 23 minutes after 0am, 2am, 4am ..., on Feb 1"
     5  4 *   * sun echo "run at 04:05 every sunday"
     # run at 03:40 on the first Monday of each month
     40 3 1-7 * *   [ "$(date +%a)" == "Mon" ] && command -args

Run the at command to schedule a one-time job:

     $ echo 'command -args'| at 3:40 monday

Install netkit-ping, traceroute, dnsutils, ipchains (for 2.2 kernel), iptables (for 2.4 kernel), and net-tools packages and:

     $ ping yahoo.com            # check Internet connection
     $ traceroute yahoo.com      # trace IP packets
     $ ifconfig                  # check host config
     $ route -n                  # check routing config
     $ dig [@dns-server.com] host.dom [{a|mx|any}] |less
           # check host.dom DNS records by dns-server.com 
           # for a {a|mx|any} record
     $ ipchains -L -n |less      # check packet filter (2.2 kernel)
     $ iptables -L -n |less      # check packet filter (2.4 kernel)
     $ netstat -a                # find all open ports
     $ netstat -l --inet         # find listening ports
     $ netstat -ln --tcp         # find listening TCP ports (numeric)

To flush mail from the local spool:

     # exim4 -q    # flush waiting mail
     # exim4 -qf   # flush all mail
     # exim4 -qff  # flush even frozen mail

-qff may be better as an option in the /etc/ppp/ip-up.d/exim script. For Woody and older distributions, replace exim4 with exim.

To remove frozen mail from the local spool with a delivery error message:

     # exim4 -Mg `mailq | grep frozen | awk '{ print $3 }'`

For Woody and older distributions, replace exim4 with exim.

You need to manually deliver mails to the sorted mailboxes in your home directory from /var/mail/username if your home directory became full and procmail failed. After making disk space in the home directory, run:

     # /etc/init.d/exim4 stop
     # formail -s procmail </var/mail/username
     # /etc/init.d/exim4 start

For Woody and older distributions, replace exim4 with exim.

In order to clear the contents of a file such as a logfile, do not use rm to delete the file and then create a new empty file, because the file may still be accessed in the interval between commands. The following is the safe way to clear the contents of the file.

     $ :>file-to-be-cleared

The following commands will create dummy or empty files:

     $ dd if=/dev/zero    of=filename bs=1k count=5 # 5KB of zero content
     $ dd if=/dev/urandom of=filename bs=1M count=7 # 7MB of random content
     $ touch filename #  create 0B file (if file exists, updates mtime)

For example, the following commands executed from the shell of the Debian boot floppy will erase all the content of the hard disk /dev/hda completely for most practical uses.

     # dd if=/dev/urandom of=/dev/hda; dd if=/dev/zero of=/dev/hda

You can check whether two files are the same file with two hard links by:

      
     $ ls -li file1 file2

If file.img contains an image of hard disk contents and the original hard disk had a disk configuration which gives xxxx = (bytes/sector) * (sectors/cylinder), then the following will mount it to /mnt:

     # mount -o loop,offset=xxxx file.img /mnt

Note that most hard disks have 512 bytes/sector.

Basics of getting files from Windows:

     # mount -t smbfs -o username=myname,uid=my_uid,gid=my_gid \
             //server/share /mnt/smb  # mount Windows files to Linux
     # smbmount //server/share /mnt/smb \
             -o "username=myname,uid=my_uid,gid=my_gid"
     # smbclient -L 192.168.1.2 # list the shares on a computer

Samba neighbors can be checked from Linux using:

     # smbclient -N -L ip_address_of_your_PC | less
     # nmblookup -T "*"

Many foreign filesystems have Linux kernel support, and can thus be accessed simply by mounting the devices containing the filesystems. For certain filesystems, there are also a few specialized tools to access the filesystems without mounting the devices. This is accomplished with user-space programs so that kernel filesystem support is not needed.

• mtools: for MS-DOS filesystem (MS-DOS, Windows)

• cpmtools: for CP/M filesystem

• hfsutils: for HFS filesystem (native Macintosh)

• hfsplus: for HFS+ filesystem (modern Macintosh)

In order to create and check an MS-DOS FAT filesystem, dosfstools is useful.

The use of wild card file name in command line arguments such as "rm -rf .*" may cause dangerous result, since ".*" expands to include "." and "..". Fortunately for the current verion of "rm" command in the Debian distribution, it checks sanity of the argument file names and refuses to remove "." and "..". But this is not always the case. Try following to see how the wild card file names work.

• "echo *": lists every non-dot files and non-dot directories under current directory.

• "echo .[^.]*": lists every dot file and dot-directories under current directory.

• "echo .*": lists everything under parent directory and parent directory itself.

Loss of some important files such as /etc/passwd through your stupidity is tough. The Debian system makes regular backups of them in /var/backups/. When you restore these files, you may manually have to set the proper permissions.

     # cp /var/backups/passwd /etc/passwd
     # chmod 644 /etc/passwd

See also Recover package selection data, Section 6.3.4.

The easiest way to control the behavior of an init script is by changing environment variable assignments in the file named like the init script in the /etc/default directory. [39] For example, /etc/default/hotplug can be used to control how /etc/init.d/hotplug works. The /etc/default/rcS file can be used to customize boot-time defaults for motd, sulogin, etc.

If you cannot get the behavior you want by changing such variables then you can modify the init scripts themselves: they are all configuration files.

System log mode can be configured using /etc/syslog.conf. Check the colorize package for a program to colorize system logfiles. See also syslogd(8) and syslog.conf(5).

There are a few hardware optimization configurations that Debian leaves to the sysadmin to take care of.

• hdparm

  • Hard disk access optimization. Very effective.

  • Dangerous. You must read hdparm(8) first.

  • hdparm -tT /dev/hda to test disk access speed.

  • hdparm -q -c3 -d1 -u1 -m16 /dev/hda to speed up a modern IDE system. (It may be dangerous.)

• setcd

  • Compact disc drive access optimization.

  • setcd -x 2 to slow down to 2x speed.

  • See setcd(1).

• setserial

  • Collection of tools for serial port management.

• scsitools

  • Collection of tools for SCSI hardware management.

• memtest86

  • Collection of tools for memory hardware management.

• hwtools

  • Collection of tools for low-level hardware management.

    • irqtune: changes the IRQ priority of devices to allow devices that require high priority and fast service (e.g. serial ports, modems) to have it. 3x speedup of serial/modem throughput is possible.

    • scanport: scans I/O space from 0x100 to 0x3ff looking for installed ISA devices.

    • inb: a quick little hack that reads an I/O port and dumps the value in hex and binary.

• schedutils

  • Linux scheduler utilities.

  • taskset, irqset, lsrt, and rt are included.

  • Together with nice and renice (not included), they allow full control of process scheduling parameters.

Mounting a filesystem with the noatime option is also very effective in speeding up read access to the file. See fstab(5) and mount(8).

Some hardware can be tuned directly by the Linux kernel itself through the proc filesystem. See Tuning the kernel through the proc filesystem, Section 7.2.

There are many hardware-specific configuration utilities in Debian. Many of them address needs specific to the laptop PC. Here are some interesting packages available in Debian:

• tpconfig - A program to configure touchpad devices

• apmd - Utilities for Advanced Power Management (APM)

• acpi - displays information on ACPI devices

• acpid - Utilities for using ACPI

• lphdisk - prepares hibernation partition for Phoenix NoteBIOS

• sleepd - puts a laptop to sleep during inactivity

• noflushd - allow idle hard disks to spin down

• big-cursor - larger mouse cursors for X

• acme - Enables the "multimedia buttons" found on laptops

• tpctl - IBM ThinkPad hardware configuration tools

• mwavem - Mwave/ACP modem support

• toshset - Access much of the Toshiba laptop hardware interface

• toshutils - Toshiba laptop utilities

• sjog - A program to use the "Jog Dial" on Sony Vaio Laptops

• spicctrl - Sony Vaio controller program to set LCD backlight brightness

Here, ACPI is a newer framework for the power management system than APM.

Some of these packages require special kernel modules. They are already included in the latest kernel source in many cases. In case of trouble, you may need to apply the latest patch to the kernel yourself.

PAM (Pluggable Authentication Modules) allow you to control how users log in.

     /etc/pam.d/*             # PAM control files
     /etc/pam.d/login         # PAM control file for login
     /etc/security/*          # PAM module parameters
     /etc/securetty           # this controls root login by console (login)
     /etc/login.defs          # this controls login behaviors (login)

Change the contents of /etc/pam.d/login as follows, if you want insecure but passwordless console terminals at your own risk.

     #auth       required   pam_unix.so nullok
     auth       required   pam_permit.so

Similar tricks can be applied for xdm, gdm, ..., for passwordless console X.

On the other hand, install cracklib2 and set /etc/pam.d/passwd as follows, if you want to enforce a good password policy.

     password required       pam_cracklib.so retry=3 minlen=6 difok=3

A one-time login password for account activation may also help. For this, use the passwd command with the -e option. See passwd(1).

The maximum number of processes can be set with ulimit -u 1000 in a Bash shell or with settings in /etc/security/limits.conf from PAM. Other parameters such as core can be set similarly. The initial value of PATH can be set by /etc/login.defs before the shell startup script.

The documentation for PAM is packaged in the libpam-doc package. The Linux-PAM System Administrator's Guide covers configuring PAM, what modules are available, etc. The documentation also includes The Linux-PAM Application Developers' Guide and The Linux-PAM Module Writers' Guide.

This is the famous phrase at the bottom of the old info su page by Richard M. Stallman. Not to worry: the current su in Debian uses PAM, so that one can restrict the ability to use su to any group using pam_wheel.so in /etc/pam.d/su. The following will set the adm group in a Debian system as an equivalent of the BSD wheel group and allow su without a password for its members.

     # anti-RMS configuration in /etc/pam.d/su
     auth       required   pam_wheel.so group=adm
     
     # Wheel members to be able to su without a password
     auth       sufficient pam_wheel.so trust group=adm

A few interesting groups:

• root group is the default wheel group for su if pam_wheel.so is used without the group= argument.

• adm group can read logfiles.

• cdrom group can be used locally to give a set of users access to a CD-ROM drive.

• floppy group can be used locally to give a set of users access to a floppy drive.

• audio group can be used locally to give a set of users access to an audio device.

• src group owns source code, including files in /usr/src. It can be used locally to give a user the ability to manage system source code.

• staff membership is useful for helpdesk types or junior sysadmins, giving them the ability to do things in /usr/local and to create directories in /home.

For a complete list, see the "FAQ" section in the Securing Debian Manual, which can also be found as the harden-doc package in Woody. Also the new base-passwd (>3.4.6) contains an authoritative list: file:///usr/share/doc/base-passwd/users-and-groups.html.

My usage of sudo is mostly a protection from my own stupidity. Personally, I consider using sudo a better alternative to always using the system as root.

Install sudo and activate it by setting options in /etc/sudoers. Also check out the sudo group feature in file:///usr/share/doc/sudo/OPTIONS.

The sample configuration provides "staff" group members access to any commands run as root under sudo and also gives "src" members access to selected commands run as root under sudo.

The advantage of sudo is that it only requires an ordinary user's password to log in, and activity is monitored. This is a nice way to give some authority to a junior administrator. For example:

     $ sudo chown -R myself:mygrp .

Of course if you know the root password (as most home users do), any command can be run under root from a user account:

     $ su -c "shutdown -h now"
     Password:

(I know I should tighten the admin account's sudo privileges. But since this is my home server, I have not bothered yet.)

For a different program that allows ordinary users to run commands with root privileges, see the super package.

The Internet super-server, inetd, is started at boot time by /etc/rc2.d/S20inetd (for RUNLEVEL=2), which is a symlink to /etc/init.d/inetd. Essentially, inetd allows one running daemon to invoke several others, reducing load on the system.

Whenever a request for service arrives, its protocol and service are identified by looking them up in the databases in /etc/protocols and /etc/services. inetd then looks up a normal Internet service in the /etc/inetd.conf database, or a Sun-RPC based service in /etc/rpc.conf.

For system security, make sure to disable unused services in /etc/inetd.conf. Sun-RPC services need to be active for NFS and other RPC-based programs.

Sometimes, inetd does not start the intended server directly but starts the tcpd TCP/IP daemon wrapper program with the intended server name as its argument in /etc/inetd.conf. In this case, tcpd runs the appropriate server program after logging the request and doing some additional checks using /etc/hosts.deny and /etc/hosts.allow.

If you have problems with remote access in a recent Debian system, comment out "ALL: PARANOID" in /etc/hosts.deny if it exists.

For details, see inetd(8), inetd.conf(5), protocols(5), services(5), tcpd(8), hosts_access(5), and hosts_options(5).

For more information on Sun-RPC, see rpcinfo(8), portmap(8), and file:///usr/share/doc/portmap/portmapper.txt.gz.

Use Lightweight Directory Access Protocol (LDAP). References:

• OpenLDAP

• OpenLDAP Admin Guide in the openldap-guide package

• LDP: LDAP Linux HOWTO

• LDP: LDAP Implementation HOWTO

• OpenLDAP, extensive use reports

• Open LDAP with Courier IMAP and Postfix

First, any disruption of data sent to the CD-writer may cause irrecoverable damage to the CD. Though most newer devices support a Buffer Underrun protection, which is used by default. Get a CD-writer with as large a buffer as possible. If money is no object, do not bother with ATAPI/IDE, just get a SCSI version. If you have a choice of IDE interface to be connected, use the one on the PCI-bus (i.e., on the motherboard) rather than one on the ISA-bus (an SB16 card, etc.).

When a CD-writer is connected to IDE, it has to be driven by the IDE-SCSI driver instead of an ordinary IDE CD driver for Linux 2.2 and 2.4 kernels. Also, the SCSI generic driver needs to be activated. There are two possible approaches to doing this, assuming a kernel distributed with modern distributions (as of March 2001).

For Linux 2.6 kernel, you should use ordinary IDE driver and access CD-RW device directly with device name such as /dev/hdx instead. You can use DMA this way.

Add the following line to /etc/lilo.conf if you are using a stock Debian kernel. If multiple options are used, list them separated by spaces:

     append="hdx=ide-scsi ignore=hdx"

Here the location of the CD-writer, which is accessed through the ide-scsi driver, is indicated by hdx, where x represents one of the following:

     hda          for a master on the first IDE port
     hdb          for a slave on the first IDE port
     hdc          for a master on the second IDE port
     hdd          for a slave on the second IDE port
     hde ... hdh  for a drive on an external IDE port or ATA66/100 IDE port

Type the following commands as root to activate after finishing all the configuration:

     # lilo
     # shutdown -h now

Debian uses make-kpkg to create a kernel. Use the new --append_to_version with make-kpkg to build multiple kernel images. See The Linux kernel under Debian, Chapter 7.

Use the following setup through make menuconfig:

• bzImage

• Exclude the IDE CD driver (not a must, but simpler to do this)

• Compile in ide-scsi and sg, or make them modules

Kernel support for the CD-writer can be activated during booting by the following:

     # echo ide-scsi >>/etc/modules
     # echo sg       >>/etc/modules
     # cd /dev; ln -sf scd0 cdrom

Manual activation can be done by:

     # modprobe ide-scsi
     # modprobe sg

After reboot, you can check installation by:

     $ dmesg|less
     # apt-get install wodim
     # wodim -scanbus

wodim is a fork of the well known cdrecord program shipped with Sarge and older distributions. Replace wodim with cdrecord if you use such a system.

[Per Warren Dodge] Sometimes there may be conflicts between ide-scsi and ide-cd if there are both CD-ROM and CD-R/RW on the system. Try adding the following line to your /etc/modutils/aliases, running update-modules, and rebooting.

     pre-install      ide-scsi      modprobe ide-cd

This causes the IDE driver to load before ide-scsi. The IDE driver ide-cd takes control of the ATAPI CD-ROM—anything that it hasn't been told to ignore. That leaves just the ignored devices for ide-scsi to control.

To create a CD-image of files under target-directory/ as cd-image.raw (bootable, Joliet TRANS.TBL-enabled format; if not bootable, take out -b and -c options), insert a boot floppy in the first floppy drive and

     # dd if=/dev/fd0 target-directory/boot.img 
     # mkisofs -r -V volume_id -b boot.img -c bootcatalog -J -T \
             -o cd-image.raw target_directory/

One funny hack is to make a bootable DOS CD-ROM. If an ordinary DOS boot floppy disk image is in the above boot.img, the CD-ROM will boot as if a DOS floppy were in the first floppy drive (A:). Doing this with freeDOS may be more interesting.

This CD-image file can be inspected by mounting it on the loop device.

     # mount -t iso9660 -o ro,loop cd-image.raw /cdrom
     # cd /cdrom
     # mc
     # umount /cdrom

First test with (assuming double speed)

     # nice --10 wodim -dummy speed=2 dev=0,0 disk.img

Then if OK, write to CD-R with

     # nice --10 wodim -v -eject speed=2 dev=0,0 disk.img

Or write to a CD-RW disk with

     # nice --10 wodim -v -eject blank=fast speed=2 dev=0,0 disk.img

Some CD-RW drives work better with

     # nice --10 wodim -v blank=all speed=2 dev=0,0 disk.img

followed by

     # nice --10 wodim -v -eject speed=2 dev=0,0 disk.img

Two steps are needed to prevent SCSI timeouts during blanking from interfering with the burning step. The argument value to nice may require some adjustments.

Some CD-Rs and commercial CDs have junk sectors at the end that make copying by dd impossible (the Windows 98 CD is one of them). The wodim package comes with the readom command (cdrecord contained readcd instead). Use this to copy any CD contents to an image file. If it is a data disk, mount it and run df to see its actual size. Divide the number shown in blocks (1 block = 1024 bytes) by 2 to get the number of actual CD sectors (1 sector = 2048 bytes). Run readom with options and use this disk image to burn the CD-R/RW.

     # readom dev=target,lun,scsibusno # select function 11

Here, set all three parameters to 0 for most cases. Usually the number of sectors given by readom is excessive! Use the above number from an actual mount for better results.

It should be noted that the use of dd has a few problems if used on CD-ROM. The first run of the dd command may cause an error message and may yield a shorter disk image with a lost tail-end. The second run of dd command may yield a larger disk image with garbage data attached at the end on some systems if the data size is not specified. Only the second run of the dd command with the correct data size specified, and without ejecting the CD after an error message, seems to avoid these problems. If for example the image size displayed by df is 46301184 blocks, use the following command twice to get the right image (this is my empirical information):

     # dd if=/dev/cdrom of=cd.img bs=2048 count=$((46301184/2))

To obtain the latest information on Debian CDs, visit the Debian CD site.

If you have a fast Internet connection, think about installing over the network using:

• a few floppy images.

• a minimal bootable CD image.

If you do not have a fast Internet connection, think about purchasing CDs from a CD vendor.

Please do not waste bandwidth by downloading standard CD images unless you are a CD image tester (even with the new jigdo method).

One noteworthy CD image is KNOPPIX - Live Linux Filesystem On CD. This CD will boot a functioning Debian system without installing itself to the hard disk.

To copy key configuration files and data files to CD-R, use the example backup script backup. Also see Copy and archive a whole subdirectory, Section 8.3 and Differential backup and data synchronization, Section 8.4.

Not tested by me:

     # apt-get install wodim cdparanoia
     # cdparanoia -s -B
     # wodim dev=0,0,0 speed=2 -v -dao -eject defpregap=1 -audio *.wav

or,

     # apt-get install cdrdao #disk at once
     # cdrdao read-cd --device /dev/cdrom --paranoia-mode 3 my_cd # read cd
     # cdrdao write --device /dev/cdrom --speed 8 my_cd    # write a new CD

cdrdao does a real copy (no gaps, etc...).

For DVD writing, you have 2 approaches:

• Use growisofs.

• Use wodim.

There are a few (meta)packages provided to ease installation of the X system in Woody.

x-window-system-core

This metapackage provides the essential components for a stand-alone workstation running the X Window System. It provides the X libraries, an X server (xserver-xfree86), a set of fonts, and a group of basic X clients and utilities.

x-window-system

This metapackage provides substantially all the components of the X Window System as developed by the XFree86 Project, as well as a set of historically popular accessory programs. (Notably, it depends on x-window-system-core, twm, and xdm, i.e., no need to install x-window-system-core if you install this.)

xserver-common-v3

Files and utilities common to XFree86 3.x X servers (XF3)

xserver-*

Supplemental XF3 server packages to support hardware not supported by the new XF4 server (xserver-xfree86) for whatever reason. Some old ATI mach64 cards are not supported in XF4, other cards hang badly in the Woody version of XF4, etc. (For available packages, use apt-cache search xserver-|less. All of these XF3 servers depend on xserver-common-v3.)

For most cases, x-window-system is the package to install. (If you want console login, be sure to disable xdm as described in "Let me disable X on boot!", Section 8.1.4.)

To enable hardware detection during the X configuration stage, install the following packages prior to installing the X system:

• discover – hardware identification system.

• mdetect – mouse device autodetection tool.

• read-edid – hardware information-gathering tool for VESA PnP monitors.

Most X client programs can be started with a command like this:

     client $ xterm -geometry 80x24+30+200 -fn 6x10 -display hostname:0 &

Here, the optional command-line arguments mean:

• -geometry WIDTHxHEIGHT+XOFF+YOFF: the initial size and location of the window.

• -fn FONTNAME: the font to use for displaying text. FONTNAME can be:

  • a14: Normal size font

  • a24: Large size font

  • ... (check available fonts with xlsfont.)

• -display displayname: the name of the X server to use. displayname can be:

  • hostname:D.S means screen S on display D of host hostname; the X server for this display is listening to TCP port 6000+D.

  • host/unix:D.S means screen S on display D of host host; the X server for this display is listening to UNIX domain socket /tmp/.X11-unix/XD (so it's only reachable from host).

  • :D.S is equivalent to host/unix:D.S, where host is the local hostname.

The default displayname for the X client program (application side) can be set by the DISPLAY environment variable. For example, prior to running an X client program, executing one of the following commands achieves this:

     $ export DISPLAY=:0 
             # The default, local machine using the first X screen
     $ export DISPLAY=hostname.fulldomain.name:0.2
     $ export DISPLAY=localhost:0

Its startup can be customized by ~/.xinitrc. For example:

     xrdb -load $HOME/.Xresources
     xsetroot -solid gray &
     xclock -g 50x50-0+0 -bw 0 &
     xload -g 50x50-50+0 -bw 0 &
     xterm -g 80x24+0+0 &
     xterm -g 80x24+0-0 &
     twm

As described in Custom X sessions, Section 9.4.5.1, this overrides everything normal execution of Xsession does when started from startx. Use ~/.xsession instead and use this approach only as the last resort. See xsetroot(1x), xset(1x), and X resources, Section 9.4.10.

Because a remote TCP/IP socket connection without encryption is prone to an eavesdropping attack, the default setting for X in recent Debian versions disables the TCP/IP socket. Consider using ssh for a remote X connection (see Connecting to a remote X server – ssh, Section 9.4.8).

The method described here is not encouraged unless one is in a very secure environment behind a good firewall system with only trusted users present. Use the following command to verify your current X server setting for the TCP/IP socket:

     # find /etc/X11 -type f -print0 | xargs -0 grep nolisten
     /etc/X11/xinit/xserverrc:exec /usr/bin/X11/X -dpi 100 -nolisten tcp

Remove -nolisten to restore TCP/IP listening on the X server.

xhost allows access based on hostnames. This is very insecure. The following will disable host checking and allow connections from anywhere if a TCP/IP socket connection is allowed (see Using X over TCP/IP, Section 9.4.6):

     $ xhost +

You can re-enable host checking with:

     $ xhost -

xhost does not distinguish between different users on the remote host. Also, hostnames (addresses actually) can be spoofed.

This method must be avoided even with more restrictive host criteria if you're on an untrusted network (for instance with dial-up PPP access to the Internet). See xhost(1x).

The use of ssh enables a secure connection from a local X server to a remote application server.

• Set X11Forwarding and AllowTcpForwarding entries to yes in /etc/ssh/sshd_config of the remote host, if you want to avoid corresponding command-line options.

• Start the X server on the local host.

• Open an xterm in the local host.

• Run ssh to establish a connection with the remote site.

       localname @ localhost $ ssh -q -X -l loginname remotehost.domain
       Password:
       .....
  

• Run X application commands on the remote site.

       loginname @ remotehost $ gimp &
  

This method allows the display of the remote X client output as if it were locally connected through a local UNIX domain socket.

Learn everything about xterm at http://dickey.his.com/xterm/xterm.faq.html.

Many older X programs, such as xterm, use the X resource database to configure their appearance. The file ~/.Xresources is used to store user resource specifications. This file is automatically merged into the default X resources upon login. The system-wide defaults of X resources are stored in /etc/X11/Xresources/* and application defaults of them are stored in /etc/X11/app-defaults/*. Use these settings as the starting points.

Here are some helpful settings to add to your ~/.Xresources file:

     ! Set the font to a more readable 9x15
     XTerm*font: 9x15
     
     ! Display a scrollbar
     XTerm*scrollBar: true
     
     ! Set the size of the buffer to 1000 lines
     XTerm*saveLines: 1000
     
     ! Large kterm screen
     KTerm*VT100*fontList: -*-fixed-medium-r-normal--24-*,\
      -*-gothic-medium-r-normal--24-*,\
      -*-mincho-medium-r-normal--24-*

To make these settings take effect immediately, merge them into the database using the command:

     xrdb -merge ~/.Xresources

See xrdb(1x).

The xmodmap program is used to edit and display the keyboard modifier map and keymap table that are used by client applications to convert event keycodes into keysyms in X.

     $ xmodmap -pm 
      ... display the current modifier map
     $ xmodmap -pk | pager
      ... display the current keymap table
     $ xmodmap -e "pointer = 3 2 1" # set mouse for the left hand.
     $ xmodmap ~/.xmodmaprc # set keyboard as in ~/.xmodmaprc

It is usually run from the user's session startup script, ~/.xsession.

To get the keycode, run xev in X and press keys. To get the meaning of keysym, look into the MACRO definition in /usr/include/X11/keysymdef.h file. All the #define statements in this file are named as XK_ prepended to the keysym names.

See xmodmap(1x).

If a GUI program needs to be run with root privilege, use the following procedures to display program output on a user's X server. Never attempt to start an X server directly from the root account in order to avoid possible security risks.

Start the X server as a normal user and open an xterm console. Then:

     $ XAUTHORITY=$HOME/.Xauthority
     $ export XAUTHORITY
     $ su root
     Password:*****
     # printtool &

When using this trick to su to a non-root user, make sure ~/.Xauthority is group readable by this non-root user.

To automate this command sequence, create a file ~/.xsession from the user's account, containing the following lines:

     # This makes X work when I su to the root account.
     if [ -z "$XAUTHORITY" ]; then
             XAUTHORITY=$HOME/.Xauthority
             export XAUTHORITY
     fi
     unset XSTARTUP
     # If a particular window/session manager is desired, uncomment
     # the following and edit it to fit your needs.
     #XSTARTUP=/usr/bin/blackbox
     # This starts x-window/session-manager program
     if [ -z "$XSTARTUP" ]; then
       if [ -x /usr/bin/x-session-manager ]; then
         XSTARTUP=x-session-manager
       elif [ -x /usr/bin/x-window-manager ]; then
         XSTARTUP=x-window-manager
       elif [ -x /usr/bin/x-terminal-emulator ]; then
         XSTARTUP=x-terminal-emulator
       fi
     fi
     # execute auto selected X window/session manager
     exec $XSTARTUP

Then run su (not su -) in an xterm window of the user. Now GUI programs started from this xterm can display output on this user's X window while running with root privilege. This trick works as long as the default /etc/X11/Xsession is executed. If a user set up his customization using ~/.xinitrc or ~/.xsession, the above mentioned environment variable XAUTHORITY needs to be set similarly in those scripts.

Alternatively, sudo can be used to automate the command sequence:

     $ sudo xterm
     ... or
     $ sudo -H -s

Here /root/.bashrc should contain:

     if [ $SUDO_USER ]; then
         sudo -H -u $SUDO_USER xauth extract - $DISPLAY | xauth merge -
     fi

This works fine even with the home directory of the user on an NFS mount, because root does not read the .Xauthority file.

There are also several specialized packages for this purpose: kdesu, gksu, gksudo, gnome-sudo, and xsu. Some other methods can be used to achieve similar results: creating a symlink from /root/.Xauthority to the user's corresponding one; use of the script sux; or putting "xauth merge ~USER_RUNNING_X/.Xauthority" in the root initialization script.

See more on the debian-devel mailing list.

The standard xfs in XFree86-4 works fine with TrueType fonts. You have to install a third-party font server such as xfs-xtt, if you are using XFree86-3.

You just need to make sure that whatever applications you want to use the TrueType fonts are linked against libXft or libfreetype (you probably don't even have to worry about this if you're using pre-compiled .debs).

First set up font support infrastructure:

• Install x-ttcidfont-conf and defoma packages. This automates generation of the fonts.scale and fonts.dir files.

       # apt-get install x-ttcidfont-conf
  

• Edit /etc/X11/XF86Config-4 in the Section "Files" as:

       Section "Files"
           FontPath  "/var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType"
           FontPath  "/usr/share/fonts/truetype"
           FontPath  "/usr/lib/X11/fonts/CID"
           FontPath  "/usr/lib/X11/fonts/Speedo"
           FontPath  "/usr/lib/X11/fonts/misc"
           FontPath  "/usr/lib/X11/fonts/cyrillic"
           FontPath  "/usr/lib/X11/fonts/100dpi:unscaled"
           FontPath  "/usr/lib/X11/fonts/75dpi:unscaled"
           FontPath  "/usr/lib/X11/fonts/Type1"
       EndSection
  

  The first line will setup XFree86 to use any TrueType fonts you install from Debian packages. Type1 font entry is moved down since XFree86 does a rather poor job of rendering Type1 fonts. The :unscaled trick for bitmap fonts should not be needed for new XF4 anymore but I included it here just be sure.

  In order to preserve manual changes of /etc/X11/XF86Config-4 follow instructions in Configuring the X server manually, Section 9.4.3.3.

Then install DFSG font packages:

• Western TrueType fonts:

  • ttf-bitstream-vera: A set of high-quality TrueType fonts created by Bitstream, Inc. [40]

  • ttf-freefont: A set of free high-quality TrueType fonts covering the UCS character set.

  • ttf-thryomanes: A TrueType Unicode font covering Latin, Greek, Cyrillic, and IPA.

• Asian fonts:

  • tfm-arphic-bsmi00lp: Chinese Arphic "AR PL Mingti2L Big5" TrueType font TeX font metric data

  • tfm-arphic-bkai00mp: Chinese Arphic "AR PL KaitiM Big5" TrueType font TeX font metric data

  • tfm-arphic-gbsn00lp: Chinese Arphic "AR PL SungtiL GB" TrueType font TeX font metric data

  • tfm-arphic-gkai00mp: Chinese Arphic "AR PL KaitiM GB" TrueType font TeX font metric data

  • ttf-baekmuk: Korean Baekmuk series TrueType fonts

  • hbf-jfs56: Chinese Jianti Fangsong 56x56 bitmap font (GB2312) for CJK

  • hbf-cns40-b5: Chinese Fanti Song 40x40 bitmap font (Big5) for CJK

  • hbf-kanji48: Japanese Kanji 48x48 bitmap font (JIS X-0208) for CJK

Since Free fonts are sometimes limited, installing or sharing some commercial TrueType fonts is an option for a Debian users. In order to make this process easy for the user, some convenience packages have been created:

• ttf-commercial

• msttcorefonts (>1.1.0) [41]

You'll have a really good selection of TrueType fonts at the expense of contaminating your Free system with non-Free fonts.

All these font packages in Debian should work without any efforts and appear available to all X programs that use the regular "core" font system. This includes things like Xterm, Emacs, and most other non-KDE and non-GNOME applications.

Now, run xfontsel and select any TrueType fonts in the fndry menu, you should be able to see many ungrayed out entries in the "fmly" menu.

For KDE2.2 and GNOME1.4 (with libgdkxft0, which is a hack to get GTK 1.2 to do anti-aliased font rendering), you need to setup Xft1, as well. Xft1 is highly deprecated, and is basically only used by GNOME1.4 and KDE2.2. Edit /etc/X11/XftConfig and add a line like

         dir "/var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType"

before the other dir lines. [42]

For GNOME2 and KDE3 (post Sarge release), you need to setup fontconfig which Xft2 uses to find fonts. [43] You shouldn't need to install anything extra for this because every package using fontconfig Depends on it (indirectly) already.

First, look in /etc/fonts/fonts.conf. There should be a line like the one below. If not, open up /etc/fonts/local.conf and add this

         <dir>/var/lib/defoma/x-ttcidfont-conf.d/dirs/TrueType</dir>

just after the <fontconfig> line.

Fontconfig should pick these up immediately, and "fc-list" should list your new fonts. Another neat feature of fontconfig is that you can just drop fonts in ~/.fonts/ and all your fontconfigified programs will have access to them immediately.

If you manually install a new set of TrueType fonts while in X without using Debian package, run

     # xset fp rehash

to get XFree86 to look at the contents of that directory again and to pickup new ones.

There are many web browser packages with graphical display capabilities as of the Sarge release:

• mozilla The Mozilla browser

• mozilla-firefox Mozilla browser variant (stand-alone)

• epiphany-browser Mozilla browser variant (Gnome)

• konqueror KDE browser

• amaya W3C reference browser

• ...

In testing or unstable, you may face version mismatch problems among mozilla variant browsers since they may require matched versions of shared libraries.

Plug-ins for browsers such as mozilla can be enabled by installing "*.so" manually in the plug-in directory and restarting the browsers.

Plug-in resources:

• Java plug-in: install binary "J2SE" from http://java.sun.com.

• Flash plug-in: install binary "Macromedia Flash Player 5" from http://www.macromedia.com/software/flashplayer/.

• freewrl: VRML browser and Netscape plug-in

• ...

There are sevral client packages with graphical display capabilities as of the Sarge release:

• mozilla-thunderbird stand-alone mail client

• kmail KDE mail client

• evolution groupware suite from Novell

• ...

First install the OpenSSH server and client.

     # apt-get update && apt-get install ssh

/etc/ssh/sshd_not_to_be_run must not be present if one wishes to run the OpenSSH server.

SSH has two authentication protocols:

• SSH protocol version 1:

  • Potato version only supports this protocol.

  • available authentication methods:

    • RSAAuthentication: RSA identity key based user authentication

    • RhostsAuthentication: .rhosts based host authentication (insecure, disabled)

    • RhostsRSAAuthentication: .rhosts authentication combined with RSA host key (disabled)

    • ChallengeResponseAuthentication: RSA challenge-response authentication

    • PasswordAuthentication: password based authentication

• SSH protocol version 2:

  • post-Woody versions use this as the primary protocol.

  • available authentication methods:

    • PubkeyAuthentication: public key based user authentication

    • HostbasedAuthentication: .rhosts or /etc/hosts.equiv authentication combined with public key client host authentication (disabled)

    • ChallengeResponseAuthentication: challenge-response authentication

    • PasswordAuthentication: password based authentication

Be careful about these differences if you are migrating to Woody or using a non-Debian system.

See /usr/share/doc/ssh/README.Debian.gz, ssh(1), sshd(8), ssh-agent(1), and ssh-keygen(1) for details.

Following are the key configuration files:

• /etc/ssh/ssh_config: SSH client defaults. See ssh(1). Notable entries are:

  • Host: Restricts the following declarations (up to the next Host keyword) to be only for those hosts that match one of the patterns given after the keyword.

  • Protocol: Specifies the SSH protocol versions. The default is "2,1".

  • PreferredAuthentications: Specifies the SSH2 client authentication method. The default is "hostbased,publickey,keyboard-interactive,password".

  • PasswordAuthentication: If you want to log in with a password, you have to make sure this is not set no.

  • ForwardX11: The default is disabled. This can be overridden by the command-line option "-X".

• /etc/ssh/sshd_config: SSH server defaults. See sshd(8). Notable entries are:

  • ListenAddress: Specifies the local addresses sshd should listen on. Multiple options are permitted.

  • AllowTcpForwarding: The default is disabled.

  • X11Forwarding: The default is disabled.

• $HOME/.ssh/authorized_keys: the lists of the default public keys that clients use to connect to this account on this host. See ssh-keygen(1).

• $HOME/.ssh/identity: See ssh-add(1) and ssh-agent(1).

The following will start an ssh connection from a client.

     $ ssh username@hostname.domain.ext
     $ ssh -1 username@hostname.domain.ext # Force SSH version 1
     $ ssh -1 -o RSAAuthentication=no -l username foo.host
         # force password on SSH1
     $ ssh -o PreferredAuthentications=password -l username foo.host
         # force password on SSH2

For the user, ssh functions as a smarter and more secure telnet (will not bomb with ^]).

To establish a pipe to connect to port 25 of remote-server from port 4025 of localhost, and to port 110 of remote-server from port 4110 of localhost through ssh, execute on the local machine:

     # ssh -q -L 4025:remote-server:25 4110:remote-server:110 \
                username@remote-server

This is a secure way to make connections to SMTP/POP3 servers over the Internet. Set the AllowTcpForwarding entry to yes in /etc/ssh/sshd_config of the remote host.

One can avoid having to remember a password for each remote system by using RSAAuthentication (SSH1 protocol) or PubkeyAuthentication (SSH2 protocol).

On the remote system, set the respective entries, "RSAAuthentication yes" or "PubkeyAuthentication yes", in /etc/ssh/sshd_config.

Then generate authentication keys locally and install the public key on the remote system:

     $ ssh-keygen          # RSAAuthentication: RSA1 key for SSH1
     $ cat .ssh/identity.pub | ssh user1@remote \
             "cat - >>.ssh/authorized_keys"
     ...
     $ ssh-keygen -t rsa   # PubkeyAuthentication: RSA key for SSH2
     $ cat .ssh/id_rsa.pub | ssh user1@remote \
             "cat - >>.ssh/authorized_keys"
     ...
     $ ssh-keygen -t dsa   # PubkeyAuthentication: DSA key for SSH2
     $ cat .ssh/id_dsa.pub | ssh user1@remote \
             "cat - >>.ssh/authorized_keys"

One can change the passphrase later with "ssh-keygen -p". Make sure to verify settings by testing the connection. In case of any problem, use "ssh -v".

You can add options to the entries in authorized_keys to limit hosts and to run specific commands. See sshd(8) for details.

Note that SSH2 has HostbasedAuthentication. For this to work, you must adjust the settings of HostbasedAuthentication to yes in both /etc/ssh/sshd_config on the server machine and /etc/ssh/ssh_config or $HOME/.ssh/config on the client machine.

There are a few free SSH clients available for non-Unix-like platforms.

Windows

puTTY (GPL)

Windows (cygwin)

SSH in cygwin (GPL)

Macintosh Classic

macSSH (GPL) [Note that Mac OS X includes OpenSSH; use ssh in the Terminal application]

See also SourceForge.net, site documentation, "6. CVS Instructions".

It is safer to protect your SSH authentication key with a passphrase. If it was not set, use ssh-keygen -p to set it.

Place your public key (e.g. ~/.ssh/id_rsa.pub) into ~/.ssh/authorized_keys on a remote host using a password-based connection to the remote host as described in Connecting with fewer passwords – RSA, Section 9.5.3.

     $ ssh-agent bash # or run zsh/tcsh/pdksh program instead.
     $ ssh-add ~/.ssh/id_rsa
     Enter passphrase for /home/osamu/.ssh/id_rsa:
     Identity added: /home/osamu/.ssh/id_rsa (/home/osamu/.ssh/id_rsa)
     $ scp foo user@remote.host:foo
      ... no passphrase needed from here on :-)
     $^D
      ... terminating ssh-agent session

For the X server, normal Debian startup scripts execute ssh-agent as parent process. So you only need to execute ssh-add once.

For more, read ssh-agent(1)and ssh-add(1).

If you have problems, check the permissions of configuration files and run ssh with the "-v" option.

Use the "-P" option if you are root and have trouble with a firewall; this avoids the use of server ports 1–1023.

If ssh connections to a remote site suddenly stop working, it may be the result of tinkering by the sysadmin, most likely a change in host_key during system maintenance. After making sure this is the case and nobody is trying to fake the remote host by some clever hack, one can regain a connection by removing the host_key entry from $HOME/.ssh/known_hosts on the local machine.

fetchmail is run in daemon mode to fetch mail from a POP3 account with an ISP into the local mail system. Configure:

     /etc/init.d/fetchmail   
     /etc/rc?.d/???fetchmail run update-rc.d fetchmail default priority 30
     /etc/fetchmailrc        configuration file (chown 600, owned by fetchmail)

Information on how to start fetchmail as a daemon from the init.d script for Potato is confusing (Woody fixed this). See the sample /etc/init.d/fetchmail and /etc/fetchmailrc files in the example scripts.

If your email headers are contaminated by ^M due to your ISP's mailer, add "stripcr" to your options in $HOME/.fetchmailrc:

     options fetchall no keep stripcr

procmail is a local mail delivery and filter program. One needs to create $HOME/.procmailrc for each account that uses it. Example: _procmailrc

crm114 package provides /usr/share/crm114/mailfilter.crm script which is written in CRM114. This script provides a very effective spam filter which can be trained by feeding the spam and the ham.

CRM114 is a small language designed to write filters in; consider it to be a version of grep with super powers. See crm(1).

Use mutt as the mail user agent (MUA) in combination with vim. Customize with ~/.muttrc; for example:

     # use visual mode and "gq" to reformat quotes
     set editor="vim -c 'set tw=72 et ft=mail'"
     #
     # header weeding taken from the manual (Sven's Draconian header weeding)
     #
     ignore *
     unignore from: date subject to cc
     unignore user-agent x-mailer
     hdr_order from subject to cc date user-agent x-mailer
     auto_view application/msword
     ....

Add the following to /etc/mailcap or $HOME/.mailcap to display HTML mail and MS Word attachments inline:

     text/html; lynx -force_html %s; needsterminal;
     application/msword; /usr/bin/antiword '%s'; copiousoutput;
     description="Microsoft Word Text"; nametemplate=%s.doc

Debian supports locale technology. Locale is a mechanism that allows programs to provide suitable output and functionality according to local conventions such as character set, format for date and time, currency symbol, and so on. It uses environment variables to determine the appropriate behavior. For example, assuming you have both the American English and German locales installed on your system, the error messages of many programs can be multilingual:

     $ LANG="en_US" cat foo
     cat: foo: No such file or directory
     $ LANG="de_DE" cat foo
     cat: foo: Datei oder Verzeichnis nicht gefunden

Glibc offers support for this functionality to programs as a library. See locale(7).

Full locale description consists of 3 parts: xx_YY.ZZZZ.

• xx: ISO 639 language codes (lower case)

• YY: ISO 3166 country codes (upper case)

• ZZZZ: codeset, i.e., character set or encoding identifier.

For language codes and country codes, see pertinent description in the info gettext.

Please note this codeset part may be normalized internally to achieve cross platform compatibility by removing all - and by converting all characters into lower case. Typical codesets are:

• UTF-8: Unicode for all regions, mostly in 1-3 Octets (new de facto standard)

• ISO-8859-1: western Europe (de facto old standard)

• ISO-8859-2: eastern Europe (Bosnian, Croatian, Czech, Hungarian, Polish, Romanian, Serbian, Slovak, Slovenian)

• ISO-8859-3: Maltese

• ISO-8859-5: Macedonian, Serbian

• ISO-8859-6: Arabic

• ISO-8859-7: Greek

• ISO-8859-8: Hebrew

• ISO-8859-9: Turkish

• ISO-8859-11: Thai (=TIS-620)

• ISO-8859-13: Latvian, Lithuanian, Maori

• ISO-8859-14: Welsh

• ISO-8859-15: western Europe with euro

• KOI8-R: Russian

• KOI8-U: Ukrainian

• CP1250: Czech, Hungarian, Polish (MS Windows origin)

• CP1251: Bulgarian, Byelorussian (MS Windows origin)

• eucJP: Unix style Japanese (=ujis)

• eucKR: Unix style Korean

• GB2312: Unix style Simplified Chinese (=GB, =eucCN) for zh_CN

• Big5: Traditional Chinese for zh_TW

• sjis: Microsoft style Japanese (Shift-JIS)

As for the meaning of basic encoding system jargons:

• ASCII: 7 bits (0-0x7f)

• ISO-8859-?: 8 bits (0-0xff)

• ISO-10646-1: Universal Character Set (UCS) (31 bits, 0-0x7fffffff)

• UCS-2: First 16 bit of UCS as straight 2 Octets (Unicode: 0-0xffff)

• UCS-4: UCS as straight 4 Octets (UCS: 0-0x7fffffff)

• UTF-8: UCS encoded in 1-6 Octets (mostly in 3 Octets)

• ISO-2022: 7 bits (0-0xff) with the escape sequence. ISO-2022-JP is the most popular encoding for the Japanese e-mail.

• EUC: 8 bits + 16 bits combination (0-0xff), Unix style

• Shift-JIS: 8 bits + 16 bits combination (0-0xff), Microsoft style.

ISO-8859-?, EUC, ISO-10646-1, UCS-2, UCS-4, and UTF-8 share the same code with ASCII for the 7 bit characters. EUC or Shift-JIS uses high-bit characters (0x80-0xff) to indicate that part of encoding is 16 bit. UTF-8 also uses high-bit characters (0x80-0xff) to indicate non 7 bit character sequence bytes and this is the most sane encoding system to handle non-ASCII characters.

Please note the byte order difference of Unicode implementation:

• Standard UCS-2, UCS-4: big endian

• Microsoft UCS-2, UCS-4: little endian for ix86 (machine-dependent)

See Convert a text file with recode, Section 8.6.12 for conversion between various character sets. For more see Introduction to i18n.

Debian does not come with all available locales pre-compiled. Check /usr/lib/locale to see which locales (besides the default "C") are compiled for your system. If the one you need is not present, you have two options:

• Edit /etc/locale.gen to add the desired locale, then run locale-gen as root to compile it. See locale-gen(8) and the manpages listed in its "SEE ALSO" section.

• Run dpkg-reconfigure locales to reconfigure the locales package. Or if it is not already installed, installing locales will invoke the debconf interface to let you choose needed locales and compile the database.

The following environment variables are evaluated in this order to provide particular locale values to programs:

1. LANGUAGE: This environment variable consists of a colon-separated list of locale names in order of priority. Used only if the POSIX locale is set to a value other than "C" [in Woody; the Potato version always has priority over the POSIX locale]. (GNU extension)

2. LC_ALL: If this is non-null, the value is used for all locale categories. (POSIX.1) Usually "" (null).

3. LC_*: If this is non-null, the value is used for the corresponding category (POSIX.1). Usually "C".

   LC_* variables are:

   • LC_CTYPE: Character classification and case conversion.

   • LC_COLLATE: Collation order.

   • LC_TIME: Date and time formats.

   • LC_NUMERIC: Non-monetary numeric formats.

   • LC_MONETARY: Monetary formats.

   • LC_MESSAGES: Formats of informative and diagnostic messages and interactive responses.

   • LC_PAPER: Paper size.

   • LC_NAME: Name formats.

   • LC_ADDRESS: Address formats and location information.

   • LC_TELEPHONE: Telephone number formats.

   • LC_MEASUREMENT: Measurement units (Metric or Other).

   • LC_IDENTIFICATION: Metadata about the locale information.

4. LANG: If this is non-null and LC_ALL is undefined, the value is used for all LC_* locale categories with undefined values. (POSIX.1) Usually "C".

Note that some applications (e.g., Netscape 4) ignore LC_* settings.

The locale program can display active locale settings and available locales; see locale(1). (NOTE: locale -a lists all the locales that your system knows about; this does not mean that all of them are compiled! See Activating locale support, Section 9.7.4.)

The locale support for the international date standard of yyyy-mm-dd (ISO 8601 date format) is provided by the locale called en_DK, "English in Denmark", which is a bit of joke :-) This seems to work only in a console screen for ls.

Add the following lines to ~/.bash_profile:

     LC_CTYPE=en_US.ISO-8859-1
     export LC_CTYPE

Add the following lines to ~/.bash_profile:

     LANG=fr_FR@euro
     export LANG
     LC_CTYPE=fr_FR@euro
     export LC_CTYPE

Configure the keyboard for French "AZERTY" as described in Localizing the keyboard, Section 9.7.1.1; add French manual pages by installing manpages-fr. The Right-Alt key in the US is called Alt-Gr in Europe. Pressing this together with other keys creates numerous accented and special characters. For example, Alt-Gr+E creates a Euro sign.

Most western European languages can be configured similarly.

See Debian Euro HOWTO for adding support for the new Euro currency and Utiliser et configurer Debian pour le français for more details in French.

Let us set up a multilingual X window system which simultaneously supports Japanese, English, German, and French with EUC, UTF-8, and ISO-8859-1 encodings in different consoles.

I will show you a customization using the Debian menu system. See the details of Debian menu system in file:///usr/share/doc/menu/html/index.html. I also create a shortcut to the mozilla web browser in this example. [46]

• add locale support for the Japanese ja_JP.eucJP locale and other required locales using the method described at Localization (l10n), Section 9.7. (for all)

• install Kana-to-Kanji conversion system and dictionary (for Japanese):

  • canna – Local server ("free-beer" license), or

  • freewnn-jserver – Network-extensible server (Public Domain)

• install Japanese input method system (for Japanese):

  • kinput2-canna – for X, or

  • kinput2-canna-wnn – for X, and

  • egg – directly works with Emacsen even in console (optional)

• Install compatible terminals (for all):

  • xterm – X (for ISO-8859-1 and UTF-8),

  • kterm – X (for Japanese EUC), and

  • mlterm – X (multilingual).

• add all the required font packages. (for all)

• create the ~/.xsession that sets the user-specific X environment as described in Custom X sessions, Section 9.4.5.1 (for all):

       #!/bin/sh
       # This makes X work when I su to root.
       if [ -z "$XAUTHORITY" ]; then
               XAUTHORITY=$HOME/.Xauthority
               export XAUTHORITY
       fi
       
       # Set specific environment through debian menu system.
       # Reset locale
       unset LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES 
       unset LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT 
       unset LC_IDENTIFICATION LC_ALL LANG LANGUAGE PAGER
       # set locale default in X
       LANG=C
       # export locale
       export LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES 
       export LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT 
       export LC_IDENTIFICATION LC_ALL LANG LANGUAGE PAGER
       ###
       # activate input method for Japanese with kinput2
       kinput2 &
       XMODIFIERS="@im=kinput2"
       export XMODIFIERS
       # How about blackbox window manager (lightweight)
       exec blackbox
       #exec xfwm
       #exec wmaker
  

• set locale in ~/.bash_profile for Linux consoles (for all).

• remove locale settings from ~/.bashrc, if existed (for all).

• create few files in /etc/menu/ (for all).

  • /etc/menu/xterm-local: (add new entries to menu) [47]

         ?package(xterm):\
          needs=x11\
          section=XShells\
          longtitle="XTerm: terminal emulator (en_US.ISO-8859-1)"\
          title="XTerm (en_US.ISO-8859-1)"\
          command="sh -c 'LC_ALL=en_US.ISO-8859-1 xterm'"
         ?package(xterm):\
          needs=x11\
          section=XShells\
          longtitle="XTerm: terminal emulator (de_DE.ISO-8859-1)"\
          title="XTerm (de_DE.ISO-8859-1)"\
          command="sh -c 'LC_ALL=de_DE.ISO-8859-1 xterm -T xterm-de'"
         ?package(xterm):\
          needs=x11\
          section=XShells\
          longtitle="XTerm: terminal emulator for X with Unicode support (Japanese)"\
          title="UXTerm (ja_JP.UTF-8)"\
          command="sh -c 'LC_ALL=ja_JP.UTF-8 uxterm'"
    

  • /etc/menu/kterm: (override the system default) [48]

         ?package(kterm):\
          needs="x11"\
          section="XShells"\
          command="sh -c 'LC_ALL=ja_JP.eucJP PAGER=w3m /usr/X11R6/bin/kterm -xim'" \
          title="Kanji Terminal"
         ?package(kterm):\
          needs="x11"\
          section="XShells"\
          command="sh -c 'LANG=ja_JP.eucJP \
                   LC_MESSAGES=en_US.ISO-8859-1 PAGER=w3m /usr/X11R6/bin/kterm -xim'" \
          title="Kanji Terminal (bilingal)"
    

  • /etc/menu/mozilla-local: (add a new shortcut) [49]

         ?package(mozilla-browser):needs="x11" section="/" \
                 title=" Mozilla Navigator" command="mozilla-1.5" hints="Web browsers" \
                 icon=/usr/share/pixmaps/mozilla.xpm
    

  • run update-menus from the root account.

• add the following lines to ~/.muttrc (for Japanese):

       # UTF-8 support is not popular in popular Japanese EMACS environment
       # 7-bit encoding of iso-2022-jp is easier for everyone.
       # default encoding order = us-ascii --> iso-8859-1 --> iso-2022-jp
       set send_charset="us-ascii:iso-8859-1:iso-2022-jp"
       set allow_8bit=no
  

• activate XIM kinput2 for X applications (for Japanese):

  • add *inputMethod: kinput2 and KTerm*VT100*OpenIm: true to your X resources file, ~/.Xresources (it looks like Debian takes care of this automatically somehow).

  • Some applications (such as mlterm) also allow you to set up *inputMethod: and other information dynamically at runtime (press Ctrl-MouseButton-3 in mlterm).

• start X by typing startx or from one of the display managers (xdm, gdm, kdm, wdm, ...) (for all).

• start a Japanese-compatible application such as Vim 6, (x)emacs21, mc-4.5, mutt-1.4, ... in kterm (for Japanese). (Emacs seems to be the most popular platform, though I do not use it.)

• press Shift+Space to toggle Japanese character input mode on and off (for Japanese).

• read the localized manual page by starting command in localized console (for all).

For other CJK language supports, see the following sections and SuSE pages for CJK.

There are many alternative X input methods support packages available:

     Language   LC_CTYPE     XIM server XMODIFIERS              Start key
     Japanese   ja_JP*       kinput2    "@im=kinput2"           Shift-Space
     Korean     ko_KR*       ami        "@im=Ami"               Shift-Space
     Chinese(T) zh_TW.Big5   xcin       "@im=xcin-zh_TW.big5"   Ctrl-Space
     Chinese(S) zh_CN.GB2312 xcin       "@im=xcin-zh_CN.GB2312" Ctrl-Space

Japanese input method kinput2 is offered by the packages such as kinput2-canna-wnn, kinput2-canna, and kinput2-wnn. Japanese needs dictionary server such as canna and freewnn-jserver to be practical.

There are many X consoles which support simple 8 bit encodings when pertinent font packages are installed:

• xterm – The X terminal emulator

• gnome-terminal – xterm for Gnome

• konsole – xterm for KDE

• rxvt – VT102 terminal (lighter)

• aterm – VT102 for Afterstep WM

• eterm – VT102 for Enlightment WM

• wterm – VT102 for WindowMaker WM

Multi-byte encoding supports of X console are provided by xterm through UTF-8 encoding (UTF-8 support for the X terminal emulator, Section 9.7.12). Other traditional encoding supports are in progress (as of 2003). Following packages offer traditional encoding supports:

• aterm-ml – Multi-lingual

• kterm – Multi-lingual (Japanese, ...)

• rxvt-ml – Multi-lingual

• wterm-ml – Multi-lingual

• cxterm-big5 – Chinese (Trad., Big5)

• cxterm-gb – Chinese (Simp., GB)

• cxterm-ks – Chinese (KS)

• cxterm-jis – Japanese

• hanterm-classic – Korean (Hangul)

• hanterm-xf – Korean (Hangul)

• hztty – Chinese (GB, Big5, zW/HZ)

For kterm (and possibly others), you may want to activate XIM through menu after Ctrl-middle-click mouse action.

UTF-8 support for X terminal emulator is provided by the uxterm program in the xterm package for XFree86 4.x. It enables support for all languages. It is a wrapper around the xterm(1) program that invokes the latter program with the "UXTerm" X resource class set.

For example, to enable nice large display of English, Russian, Japanese, Chinese, and Korean characters, add following to your ~/.Xresources after installing all the pertinent fonts:

     ! set large font
     UXTerm*font: -misc-fixed-medium-r-normal-*-18-120-100-100-c-90-iso10646-1
     ! Use XIM for Japanese
     *inputMethod: kinput2

Then run xrdb -merge ~/.Xresources to update X resources as described in X resources, Section 9.4.10.

Although most of the popular console program packages such as vim, mutt, and emacs have been made compatible with UTF-8 recently (Woody-Sarge). Program such as mc still is not UTF-8 compatible but simply 8-bit clean. If you are editing 7 bit ASCII part of unknown or mixed encoding file, it is safer to use the locale unaware 8-bit clean editor.

See The Unicode HOWTO.

UTF-8 support on a FB console is provided by bterm used in the debian-installer.

When you are first setting the system up for a national language environment, please consider using tasksel or aptitude to find out what packages are selected by choosing the corresponding language environment task. The package choice made is useful even for a multilingual setup. If you encounter any package dependency conflicts during the install to your carefully configured system, avoid installing any software that conflicts with the existing system. You may have to use update-alternative to regain the original state for some commands since a newly installed one may have higher priority than existing ones.

Newer major programs are using glibc 2.2 and are mostly internationalized. So a specially localized version such as jvim for Vim may not be needed as its functionality is offered by vim version 6.0 in X. In reality, it is still somewhat rough-edged. Since jvim has a version compiled with direct Japanese input method (canna) support even in the console and addresses many other Japanese-specific issues maturely, you may still want it :-)

Programs may need to be configured beyond locale configuration to enable a comfortable working environment. The language-env package and its command set-language-env greatly eases this process.

Also see the internationalization document, Introduction to i18n. It is aimed at developers but is also useful for system administrators.