Troubleshooters.Com and T.C Linux Library Present

Coasterless DVD Burning
Copyright (C) 2005 by Steve Litt
[ Linux Library | Troubleshooters.Com | Email Steve Litt | Copyright Notice ]
Contents:

Executive Summary

A "coaster" is defined as a DVD, that you've burned, that does not contain the correct data. It's called a coaster because its only use is as a coaster to put drinks on.

DVD burning is somewhat similar, and very different to CD burning. This document discusses DVD+R and DVD+RW, because that's the media I have. Here are the main points of this document:

Disclaimer

Obviously, you use this document at your own risk. I am not responsible for any damage or injury caused by your use of this document, or caused by errors and/or omissions in this document. If that's not acceptable to you, you may not use this document. By using this document you are accepting this disclaimer.

Context of this Document

DVD recording is a HUGE subject. This document doesn't attempt to cover the whole of it. For instance, it covers only DVD+R and DVD+RW for this simple reason -- my DVD burner can't do DVD RAM, and I declined to spend the money for DVD-R and DVD-RW.

DVDs can be burned for many purposes: backups, archiving, audio, video, temporary random access storage, and probably several others. This document sticks to backups and archiving for a very simple reason -- that's what I need.

This document is written for a person just getting started with DVD burning. It details the pitfalls and how to avoid them. The reason is simple enough -- I'm just now learning DVD burning, and have recently fallen into those pitfalls.

This document emphasises the growisofs program, which calls cdrecord to do its work. There are many alternatives for burning from commands -- cdrecord (plus a kernel patch or special license), cdrdao, and probably others. I've stuck to growisofs because it seems to work universally, at least on the 2.6 kernel.

I stayed away from cdrecord because I initially read that the cdrecord program no longer works with DVD unless you get a licence for a special cdrecord version. Although such license is without cost, it made me a little uneasy to trust my backup system to something requiring a license. In fact, I was later able to burn a 1.2GB DVD using mkisofs and cdrecord in a similar manner as on a CD, using the mkisofs and cdrecord that came with Mandrake 10.0. However, by that time, I was well acquainted with growisofs and chose to stay with it.

Neither backups nor archiving require adding to completed backups, so no sessioning is needed. The Disk At Once (also called DAO and SAO (Session At Once)) method skips multisessioning, and appears to be more reliable. For backups and archives, use it!

Backing up must be easy or it won't be done. It must not be costly in terms of time missed from productive work. What this means is that it must be configured such that you click a button or issue a command, and the backup plus accuracy check be done without further operator intervention. I've found scripts to accomplish this best. My script does the following:
  1. Create .tgz files from the directories that need backing up
  2. Compare the backed up files inside the .tgz files with their matches on the hard disk, abort if differences
  3. Write md5sum files for each .tgz
  4. Use growisofs to burn the .tgz files to the DVD
  5. Mount the DVD
  6. Compare the md5sums for the .tgz files on the DVD to the md5sum files created in step 3
  7. If all md5sums match, the backup is good
You'll notice some uncertainty in this document. Right now in early 2005, the facts surrounding DVD burning are murky. The DVD burning drive manufacturers withhold vital technical information, so nobody can really be sure of anything. Andy Polyakov, the author of growisofs is perhaps the foremost DVD burning authority in the open source world, yet even his documentation contains quite a bit of uncertainty. Even Andy can't reverse engineer every secret kept secret by the manufacturers. So if someone tells you they know the One True Way(TM), be careful of their advice.

I'm writing this document as I learn it. It probably contains errors. From time to time I'll improve this document, both in terms of breadth and accuracy.

Backup vs. Archiving

Much ink is spilled when people don't clearly define whether they're archiving or backing up. The following chart lists the properties of each.


 Backup
 Archive
Description
 A backup is a copy of some or all of the data on your computer. The backup's purpose is to give you a fallback position in case the data on the computer is lost. Under normal circumstances, the data continues to be accessed via the computer.
 An archive is an actual transfer of data from your computer to the DVD. After a successful archive, the data is erased from your computer to make room for other data. Thus, the data can be accessed only from the archive.
Frequency
 Backups are made frequently -- typically once a week or once a month, or a combination (once a day or once a week on rewriteable media, and once a month on write-once media.
 Archives are typically made once. For important data, perhaps two copies are made of the data, but with the data removed from the computer, there is no reason to remake the archive.
Priorities
  1. Restorability (accessability)
  2. Standard hardware and software
  3. Ability to test restorral accuracy
  4. Ease of creation
  5. Compression
  6. Ease of access
  1. Restorability (accessability)
  2. Standard hardware and software
  3. Ease of access
  4. Ability to test restorral accuracy
  5. Compression
  6. Ease of creation

In normal use, an archive could be consulted often. It needs to be easy to consult. That rules out whole-archive compression. Also, there is typically only one, or maybe two, copies of an archive, meaning that the archive must be constructed such that if one file goes bad, the others are still readable.

Contrast the archive with the backup. Backups are more amenible to whole-archive compression because the backup is seldom used (emergencies only), and in a well thought out backup strategy, if corruption of a single file makes the whole archive unreadable, one can simply go back to the backup before that one.

Ease of creation is much more important with backups, because you back up every day, every week, and/or every month. If it's not easy, you won't do it. For practical purposes, easy creation means a "one click" backup, which in the Linux world probably means a script. You basically run the script before lunch, and when you come back from lunch your backup is on the CD, and has been tested for accuracy. Because you back up so often, the time you spent creating the script is more than compensated by the hours of point and click and waiting you didn't have to do on every backup.

Archives are different. Each one is a one-of-a-kind. It makes no sense to make a script -- instead you use a product like k3b or Nautilus CD Burner to gather the files and directories and burn them. Then you test the burn, and once satisfied, you delete them from the system.

You will find people who vehemently insist that backups should be whole archive compression. Whole archive compression often compresses better. They'll also point out that archive formats such as tar/gzip do a much better job with long filenames than does iso9660, at least without Rock Ridge or Joliet extensions. And who knows whether software 10 years from now will recognize Rock Ridge or Joliet? They'll point out that the md5sum for a large archive can be stored with the archive, so any time in the future the archive can be tested for accuracy. As far as one bad bit bombing an entire archive, they will point out that CD-R and DVD+R, when burned properly with the right burner and media, have an incredibly long shelf life.

Then there are those who insist you never use whole archive compression. One bad bit will trash the archive! Retrieval of a single file is difficult and time consuming. With file by file archiving, you can use the files right off the archive media.

They're both right and they're both wrong. Your choice of single archive vs. file by file depends on your needs and your preferences. For me, backups, which are seldom used, must be easy to create and test for accuracy at any time, are best done whole archive compression. For me, archives, which are done once, need not be easy to create, but absolutely must survive a single bad bit. Your mileage may vary -- there's no right answer.

Burning With mkisofs and cdrecord

As mentioned earlier in this document, the cdrecord program is rumored to have problems with DVD burning, including the need for a special version requiring a license. That is why I chose growisofs. That being said, I have successfully burned DVDs with over 1 GB of data, so they were definitely burned as DVDs. Here are the commands I used:

mkisofs -pad -J -R -otest.iso directory1 directory2 directory3
cdrecord dev=ATA:1,0,0 speed=2 -sao  -pad -v -eject test.iso

There are some points to note in the preceding. The -J and -R in mkisofs stand for Joliet and Rock Ridge -- the Windows and Linux methods of long filenames. The ATA in the device specification of the cdrecord program became necessary when I switched to the 2.6 kernel, and is exactly the same as what you use for burning CDs. The speed=2 specification refers to the DVD speed, not the speed on a CD. It burns much faster than a CD burned at 2x speed.

The -sao specification is interesting. It stands for "Session At Once", which is supposedly the same as "Disk At Once" (DAO). However, in my experiences -dao didn't work for some reason. -pad is necessary in DVDs for the same reason as in CDs -- the Linux readahead bug. It's likely that a -padsize needs to be specified also, but I don't know the correct value as of 2/1/2005.

Perhaps most interesting, the preceding burn was to a DVD+RW rewriteable disc. So why is there no blank=fast? With DVD+RW, you needn't blank the disk, or it gets done automatically, or something like that. All I know is that if you include blank=fast it doesn't work, but if you don't include it, it works.

Which is better -- growisofs or mkisofs plus cdrecord? I don't know. All I can tell you is that I use growisofs, and therefore this document's burning instructions use growisofs.

Gotchas

DVD burning isn't exactly like CD burning (in spite of what you might have heard), so forewarned is forearmed.

Don't blank or format DVD+RW

If you burn a CD-RW, you must include blank=fast or another type of blanking. With DVD+RW, blanking isn't necessary. What is necessary is that the first time the DVD+RW media is used, it be formatted. Furtunately the growisofs program detects that condition, and performs the format for you. As a DVD+RW user, you should never need to blank or format a DVD+RW.

If for some outlandish reason you need to reformat a DVD+RW, you can use the dvd+rw-format program. Just be aware that doing so will substantially reduce the lifespan of your media. Also, occasionally you might need to close a session on a DVD+RW disc, and once again, you can do so with the dvd+rw-format program. However, this will never be necessary if you follow this document's advice and always use Disc At Once (DAO).

Disc At Once (DAO)

You can write a DVD one track at a time (TAO -- Track At Once), or the whole disc at a time (DAO -- Disc At Once), or raw. Disc at once is the most trustworthy, and it can also be used with growisofs special -dvd-compat option, which makes the resulting DVD more readable everywhere and more portable. A typical command to burn file1, file2 and file3 onto a DVD might look like this:
growisofs -use-the-force-luke=dao -dvd-compat speed=2 -Z /dev/hdc -R -J -pad file1 file2 file3
The -use-the-force-luke=dao option enables DAO. I'm not sure of the exact meaning of the -use-the-force-luke, which is poorly documented anywhere. Right now I'll assume it means to force the burn to be DAO. From what I've read, on DVD+R and DVD+RW, the -dvd-compat only works on DAO burned discs, so this is another reason to go DAO.

Speed? What Speed?

Speed is much more important with DVD than it was with CDs. CDs have the good grace, for a given speed, to record more slowly on the inner parts of the CDs recordable surface, where it's hard to get a good recording, and then speed up as recording goes out, so you get good speed. DVDs appear to record full speed ahead right from the start (inner parts), so that on all but the highest quality media, the inner parts aren't burned well. So you crank down the burn speed, and then the first files recorded are readable, but the whole burning process proceeds at that slllloooooowwwww speed. This is why you'll often burn 4x DVD+RW discs at 2x speeds. Every time you get a new brand or type of media, you'll need to experiment.

The more "cheap" and "noname" your media, the slower you'll need to burn it, in all probability. I would imagine this will change over time, but right now, from what I've observed on DVD+RW, and what I've read about DVD+R, for some brands you'll need to burn it slower than the media's labeled maximum speed.

ALWAYS check the accuracy of your recordings, especially when switching media types, brands or vendors. Adjust your speed as necessary.

Dispose of Automount

Automount is wonderful for CD drives. It enables even the densest of users to load and use a CD. Unfortunately, you should never burn a DVD on a mounted drive. Given the fact that growisofs and mkisofs both eject and reload the CD, it's all too likely that it will automount if you let it.

The hot tip is to remove the automounting code from /etc/fstab, replacing it with something like this:
/dev/hdc /mnt/cdrom auto umask=0,user,codepage=850,iocharset=iso8859-1,users,noauto,ro,exec 0 0

/dev/hdc /mnt/cdrom auto umask=0,user,codepage=850,iocharset=iso8859-1,users,noauto,ro,exec 0 0

Field #
 Field meaning
 Contents in preceding example
 Explanation (if necessary)
1
 device
 
/dev/hdc
 The device being mounted. This could be /dev/dvd or /dev/cdrom or /dev/hdc or /dev/scd0 or any of several other device names. The best way to figure out the proper device name is to burn a DVD in the K3b program, which is a graphical front end to growisofs, and while it's burning, grep for growisofs and note the devicename.
2
 mountpoint
 
/mnt/cdrom
  
3
 filesystem type
 
auto
 Auto means look at the contents of the DVD, try to find a matching filetype, and mount it. If this field had contained iso9660, it would only mount iso9660 DVDs.
4
 mount options (comma delimited)
 
umask=0,user,codepage=850,iocharset=iso8859-1,users,noauto,ro,exec
 noauto means that this drive is not automatically mounted at boot. This eliminates the necessessity of unmounting a DVD on boot.

user means any user can mount it with this command:
mount /dev/hdc
users means any user, including a user not the same as the one who mounted it, can unmount it with this command:
umount /dev/hdc
The ability to mount and unmount is important because it must be unmounted to burn, and mounted to verify.
5
 dump frequency in days
 
0
 dump is a backup program that can be run automatically. Typical backups are run on directories, not partitions, so this is typicaly unnecessary in modern systems. If you set it to 0, dump doesn't get done. If you set it to an integer i, dump gets done every i days.
6
 check order
 
0
 On boot, the root filesystem must be checked first. Once it's found OK, all other filesystems may (and probably should) be checked in parallel. The following are usual values for this field:
1: fsck on the first pass (typically root filesystem)
2: fsck on the next pass (typically all others needing checking
0: Never fsck (cdroms and the like).

Bottom line: Your DVD burner should never be automounted.

Backing Up Whole Several Directories

Using growisofs (or mkisofs for that matter) to back up a single directory is trivial. Just list the directory at the end of the command. The directory's contents (not the directory, but its contents) are burned to the root of the DVD, which is often what you want.
growisofs -J -R -Z /dev/hdc -speed=2 /home/myuid/golug

As mentioned, the preceding command puts the contents of the golug directory on the root of the DVD. But what if you need to back up several directories? Their contents will clash and get mixed together in the root of the DVD. That's no good.

The solution is proper use of the -graft-points option for mkisofs, which can also be used in growisofs. The following command burns the golug directory to golug, and the toastmasters directory to toastmasters:
growisofs -J -R -Z /dev/hdc -speed=2 -graft-points /golug=/home/myuid/golug /toastmasters=/home/myuid/toastmasters
After the preceding command, the root of the DVD will have exactly two (non-dot) entries, golug and toastmasters.

growisofs Versions

The growisofs program is constantly improved, as are mkisofs and cdrecord. These programs might perform differently on different machines, not because the machines or operating systems are different, but because the versions of these programs are different. On any Linux or BSD machine, the version can be updated with source or with packaging.

If you have a modern Linux distro, versions shouldn't be a big deal, but if things go wrong, this is one thing to check.

Newbie Guide: Buying and Installing a DVD Burner

A quick Google search lets you know that DVD burners aren't without their challenges. Which one do you buy? How do you interface it with Linux? How do you install it? How do you test it? How do you gain confidence?

My understanding is that most DVD burners currently (2005) manufactured are with MMC compatibility, and therefore should work just fine with your existing x86 computer, and with Linux. This doesn't mean it will be easy. If you're new to DVD recording, I recommend you approach it like I did -- by starting on an experimental machine:
Why the Knoppix computer? Because Knoppix has truly outstanding hardware recognition, and because its software is modern. If the DVD burner doesn't work with the latest Knoppix, chances are it won't work anywhere in LinuxLand, and you should consider returning it. Once it works in Knoppix, you know that you can strongarm it into working on any distro.

Another plus for the Knoppix experimental computer is that if you mess up its configuration, you can reinstall it in 30 minutes and there's no valuable data to restore. It's a risk free environment for experimentation.

Doing the Burn

You can burn with the one step or two step burn methods, depending on your needs.

The Two Step Method


The two step method is like the old mkisofs->cdrecord method for CDs, in that you make an ISO first and then burn off that ISO. This makes it perfect for burning several identical CDs, or for burning CDs until you get one right (for instance, testing different speeds). This is also a very effective method to preserve a snapshot of a set of files, so that you can keep modifying those files and yet burn a DVD of the snapshot any time you want.

The two step method isn't perfect. It requires more human intervention, or a more complex script. It's consecutive instead of concurrent, so I think it's likely to take longer, because a one step method could create new ISO information while the (slow) DVD burn takes place. It also requires disk space for the ISO file.

The following is an example of the 2 step method:
mkisofs -J -R -pad -o myfiles.iso file1 file2 file3
growisofs -Z /dev/hdc=myfiles.iso -speed=2

The One Step Method

The one step method performs the ISO construction and the DVD burning concurrently (I think), so it should be faster. It also requires less operator intervention or a simpler script. It is likely to require less disk space. However, it's less versatile, in that multiple copies cannot be burned. As far as I know, there is no way for growisofs to create an ISO onto your hard disk, only mkisofs can do that.

The one step method is perfect for those instances when you're backing up, and you'll only make one copy (which is typical of backups).

The following is an example of the 1 step method, eqivalent to the previous example of the 2 step method:
growisofs -J -R -Z /dev/hdc -speed=2 file1 file2 file3

DAO

In both the one and two step methods, to obtain added reliability, use DAO and -dvd-compat. Here is the one step equivalent with those two additional features:
growisofs -use-the-force-luke=dao -dvd-compat -J -R -Z /dev/hdc -speed=2 file1 file2 file3
And here is the two step equivalent:
mkisofs -J -R -pad -o myfiles.iso file1 file2 file3
growisofs -use-the-force-luke=dao -dvd-compat -Z /dev/hdc=myfiles.iso -speed=2
Using DAO and -dvd-compat is the best way to make your new DVD readable not only on your burner, but on all DVD readers.

Pinpoint Directory Accuracy

As mentioned  in the Gotchas article, the normal commands, when run on a directory, put the contents of that directory in the root. You typically want the directory, or the last element of the directory's path, in the root. That's accomplished like this:
growisofs -J -R -Z /dev/hdc -speed=2 -graft-points /dir1=/full1/path1/dir1 /dir2=/full2/path2/dir2 /dir3=/full3/path3/dir3
Perhaps you want to copy parts of whole trees to the DVD, so that the directory structure of the DVD matches that of your hard disk, except only a few directories are included. Then do this:
growisofs -J -R -Z /dev/hdc -speed=2 -graft-points /data/biz/code=/data/biz/code /data/biz/admin=/data/biz/admin /data/fun/skating=/data/fun/skating
The preceding command yields a DVD with the following directory structure:
data-.
     |---biz-.
     |       |---code
     |       |
     |       `---admin
     |
     |---fun-.
     |       `---skating

Obtaining Necessary Software

Most distros have everything you'll need to use growisofs. But in case yours doesn't, you can download and compile everything you need. Here's what you need to download:
Note that because the dvd+rw tools are wrappers for the cdrtools, the cdrtools are a must. Compiling dvd+rw-tools is as simple as untarring, then typing make with no arguments, and then copying the executables to a place on your path. This produces the following executables:
Compiling cdrtools is a little more complex. Start by untarring. Then read README.compile, and then read your OS dependent README, such as README.linux or README.linux, README.xxxBSD, README.BSDi, README.hpux, README.aix, README.solaris, or README.macosX. Many mention a make program called smake. README.linux exhorts you to use Gmake.linux, which is included in the cdrtools distribution. In Linux, do this:
./Gmake.linux
./Gmake.linux INS_BASE=/home/you/cdrtools_bin install
The preceding compiles everything and puts it in /home/you/cdrtools_bin/bin, /home/you/cdrtools_bin/sbin, and some library, include and man directories below /home/you/cdrtoops_bin. I think the intent is for you to set INS_BASE=/usr/local, which I didn't want to do because I didn't want to conflict with my distro-provided tools.

Here are some of the programs that are created:

Summary

Ordinary people have been burning CDs since the mid to late 1990's. DVD burning is a much newer phenomenon for non-millionaires. DVD burning still has some kinks.

First, you need to know the devicename to which to burn. Do this by burning in the K3B program, and using the following command to see the actual growisofs command spawned by K3B:
ps-efw | grep growisofs
Then there's the matter of speed. Start with a number that's the lower of either the maximum speed of the drive for the media type, or the maximum speed of the media itself. Completely check the accuracy of the burns, and if there are any errors, lower the speed.

In my opinion so far, the easiest and most likely to succeed method for burning DVDs is the growisofs program. When burning tarballs, the easiest method would look like this:
growisofs -use-the-force-luke=dao -dvd-compat -J -R -Z /dev/hdc -speed=2 data.tgz inst.tgz
When burning a series of directories, let's say /data and /backup/inst, you could do it like this:
growisofs -use-the-force-luke=dao -dvd-compat -J -R -Z /dev/hdc -speed=2 -graft-points /data=/data /inst=/backup/inst
Or, if you want to have the DVD reflect the directory structure of the hard disk, you would do it like this instead:
growisofs -use-the-force-luke=dao -dvd-compat -J -R -Z /dev/hdc -speed=2 -graft-points /data=/data /backup/inst=/backup/inst
To use a two step method, where you first make an ISO image and then burn it with growisofs, you'd do this, assuming backup of tarballs data.tgz and inst.tgz:
mkisofs -J -R -o myiso.iso data.tgz inst.tgz
growisofs -use-the-force-luke=dao -dvd-compat -J -R -Z /dev/hdc=myiso.iso -speed=2
Because -graft-points is identical in growisofs and mkisofs, you can burn whatever directory structure you want using the two step method, similarly to what you'd do with the one step method. Generally speaking, you'd use the two step method when you anticipate needing multiple copies, or when you need a snapshot of the directories that you can burn now and/or later.

See also: [ Linux Library | Troubleshooters.Com | Email Steve Litt | Copyright Notice ]

Copyright (C) 2005 by Steve Litt. -- Legal