Troubleshooters.Com and T.C Linux Library Present

Converting .ra Files to .ogg
Copyright (C) 2004 by Steve Litt

[ Linux Library | Troubleshooters.Com | Email Steve Litt | Copyright Notice ]

CONTENTS:

Why?

RealAudio is not free software. Source isn't easily available. As time continues, RealAudio files may become increasingly unsupported under Linux. In fact, I've found it increasingly more difficult to run realplay under Linux. The binaries distributed by the Real folks haven't been updated for years, and are hard to find. Linux libraries and kernels keep evolving, so it's possible that sooner or later a modern Linux distro will NOT be able to run RealPlayer under any circumstances. It's also possible that the Real folks will stop distributing the RealPlayer binaries.

Other .ra conversion techniques exist, but at least one was laid low by DMCA type legislation. The vsound project was abandoned by its originator for legal reasons, as described on http://www.mega-nerd.com/erikd/vsound/...
October 27th, 2002

 Although I have maintained vsound for nearly three years, I can no longer do so, nor can I continue to make it available from this web site.

 I live in Australia which has a law (Digital Agenda Bill 2000) which is similar to the DCMA in the US in that it makes the distribution of a devices for circumventing copyright protection illegal. I have neither the time, money or inclination to make myself a possible target for such legal action by companies with endless legal and financial resources.

 However, vsound is probaly available from other web sites. If you want a copy, you should search the web. Do not email me as I cannot and will not provide you with a copy.

 Erik de Castro Lopo.



Even if RealAudio is perpetually supported under Windows, conversion under Windows might be problematic.

As states pass "Super DMCA" laws, even the most innocuous format translations of files or content you legally own might become illegal. If "Digital Rights Management" becomes the law of the land, such innocuous format translations could become technologically impossible without dabbling in the dangerous and illegal world of black market contraband equipment and software.

The bottom line, if you have important .ra files, now might be the time to convert them to a different format, or risk losing them forever.

Be Legal or Be Gone

Do not use this document to violate the law. Convert only what you have the right to possess. NEVER redistribute audio not created by yourself.

To Whence?

What should be the target new format? .mp3 is patent encumbered, meaning although MP3 players are free right now, they could become costly or unavailable later. If the patent is asserted, it could become unavailable in Linux. Windows Media Player format is proprietary -- not a good repository for your important data. Of all alternatives, .ogg files offer the best future. Free software from the GNU people, .ogg files are guaranteed patent free. .ogg players and recorders cannot be removed from Free Software unless the government passes a "digital management rights" law.

Executive Summary

This document discusses a method to convert .ra files to .ogg files. This conversion is definitely not easy or obvious. It's not the only method, or even the best method. It requires a great deal of user intervention, and some trial and error. But the result is a reasonably clean .ogg format of your original .ra file.

You should also know that the author is not an expert on either sound or RealAudio. The information in this document details how I successfully converted a .ra file. There are almost certainly better methods, but such better methods are not widely known on the Internet.

Conversion Quickstart


NOTE:

Throughout this document, all examples assume that the original .ra file(s) was encoded at 11025 samples per second.

Obtaining realplay for Linux

This conversion method requires realplay for Linux. If you don't have realplay for Linux, grab it at http://scopes.real.com/real/player/unix/unix.html. Install it as per instructions and installer program. I found it rather straightforward, except that you might get the mostly black window or the no-sound as detailed in the Symptoms and Solutions section of this document.

Procedure

  1. Find out the sample frequency of the original .ra file(s)
  2. Set realplay to output ESound
  3. Run ESD
  4. Pipe and translate /dev/dsp to out.ogg:
  5. Immediately play the song
    1. ~/RealPlayer8/realplay mysong.ra
  6. When the song stops playing and the spinning stick stops spinning

Notes

Principles of .ra and .ogg

.ra files are RealAudio streamer files designed to be played from a hard disk. Although I haven't tested it, I believe these same principles could be used to convert true streamer .ram files to .ogg.

The conversion method detailed in this document is to "play" the RealAudio file in RealPlayer, pick off the digital output, and convert that. This is a little like recording from the line out jack of your radio. You don't truly record the radio waves, but instead pick off the play of your radio.

NOTE:

Throughout this document, all examples assume that the original .ra file(s) was encoded at 11025 samples per second.


In this case, you pick off the digital audio from /dev/dsp using the esdmon program. The process looks something like this:
Diagram of the process
The .ra song is played by realplayer, which is configured with the ESound  audio driver, which outputs via ESD to /dev/dsp because you have previously run a command somewhat like this:


esd -r 11025 -d /dev/dsp
The preceding sends esd output to the /dev/dsp device. The -r 11025 specifies that it will be sampled at 11025 samples per second. It is vital that you make this number match the sampling rate of the .ra files. If the number is lower than the sampling rate of the .rafiles, you'll lose high frequencies and crispness. It it's higher than the.ra files, the computer's software will try to sample where no samples exist. At first glance this might seem a great way to enhance highs for an inadequately recorded .ra file, but what it really does is interpolate the existing, gettting some of it right and some of it wrong. This is very much like zooming in on a low resolution picture -- you don't get a better picture -- you get pixelation. In this case, if the number is greater than the sampling rate of the .ra file, you get harsh high frequency distortion coupled with a distracting resonance.

How do you set realplay to output to the ESound audio driver? Perform the following within realplay:

View->Preferences->Performance_tab->ESound_Support

If you read the prompt by "ESound Support", you'll see that it's good for Linux only, and that ESD must be running. Luckily, you already started ESD with this command.

The following is the code to "pick off" data from /dev/dsp and convert it to .ogg:

esdmon | sox -r 11025 -v2 -c 2 -t raw -w -s - -t wav - | oggenc -r --resample 11025 -o $1 -

The esdmon program "picks off" the digital audio from /dev/dsp and outputs it to stdout. Via a pipe, that output is sent to the sox program, where the sampling rate is adjusted to match that set in the esd program, the filetype is set to raw because the output of esdmon has no header, -c is set to 2, meaning 2 channels (use 1 if the original .ra was monaural or if you want to combine channels to save disk space), specify sampling in 16 bit words, signed linear sample encoding, output .wav format, and adjust the volume to best match native .ogg files.

Now that .wav output is produced, the final step is to convert the .wav to .ogg. This is done by the oggenc program, with options to resample at the original .ra file's sample rate, output to a specific file, and accept the incoming input as raw data.

The following table shows the meaning of the options for the sox and oggenc portions of the command:

sox arguments and options
oggenc arguments and options
sox
Sound translation program
-r 11025
Resample at 11025
-v2
Crank up the volume to level 2
-c 2
2 channels (stereo)
-t raw
Incoming data is raw (no header)
-w
Incoming data samples are 16 bit words
-s
Incoming data samples encoded as
signed linear  (2's  complement)
-
Incoming data read from stdin
-t wav
Outgoing data is .wav format
-
Outgoing data is sent to stdout
oggenc
The ogg encoder
-r 
Incoming data is raw
--resample 11025 
Resample at 11025
-o $1 
Output file is arg1
-
Get input from stdin

Symptoms and Solutions

Although at this time (7/20/2004) the  realplay program can be obtained without payment, it is a proprietary black box depending on many unknown (and unknowable because you don't have the source) variables. Therefore, it can act intermittent with respect to computer, OS installation, reboot, time, and who knows what else. These unknowns make troubleshooting very difficult.

Perhaps the best you can hope for is to get it up long enough to convert your .ra files. It might even require installing a different distro, possibly an older distro, on a separate box.

This symptom and solution article attempts to guide you through solutions to the most common problems. Keep in mind that these solutions might not always work, and even more perplexingly, the listed error modes might not produce the listed error conditions. Do your best, nurse realplaythrough the conversions, and in the future try not to record or obtain anything in .ra format.

Realplayer doesn't run, window mostly black

This typically occurs when the esd program is running, but realplay is set to use native sound drivers. The solution is to kill the esd program, wait for a while, then try again. If you look in the /etc/esd.conf file you'll see a delay called spawn_wait_ms=300 and another delay -as 2. These are two different delay intervals that might, or might not need to be exceeded before the killing of the esd program takes effect.

Once you can get realplay running correctly, you can manipulate its menu to switch it to use ESound:

View->Preferences->Performance_tab->ESound_Support

From that point, exit realplay and in a different command prompt run esd:
esd  -r 11025 -d /dev/dsp

Cannot open the audio device

The complete error message might look something like this: "Cannot open the audio device. Another application may be using it." First, check to make sure no other sound programs are running, and shut them down if they are. If that doesn't help, the esd program probably isn't running or isn't running right. Exit the realplay program and run esd. Assuming the .ra program was recorded at 11025 samples per second, it would look something like this:
esd  -r 11025 -d /dev/dsp
Then run realplay. It should now play its opening welcome message, and should be able to open and play other .ra programs.

Inability to play more than one realaudio clip

4Occasionally you'll get the "Cannot open the audio device. Another application may be using it." message when trying to play a second sound clip. Under these circumstances, if you look at the window running the esd command, you'll probably find that esd terminates after the completion of a realaudio clip. The way I've found to make esd more durable is to modify /etc/esd.conf. The original probably looks something like this:
[esd]
auto_spawn=1
spawn_options=-terminate -nobeeps -as 2
spawn_wait_ms=300

BACK UP YOUR EXISTING /etc/esd.conf!!! Then modify /etc/esd.conf. Try eliminating the -terminate and then add the -d and-r to it:

[esd]
auto_spawn=1
spawn_options=-nobeeps -as 2 -r 11025 -d /dev/dsp
spawn_wait_ms=300

Now you can run esd with this simple command:
esd
Be aware that the preceding is a coathanger solution, and like all coathanger solutions that "fix" the symptom rather than eliminating the cause, it will cause side effects on all other programs using ESD. Therefore, once you've converted your .ra files, be sure to restore /etc/esd.conf from backup. If you desire, you can save the tweaked file as /etc/esd.conf.ra2ogg.

Distortion: Sound displeasing, screachy, overly treble, oscillatory, ringy, scratchy, noisy, artifacts


Bad sound, especially distortion of high frequencies, is typically caused by a too-high sample rate, an off-divisor sample rate, or even a too low sample rate. The first step is to try the exact sample rate of the .ra clip. The realplay window gives an approximation of the sample rate. The exact rate can be found by playing the clip, and while the clip is playing, do this: View->Statistics.

Try running esd with the -r value as the exact rate reported by View->Statistics. If that sounds bad, try finding a simple fractional divisor near that amount. For instance, 44100/(25/9) is 15876. If such attempts still sound bad, trial and error may be necessary. Start with 11025, and move up and down to see the improvement or degradation. Remember, you're looking for something not too harsh and not too muffled. Also, remember that you'll likely have some distortion no matter what you do. RealAudio was meant for efficient web streaming, not for great fidelity, and you can't completely recover the information from its highly compressed format.

Remember also that grossly low rates introduce distortion, often in the form of echos and other artifacts. Also, if you go too low, esd  refuses to run at that rate and instead defaults to 44100 with 8 bit samples. 8 bit samples are guaranteed to give horrible sound.

The following table lists some sample rates:
 
Rate
Divisor
Fidelity
Comments
44100
44100/1
Full fidelity
CD quality
22050
44100/2
half rate
You'd need good ears
to discern the decreased quality
16000
44100/2.75625

Probably won't convert well at this rate
15876
44100/(25/9)
custom
divisor

14700
44100/3
third rate
Decreased highs, but still enjoyable
11025
44100/4
fourth rate
Lowest sample rate enjoyable as music
8820
44100/5
fifth rate
Good, understandable speech
4410
44100/10
tenth rate
Muffled speech, but reasonable understandable.
A little worse than telephone quality.
Consonants such as S and F cannot be discerned
except by context.

Another source of distortion is a too high volume multiplier in the sox program. This distortion sounds more like ordinary clipping (volume too high on an amplifier) rather than the high frequency phenomenon caused by a too high sample rate. You can test by reducing the volume multiplier to 0.5 and boosting the speaker volume to a hearable level. If the distortion goes away, the problem was a too high volume multiplier. If not, it was something else.

Song plays too fast or too slow

Check all your sox and oggenc options. Try them identical to those in this document, and see if the problem persists.

Mass Conversion

What if you have a directory full of .ra files all recorded at similar volume and at identical sample rates? As far as I know, realplay itself doesn't have a mass conversion facility. Theoretically, you could convert the whole playlist to a single .ogg, and then use an audio editor to break it into individual songs. Do it if practical.

I found the following to be effective, although requiring intensive operator intervention. First, create the following esdmon.sh:
esdmon | sox -r 11025 -v2 -c 2 -t raw -w -s - -t wav - | oggenc -r --resample 11025 -o $1 -

Now create the following do1.sh, which converts a single song contained in /mnt/songs:

FN=$1
gnome-terminal -e "./esdmon.sh $FN.ogg -" &
~/RealPlayer8/realplay /mnt/songs/$FN.ra

In the preceding, note that arg1 is merely the filename without extension, and without directory path. Thus, to convert /mnt/songs/mysong.ra to ./mysong.ogg, execute the following command:

./esdmon.sh mysong
Look closely at the do1.sh script. Notice that for each song, it runs esdmon.sh in a terminal in the background, and then runs realplay, on the appropriate .ra song, in the foreground. Thus, the conversion script is guaranteed to be running before the song starts in realplay, thus preventing elimination of the front of the song. However, the only way to stop esdmon.sh is to Ctrl+C the terminal BEFORE closing realplay. If you had closed realplay before terminating the terminal, you'd have gotten multiple terminals, and who knows what that would have done to your audio.

Finally, create a script to repeatedly run do1.sh on every song in /mnt/songs:

./do1.sh mysong
./do1.sh yoursong
./do1.sh freds_song

Run the script, and notice a terminal running esdmon.sh and a realplay window are immediately started, with music playing and the terminal counting time and marking movement with a spinning stick prompt.

For each song, do the following:
  1. Wait for the spinning stick to stop
  2. Ctrl+C the terminal to terminate esdmon.sh
  3. Close the realplay program. A terminal and realplay window for the next song automatically runs.

Immediately Converting .ram Files

I haven't tried it, but I believe that this method can also be used to create the pure streaming version of RealAudio files, .ram files. You'd simply run  esd with the proper options, run the esdmon | sox | oggenc command chain discussed previously in this document, and then play the piece. As long as your realplay is set up for the ESound driver, it should work.

However, keep in mind that as time goes on, the likelihood of realplay's continued Linux availability and Linux functionality decreases. It's best to try to distance yourself from the .ra format entirely, and concentrate on .ogg, or else on .mpg, which you can then convert to .ogg.

Summary

The .ra format is a great way to ship music over a modem connection to be played realtime on the other end, but it's not particularly high fidelity. Furthermore, it's primary playback mechanism is The Real folks' realplay, and there's no guarantee of continued Linux availability and function of realplay. If realplay becomes unusable on Linux and nothing else takes its place, your .ra files become unplayable AND unconvertable. Perhaps they could be converted on a Windows box. Perhaps!

In other words, it's time to convert any .ra files you own to another format. But what format? .mp3 is patent encumbered, meaning at any time huge fees could be charged for it. Remember how Unisys, the owner of the lzw compression used in .zip files, tried to impose large fees on webmasters displaying.gif files? This could happen with .mp3 files.

The .wav and many other formats are too big. The Windows Media Player format is proprietary making it not much better than .ra.

Then there's .ogg. It's patent free, so you can use it with confidence now and in the future. The only way to encumber .ogg is for the government to pass specific laws banning it as part of a "manditory digital rights management" law. If that happens, our society, especially Linux users, face much more daunting concerns that loss of songs. Such a law would make Linux itself illegal.

So the question is "how to convert from .ra to .ogg? Here are the basics:

  1. Find out the sample frequency of the original .ra file(s)
    • Throughout this document we assume 11025
  2. Set realplay to output ESound
    • View->Preferences->Performance_tab->ESound_Support
  3. Run ESD
    • esd  -r 11025 -d /dev/dsp
  4. Pipe and translate /dev/dsp to out.ogg:
    • esdmon | sox -r 11025 -v2 -c 2 -t raw -w -s - -t wav - | oggenc -r --resample 11025 -o out.ogg -
  5. Immediately play the song
    1. ~/RealPlayer8/realplay mysong.ra
  6. When the song stops playing and the spinning stick stops spinning
    • Ctrl+C out of the esdmon command
    • Close realplay

The course of conversion seldom runs smooth. If you have problems, look in the Symptoms and Solutions section. If you want to know a little more about the underlying technology, take a look at the Principles of .ra and .ogg section. Mass conversion and immediate stream conversion are possible.


See also: [ Linux Library | Troubleshooters.Com | Email Steve Litt | Copyright Notice ]

Copyright (C)2004 by Steve Litt. -- Legal