Volume
2, Issue 5, May 1998
You loved it! According to my stats (url at bottom of page), in early and mid April, the April issue was Troubleshooters.Com's second most hit page -- right behind the home page. And the email!
Given the popularity of the theme, this month is Corporationally Incorrect Technology, Free Software. In an industry increasingly dominated by a monopoly whose major innovation seems to be marketing, these free products are actually gaining users the old-fashioned way: they earn them with quality. Microsoft -- are you listening?
This issue is blessed with top-notch authors with far-reaching reputations. Amit Patel, a C++ guru I often consult on the finer points of the language, has a marvelous piece on Python, a free software language he uses when most of us would use PERL. Forrest M. Hoffman and William W. Hargrove of Oak Ridge National Laboratory discuss the process by which their organization built a supercomputer out of surplus 486's and LINUX.
So kick back, relax, and read this issue. And remember, if you're a Troubleshooter, this is your magazine. Enjoy!
The license that's usually called "free software" says you can give the software away, you can sell it, you can even modify it and give it away or sell it. But the complete source code must be part of the package, and the buyer must be given all the same rights the seller had.
So if somebody tried to sell their modification to a free software product (lets say Linux for example), for a million dollars, they'd sell one package at the most. After that, the person who bought it would resell like the autumn leaves, locking the million dollar seller out of the market. In practice, "free software" sells for under $100.00, and represents not a license price, but the convenience to the customer of a complete package on a CD, with a tested installation program.
As an example, take Red Hat Linux, the Linux version that seems to be today's favorite. They'll sell you a CD set for $49.00. The CD contains both free Linux, and a few non-free software utilities (including Reelaudio and Metro X). Installing the non-free utilities on several computers requires purchasing a license for each computer.
But the core Linux kernal and its core utilities, including Xfree (GUI interface), POP server (email), Apache (http server) and Netscape (remember, it's free software now) can be installed on as many computers as you want. In other words, for $49.00 you can create multiple ISPs (minus the T3), or multiple Linux supercomputers.
The $49.00 you paid for the CD is only for the convenience of having it all together in an easy to install package, plus of course the license to the included non-free utilities. In other words, it is perfectly legal for you to go to your friend, borrow his Red Hat Linux CD, and use it to install Linux (without the included non-free software utilities) on your machine and a hundred of your closest friends.
Personally, I'm going to purchase my own CD. Call it my contribution to a free market economy. And if I need a programmer, I'll try to hire a free-software author. Another contribution to a free market economy. I'd suggest you hire free-software authors too.
It's just not a political statement, or even a contribution to assure a free market in the future. It's also your best hope of quality work. Because free-software authors have entirely redefined the concept of on time and under budget.
The light turns green. The kid hits the gas first. After all, he's younger with quicker reflexes. His friends in the back seat snicker. Then my three deuces kick in, quickly quenching the 426 hemi's thirst. Sure Grip differential firmly delivers the power to each rear wheel. Smoke pours off my tires. Parallel black skidmarks extend a hundred feet back. Smiles becomes wide-eyed surprise as the wiseguys in the Mustang see me pass, with the front tires a few inches off the ground.
Nice fantasy, huh? Too bad it will never happen. I support a family of five, so the $35,000 I'd have spent on the Dart go to a family car, or college education, or whatever. And besides that, I don't drag race.
One kid brags about his Pentium 400 with a 100mhz bus, DVD, AGP video, Microsoft Back Office and all the development tools. He shows me his website, which was "cool site of the day". The kids crack up when I tell them I've got em beat. They dare me to show them what I've got.
We pile into their brand new Mustang and drive to my house. The kid with the computer spends the trip talking on his cellphone. Once in the house I lead them to a small room containing a hodgepodge of fifty 486's, Ethernet wired in parallel, with Red Hat Linux running each one. A T3 has replaced the T1 that replaced the ISDN that replaced the dialup I had when I first built the system out of four junkyard 486's.
Just for fun I go to a console and pull up a screen that counts accesses. Within a minute my system has complete database accesses for 1000 visitors. I explain that I've wired 50 486's into a supercomputer, created an travel reservation system, and put it on the Web. It services 200,000 customers a day. I need to upgrade my supercomputer, so I'm looking for another 50 486's, or maybe even some old Pentium 100's. Anything cheap will do.
The kids leave my house a little older and a little wiser. They look in my garage at an old Dodge Dart, and I know they're wondering what I've got in it.
Now anyone with the technical knowledge to cable together hardware and software can put together a supercomputer that makes the Cray's of our youth look like balloon tire bicycles.
The two examples in the first paragraph are just the tip of the iceberg. A search of the 'net will show many such supercomputer projects. They all share one thing in common - LINUX. Linux seems to be a natural when it comes to inexpensive parallel supercomputers. If you want to be the first on your block with a supercomputer in your bedroom, the only things you'll need are time and knowledge. A 16 node supercomputer could be cheaper than a modern mail-order Pentium II.
The next article is by Forrest M. Hoffman and William W. Hargrove of Oak Ridge National Laboratory, explaining how they built a supercomputer essentially free. Read on -- this is the future!
 The Stone SouperComputer. |
When Thomas Sterling and his colleagues at NASA's Goddard Space Flight Center conceived of and built a "pile of PCs" for scientific parallel computing, they were the first to deliver a cost-effective alternative to the large, custom-manufactured supercomputers offered by only a handful of vendors. In an effort to realize significant cost savings while providing a parallel computer for environmental science at Oak Ridge (Tenn.) National Laboratory, we proposed--about one year ago--extending their work with new hardware and faster network interconnections. With our proposal rejected and much computing work staring us in the face, we turned to the resources that were readily available: surplus Intel 486 PCs.
Commandeering a nearly-abandoned computer room and scavenging as many surplus PCs as possible--from Oak Ridge National Laboratory, the Y-12 production plant, and the former K-25 site (all federal facilities in Oak Ridge)--we setup a "chop shop" to process machines and proceeded to construct the first no-cost parallel computer system. Aptly named the Stone SouperComputer (after the childrens' fable entitled Stone Soup), our cluster has--since that time--grown to 74 nodes with more soon to come on-line.
The machines contain a host of different motherboards, processors (of varying speed and design), controllers, and disk drives. Each is connected to a private ethernet network for inter-cluster communications with one of the nodes also connected to the external network for logins and remote access. The machines all run Red Hat Linux and have GNU compilers and the PVM and MPI message passing libraries for parallel software development.
Already the Stone SouperComputer has been used to perform multivariate spatial cluster analyses on high-resolution national landscape maps--an application well suited to the machine. In addition, it has run a number of simultaneous serial tasks, which--while not at all glamorous--has resulted in significant reductions in the time needed to generate a desired product.
It turns out that the Stone SouperComputer is a good place to develop parallel codes because the software tools are available in the Linux environment and the nodes don't run so quickly that debugging is difficult. Once a code is running on the Stone SouperComputer, it can usually be ported to other parallel supercomputers--including the expensive behemoths found in the country's large supercomputer centers--with no modifications.
Our next step is to start backfilling; with Microsoft poised to release an even more demanding operating system, we expect to receive a flood of cast-off machines in the coming months which will allow us to go back and upgrade nodes in the cluster. We have already started seeing a few Pentiums among the masses of computers we either fashion into nodes or cannibalize.
For more information, visit the Stone SouperComputer homepage at http://www.esd.ornl.gov/facilities/beowulf/
It was 1985 when Richard Stallman of MIT published his GNU Manifesto. Few heard of it, but if they had it would have sounded strange in those heady days of capitalism, with Ronald Reagan and Margaret Thatcher leading the world and even the Soviet Union's new General Secretary, Mikhail Gorbychev, moving his country toward capitalism.
The GNU Manifesto advocated free software, specifically a free UNIX workalike, which Stallman then began creating with the help of others. In the Manifesto, Stallman prophetically described the process of getting this to happen, including ideas in licensing (must pass on source and all rights to the receiver, etc).
He countered objections to the new paradigm, including the obvious "programmers deserve the fruits of their labor". He stated that although programmers would make less in a free-software economy, they'd still be well paid.
He described an idealized world of no-charge software creating harmony among programmers, and freeing business from being held hostage by software vendors. Yes, it must have sounded pollyanishly idealistic in those stock market soaring Reagan/Thatcher days.
By the early 90's, most of the GNU Unix workalike was done. Only the kernal remained. In 1991 Linus Torvalds created a kernal, making GNU a reality. In fact, it might be more accurate to call Linux GNU/Linux, since it uses the Linux kernal and many GNU utilities. But in one of those quirks of history, the operating system was named Linux and became the battle cry of those wanting a free choice.
And now programmers grow richer, not poorer. Because there's always customization needing to be done. And who are you going to hire -- some guy off the street, or a co-author of a major piece of software.
It's often asked, "what difference can one man make?". Depends on the man. I spent 1985 hanging out with my friends, listening to Wham and trying to break dance on the Venice Boardwalk. That same year Richard Stallman wrote the Manifesto that changed the software world forever.
Microsoft has seemingly conquered or dwarfed Microrim, WordPerfect Corporation, Novell, Netscape, Borland, Powersoft, IBM, Lotus, Symantic, Corel, and the other powerful software entities of our time. But there remains a group of programmers who can't be conquered by economic starvation, for their software is free. They seemingly work at their jobs all day, then develop free software at night. They've developed GNU/Linux, Perl, Python, Apache, Free Netscape, and a host of other OS, development, web and app software whose use appears to be growing while other non-Microsoft market share is shrinking. They now appear to be taking market share from Microsoft.
In 400 AD Rome seemed invincible. In 476 the Barbarian leader Odoacer deposed Roman emporer Romulus, thus ending the united Roman Empire. I'm really curious what our software world will look like five years from now.
Microsoft Plans Stealth Blitz to Mend Its Image |
Here's what happened. The LA Times obtained confidential documents planning
a number of unusual, and some say unethical, tactics to simulate an outpouring
of "grass roots support" for Microsoft. To quote a portion of the article:
Letters to the editor are to be solicited from regional business leaders. Opinion pieces are to be written by freelance writers, and perhaps a "national economist," according to one document. The writers would be paid with costs "billed to Microsoft as an out-of-pocket expense." |
The article goes on to describe a level of sneakiness, denial and coverup former president Nixon would have appreciated.
A link to this article appears in the "URLs Mentioned" section of this issue of Troubleshooting Professional.
Microsoft blew it big time. If you're anything like me, a month ago you'd see an anti-Microsoft article flamed with a dozen pro-Microsoft letters and say "maybe there's something I'm missing". Now you look at the pro-Microsoft letters and say "I wonder how much Microsoft paid".
A few brave programmers, denied access to the Glass House, try to eak out a living writing little programs on their kitchen tables, using DBASE III, C or assembler. These Kitchen Table Programmers are subject to derision and laughter.
The ultimate kitchen table programmers, Bill Gates and Paul Allen, are heros, with their productivity enabling their MS-DOS on every desktop.
If you want to make a living programming, you pay homage to the New Boss.
A few brave programmers, denied access to the Glass House, try to eak out a living writing free software on their kitchen tables, using GNU C++, Perl and other free software. The software is perfectly capable of running an enterprise, it's free, and support is superior to that of the Glass House Gang. But it isn't sky blue with clouds. These Kitchen Table Programmers are subject to derision and laughter.
To be continued...
Users of DBI/DBD have written me telling of enterprise applications running solidly, every day using DBI/DBD. It appears to offer a very real alternative to the "our agents are busy helping other customers, please have your credit ready" crowd.
I'm in no way an expert on this subject. To get the info from the horses mouth, see the DBI home page (listed in the URL's Mentioned section of this issue).
I have a fantasy. Go "dumpster diving", collect 16 obsolete 486's, wire em together with Ethernet, install Linux, get DBI and the proper DBD, put in a T1 connection (the only expensive part of this fantasy), and go worldwide with an OLTP system, out of my living room.
Kind of puts a new slant on the phrase "where do you want to go today".
Tomorrow's Supercomputers from Yesterday's PC's? |
Special copyright terms apply for this slogan. Here they are:
Here's the one paragraph summary: Python is a clean object oriented scripting language that is easy to read and write, yet is modular and scalable. Python is designed to be easily embedded into C/C++ programs or extended with C/C++ libraries, and is a good glue language to bring different systems together. Python includes libraries for shell access, text processing (including Perl-like regexps), database access and persistence, Internet protocols (WWW, CGI, SMTP, NNTP, FTP, Gopher, Telnet) and formats (HTML, XML, MIME, UUencode, mailbox), and GUIs (using Tk, MFC, Mac, AWT, Qt, and other toolkits). Python is used at places such as Infoseek, Four11, Lawrence Livermore Labs, and NASA. There are implementations for Unix (including Linux), Macintosh, Windows, OS/2, BeOS, Amiga, VMS, DOS, and Java. The best part is -- Python is free.
Python is often used in the same situations that one might use Perl, so how does it compare? Python is closer to a general purpose language (like Java or C++) than Perl: it was built from the start to have packages, modules, classes, good data structures (including lists/arrays, dictionaries, closures, and tuples), and objects. Its variables are local by default, and are named like in most languages -- letters, digits, and underscores, without any symbols indicating whether they're scalars or vectors. Python values don't change types automagically; like most general purpose languages, if you have "3" and you want 3, you have to call int() on it, and if you have 3 and you want "3" you have to call str() on it. Data types are kept distinct, and it's not hard to write new data structures like graphs and skip lists in Python.
On the other hand, since Python is more "general purpose" than Perl, it doesn't provide for special variables for tasks that are common in Perl (but aren't as common in Python). For example, if you want to scan standard input for a pattern and print out lines that match, in Python you'd have to load the regular expression and string libraries, then create a loop that reads a line at a time from stdin and performs the match. In Perl you can find some combination of $'s and other symbols that will do the looping and matching for you.
Just to see how simple code compares, here's a loop that prints out
things in a list:
| Perl | Python |
|---|---|
@food = ("banana", "cookie", "spam");
foreach $morsel (@food) {
print "$morsel\n";
if ($morsel eq "spam") {
print "Yuck\n";
}
else {
print "Yum yum\n";
}
}
|
food = ["banana", "cookie", "spam"] for morsel in food: print morsel if morsel=="spam": print "Yuck" else: print "Yum yum" |
As you can see, they're more similar than they're different: both languages don't require you to declare your variables, have built in list/arrays, and have an easy way to iterate through list/arrays. The Python code doesn't treat scalar ($) and vector (@) variables differently; all values can be assigned to any variable. When writing large programs, Python scales up better than Perl, but for small programs, they're either equivalent or Perl is better.
Python is also sometimes used when Java might have been used. Both languages are fairly portable, both languages compile to bytecode, and both language are object oriented. Python is a higher level language, though, so it's usually significantly easier to write a Python program (unless you really need Java's libraries, and even then, JPython gives you access to Java's libraries, like AWT). It's much easier to use lists or dictionaries (associative arrays) in Python, and the object system is far more flexible. On the other hand, Java has some nice development environments, a lot of support from large companies, and some nice concepts for security (such as interfaces and final classes). Python allows you to create new methods in each object, not only in a class; it allows you to inspect and change the contents of any module or object; and you can even change the way fields and methods are found inside an object. Python gives you more flexibility, but it makes it hard to make guarantees; that's where the restrictions in Java are useful. There are times when you don't want programmers to be able to change how your code behaves, and strong encapsulation and final classes can help you do that.
There are always tradeoffs when designing languages. Looking at Perl vs. Python, Perl seems to be more specialized, and better at handling its specialty; Python is more generalized, and can do well when in a wider range of programs, but many things that are automatic in a special purpose language become explicit in a general purpose language because they're not always appropriate. Looking at Java vs. Python, Java is more restrictive and Python more flexible, but flexibility always comes at a price: it's harder to guarantee any properties about your program. One tool isn't right for every job!
When it comes to small text processing or shell programs, Perl probably wins. When it comes to writing web applets or more traditional applications (with lots of user interface code), I'd probably pick Java. If speed is important, C++ is a good choice. However, for things in between -- programs where the user interface and raw processing speed aren't a big deal, but modularity, maintenance, and data structures are important -- Python is a good choice.
Why isn't everyone using Python? I think it's because there is a general tendency to use whatever is being used most, because that's where you can find the most support and tools. It's not enough to be better; it has to be more popular, and often that means being first. That's why people use PCs and Windows and C. If you're working on a project that's using Windows or Perl, it makes sense to use Windows or Perl yourself. However, if you're starting something new, take a good look at Python.
If you have questions you can go to the newsgroup (comp.lang.python) and get answers from experts, including the author of Python, Guido van Rossum. Usually, however, I can find the answers in the language documentation at http://www.python.org/doc/, which is split into a language reference, a library reference (the most useful part), an extensions manual, and a tutorial. There's also an O'Reilly book, Programming Python, that comes in handy.
Since I learned Python, I've used it for almost every program that doesn't require a GUI or raw speed. It's refreshingly fun!
References: There's a Python subsection in the URL's section at the bottom of the magazine.
By submitting content, you give Troubleshooters.Com the non-exclusive, perpetual right to publish it on Troubleshooters.Com or any A3B3 website. Other than that, you retain the copyright and sole right to sell or give it away elsewhere. Troubleshooters.Com will acknowledge you as the author and, if you request, will display your copyright notice and/or a "reprinted by permission of author" notice. Obviously, you must be the copyright holder and must be legally able to grant us this perpetual right. We do not currently pay for articles.
Troubleshooters.Com reserves the right to edit any submission for clarity or brevity. Any published article will include a two sentence description of the author, a hypertext link to his or her email, and a phone number if desired. Upon request, we will include a hypertext link, at the end of the magazine issue, to the author's website, providing that website meets the Troubleshooters.Com criteria for links and that the author's website first links to Troubleshooters.Com. Authors: please understand we can't place hyperlinks inside articles. If we did, only the first article would be read, and we can't place every article first.
Submissions should be emailed to Steve Litt's email address, with subject line Article Submission. The first paragraph of your message should read as follows (unless other arrangements are previously made in writing):