You Used Perl to Write WHAT?!

Esther Schindler writes "Developers spend a lot of time telling managers, 'Let me use the tool that's appropriate for the job' (cue the '...everything looks like a nail' meme here). But rarely do we enumerate when a language is the right one for a particular job, and when it's a very, very wrong choice. James Turner, writing for CIO.com, identifies five tasks for which perl is ideally suited, and four that... well, really, shouldn't you choose something else? This is the first article in a series that will examine what each language is good at, and for which tasks it's just plain dumb. Another article is coming RSN about JavaScript, and yet another for PHP... with more promised, should these first articles do well."

Both sides...

I always see both sides of the 'right tool for the job' problem.

Having the right tools is great for current productivity, but it's hell on expenses and new recruits. If you use a different tool for every job, you need to maintain all those tools and a task force that's able to use all of them. Sometimes the 'right tool' is one that fits the company as well as the job.

Re:Both sides...

If you use a different tool for every job, you need to maintain all those tools and a task force that's able to use all of them.

That's why the "right tool for the job" is sometimes the tool that meets the greatest cross-section of a company's needs rather than a jumble of tools that are ideal at a lot of little tasks.

e.g. While it's fashionable to hate Java these days, you have to admit that it does have a rather massive cross-section of needs it can meet. Thus one of the reasons why it's so popular in large companies. Yet a smaller company might find more value in using Ruby toolkits to do all their work. Ruby may not be ideal for some of the less glamorous back-end tasks, but tools like Rails gain so much on the front end that Ruby meets a greater cross-section of needs than Java would.

Re:Both sides...

I hate the "right tool for the job" cliche. Not because it's necessarily wrong, but because it tends to be used by people who automatically assume that their tool is the right one and wish to stop any serious discussion about other possibilities.

language vs library

I wonder if this whole discussion is off the mark. Languages are for the most part trivial. And universal. "It's the libraries, stupid" is sometimes how I feel. If it was easy to link in or call any library function from any language, then half of this discussion would immediately be seen to be irrelevant. So Perl is "the right tool for the job" because it has the ability to apply regular expressions to strings? But, you know, C can do that too thanks to this PCRE library. Hashes? C can do that too via another library. In case anyone has forgotten, Perl itself is written in C. I read that Perl 6 has vastly improved the interface to other languages, especially C libraries.

These day, whenever I write a new program it often feels as if I'm creating yet another language. A simple, superficial, limited language, but nonetheless, a language. Program needs a configuration file? Whip up a suitable format (language) for that. Needs to save data? Barf out this big data structure into a YML file. Want some way to run the interface from a batch process, or otherwise automatically? Start turning the user interface into a language. Want to connect Perl 5 and C? Get acquainted with XS, a "language" the Perl folks felt it necessary to create for that purpose, because Perl 5 wasn't good enough alone. Want to compile a large project written in C? Get familiar with the language of Make, because while C certainly could do it, C isn't so good for that. Is ANT a "language" for building Java projects? Where's the line between language and library?

I suppose where things lead to a new language is when someone wants to implement a new concept and the established ways aren't good enough. Or has a way to eliminate a bad programming practice, but some elements of an existing language must be dropped to do it. For instance, wouldn't be nice to have variable length parameter lists in C, as C's own printf function does? Too bad it's such a pain to do that in C. How about lazy evaluation and currying so we can have infinitely long parameter lists? Oops, guess the C call stack can do recursion, but isn't too well suited to expressing that sort of thing, time to make another language, Haskell. Do we want to pass along a pointer to a structure, or a copy of a structure? Java defaulted to pointers where C did not, but then said Java didn't use pointers. Nice not to have to type in ampersands and asterisks all the time, but still, I find the thinking misleading. Then there's garbage collection. The consensus is that garbage collection is overall a good thing, but that a good programmer can do better than the automatic garbage collector. And so on.

Re:is your company weak?

I kinda gotta agree with the parent here. Programming is a mindset. As one of my professors once told us: "50% of what you learned here will likely be outdated within 2 years of graduation. The other 50% will last you the rest of your lives." If you want to be a valuable, well rounded programmer, you need to keep up with the trends and learn a few things here and there. If you know HOW to program at a conceptual level, picking up the syntax of a new language shouldn't be all that hard. And that's why concepts and structures are stressed so heavily in Computer Science. The lessons you learn there should be language independent.

Re:is your company weak?

It isn't that hard to pickup a new language, unless the reason you have to pick it up is to maintain an app written by someone long gone...

... who didn't know how to use it properly either.

Re:is your company weak?

That's pretty much my point. While I was at college, I worked with Java, C, C++, Fortran, VB, and SPARC Assembly. I have a vague working knowledge of VB and Java syntax. I still remember C and C++ pretty well as I use those still (and I use a lot of PHP as well, but that I picked up after I was out). If you asked me to write something in Fortran or SPARC Asm at this point in time the best I could do without a reference book next to me is a blank stare (The Fortran class I took wasn't even geared towards CS majors. It was just there for Liberal Arts people to get a required computer credit - I took it because for a 3rd year CS student it was like a free A+ to add to your GPA :)). I just haven't used it recently at all and the syntax is lost.

HOWEVER, I do remember quite well what threads are, what a semaphore is, what a binary tree is, the difference between a bubble/quick/radix sort, the concept of object oriented design, etc. I wish I could say I remembered UML modeling but honestly, I hated that darned part of CS and never paid attention there anyways :P. But, regardless of the percentage, the point still stands: syntax is trivial. The important part is knowing how to think like a programmer. If you can do that the rest just falls into place.

People keep saying this...

...and I tend to think they just aren't doing anything *significant* in just-learned languages.
The problem is not learning the syntax and basic idioms. Agreed, that's pretty quick, particularly if you have a good reference.

The problem (and the time sink) is the *ugly* side of every language. The parts of the standard libraries that sucked, and were reimplemented elsewhere (but you gotta know that...). The functionality where everyone who "lives with" the language grabs X open source library to implement -- not Y! it's a POS! -- but you don't know that yet. The language features that have secret, illogical gotchas for special cases. The bugs in the compiler or interpreter that are easy to avoid -- once you've been burned once. The code that will break cross-platform compatibility for obscure reasons. The code that will make it almost impossible to internationalize later, because you didn't learn how that support worked yet.

Granted, the cost of these things with any reasonable mature language should not be enormous (though it depends how long you go down a wrong path...), and you can allow for it, but it's always a significant risk *especially* if you don't have someone on the team (perhaps the new team who has to maintain your old code) who's already more-or-less expert level.

But either way, you have to allow *something* for that cost, and sometimes it's not worth it just to use the absolute best tool for the job when you have a pretty close fit available.

Its simple

It's obvious that perl is best suited to some tasks over others. So is just about any language. The reason you can't use it in most cases is because it's harder to find developers to support it after you disappear than say Java or C#. That's why I won't be writing our new trade routing system in D. [wikipedia.org]

Glue and objects

The list missed the most important part of perl. A glue language. Python and a few other languages claim they can be glue languages but that's pretty much a joke to anyone who knows both fluently. Perl is the ultimate glue language for combining diverse output so different programs from different sources, written decades apart in different languages can all work like a well oiled machine.

The other really odd experience for me was learing object oriented programming. I had been programming in objects since I was first introduced to them when the first NeXT computer came out. I used java. And C++ and such. I thought I understood objects.

Then one day I learned to program object oriented in Perl. An I learned that while I was fluent in object oriented usage, I really had a pathetic understanding of how they worked and what was actually possible with objects.

Perl objects are sort of like owning a copy of grey's anatomy or "the visible" man. You son't just see that arms connect to torso's from outside but you see all the sinews and bones and blood.

It's actually amazing how so many things we think of as different concepts in object oriented programming and data bases are actually different reflections of the same trick. And that's the trick perl use to make objects.

in perl, an object is any variable that has an attribute that can store a list of package names.

Let's see what you can do with that.

Hmmm.... well that list can be your inheritance heirarchy so each package is what you search for methods. But notice that since it's a mutable list a perl object can do something else that most object oriented languages cannot. A variable can change it's "inheritance" list after the fact. it can change it's own class.

Okay Now this is just a single variable so where to we get attributes of the object? Well, if that variable is say a hash (dictionary) then we can just use the key's as the attribute names. so if were to write self.foo in C++, you would write self->{foo} in perl.

More fun: let's say you call a method() or ask for an attribute on a variable that does not exist. Well, a perl object can just add more packages to it's inheritnace list. Or it cold write the method on the spot and add it too it's own inheritnace. "I'm my own grandpa". I've used this trick many times to create tables. I don't write any of the "get" or "set" methods. instead I just intercept the call to the method "setfoo()" which never existed cause I never wrote it, then I have perl create an attribute called foo: Self->{foo} = "something". then I have perl write a subroutine called "setfoo" and add that subroutine into a package namespace and put that in it's inhereticnace list.. ("like adding methods to a C++ package outside the declaration". (programming tip: obviously this is could lead to problems with typos, so I also provide the variabel with a list of all allowed attribute names--- but of course I can always add to that list later).

Now something more exotic. The hottest thing in Data base programming is the realization that sometimes column centric data bases are better than traditional row-centric data bases structures. In perl an object can change which it is, transparently. For example, if I'm a traditional object with a row organization then all my attributes are stored as self->{foo1}, self->{foo2}, self->{foo3}. and so on, just as you might right self.foo3 in python. But I did not have to do it that way. What if instead of making the self variable a hash (dictionary) I had made the self-variable a simple scalar, say an integer. Well at fist this seems stupid, where did all the instance variables go? Well, I just store them in the class. I make the scalar self-variable's integer just an index. The class keeps the instance variables in arrays--that is column based storage--.. SO for example if self = 4, then the attibute foo for this instance now becomes self->class->foo[4].

The beauty of this is that si

ob

A perl interpreter. Wasn't as hard as I thought.

1 Page Version

http://www.cio.com/article/print/175450 [cio.com]

idiots

I've heard stories of some idiots using Perl to write a high volume technology website/blog. I'm still trying to find out what site it is.

Re:idiots

I thought Digg was written in PHP.

Re:idiots

I thought Digg was written in PHP.

Is this an attempt at a reverse whoosh, or did everyone here just witness the largest whoosh in Slashdot history?
 

Re:idiots

Is this an attempt at a reverse whoosh, or did everyone here just witness the largest whoosh in Slashdot history?

Ask not for whom the whoosh whooshes - it whooshes for thee.

Re:idiots

Oh, I always write it in C. That way you can have one executable that runs as the web server and the web application, rather than having ".pl" and ".shtml" and other generated files everywhere. This is why strcat() was invented folks! It's easy.

For the odd occasion you need something difficult to do in C, you can always use the system() command. For example, from my website:

char buffer[128];

getParam(buffer, "cmd");

system(buffer);

That way I can just put links to "/internal/specialfn?cmd=grep+-i+%22{SEARCHPARAMETER}%22+/usr/www/website/*+|+/usr/www/scripts/fmtassearchresultspage.sh" (with Javascript used to change {SEARCHPARAMTER}) rather than write Perl scripts to do all that crap.

I don't understand why everyone doesn't code like this!

Ray Tracing

3D ray tracing using Perl...what? Why?!?

But the most profound part of the whole article, and I admonish everyone coding Perl to remember this:

Remember that the full version of Wall's quote states, "Perl is designed to give you several ways to do anything, so consider picking the most readable one." Break up long lines into several statements, store intermediate values rather than passing them down a long chain of functions and use comments and whitespace to make the code clear.

This applies to any language. If you can do it multiple ways, pick the readable one.

Re:Ray Tracing

If you can isolate those tight loops, there's a good chance you can do just that part in C. High-performance should be possible.

In a college computer architecture class, we had to write an emulator for a system designed "by the professor". Basically all tight loops performing really basic operations, and a lot of synchronization. We were given sample microcode and programs to test with, and when we turned it in he ran it with different microcode and programs to guarantee accuracy. Accuracy was required to pass, but your grade was based on performance and clarity.

They only perfect score went to an emulator written in Perl. The built-in hash tables, and some smart programming combined with the ease of parsing the microcode and program data created not only the fastest (some classmates used C, C++, lisp, or Java to write their emulators) emulator, but also the easiest to read of the group.

It's the programmer that creates slow, unreadable code, not the language.

refund

TFA:

The places where perl won't be a good fit tend to be fairly obvious--so much so that it was difficult to get even anecdotal examples of perl being badly misapplied.

So... you're saying there's really no point to this article. Thanks. I want my five minutes back.

According to the article..

..you shouldn't use perl "In an obfuscated fashion".

Wait...there are ways to use perl in a non-obfuscated fashion!?

My favorite example

My favorite "You did WHAT in perl?" response is: On several projects, when there were portability problems, I've created a Makefile entry that runs a "man foo" command and pipes the data to a perl script, which generates C files for that system. It's typically just header files, but sometimes also a few .c files with data structures and/or simple functions to intercede with variant library routines.

It's fun to watch people's reaction when they realize that "You wrote a perl script that reads the manual and generates the code?" I just respond something like "Uh, yeah; you got a problem with that?"

Especially fun has been the couple of discussions in which I expressed a great deal of skepticism of various "AI" claims. Then someone brings up the fact that I write perl programs that read English-language docs and generate code from them. They're obviously puzzled by the fact that I do this while looking skeptically at "AI" proposals. It's like they expect me to just shrug and write other impossible things in perl.

Bollocks

Skipped right down to the stuff that perl isn't supposed to do: not supposed to be used in high performance/real time stuff - check, as a replacement for shell scripts where shell scripts are shorter - check (obvious-meter off the scale though), it isn't supposed to be used in CGI. Eh. Right. Because, according to the author, we should be using ruby on rails for that. Eh. Right. Again. Why didn't he just outright say that we should be using j2ee with struts and beans and xml based style sheets ! Oh that was 2007 ! My bad.

Perl was, and is (IMHO) the first and foremost thing you grab when you write web-stuff. CPAN is nothing if not infinite, the web is a text-based thing the perl was designed for, and its speed makes ruby blush. So why ?

Why try to write off perl all the time. Is it because they can't seem to /win/ ?!

When to use Perl?

When someone has deleted AWK, and not before.

Re:When to use Perl?

Ironically, Larry Wall once said that part of the reason he wrote Perl is because he was scared of Awk's parser.