Python 3.0 To Be Backwards Incompatible

Stony Stevenson writes "Organizations using Python will be affected in a major way by changes in store for the language over the course of the next twelve months, Linux.conf.au attendees were told this morning. The Python development community is working towards a new, backwards-incompatible version of the language, version 3.0, which is slated for release in early 2009. Anthony Baxter, the release manager for Python and a senior software engineer at Google Australia, said "We are going to break pretty much all the code. Pretty much every program will need changes." Baxter also added another tidbit for attendees, saying that Python accounts for around 15 percent of Google's code base."

It's a race

[Intron (870560)]

Will the new Perl or the new Python be the first to shoot itself in the foot with incompatibility?

Breaking backwards comp. in languages good

[Crazy Taco (1083423)]

Will the new Perl or the new Python be the first to shoot itself in the foot with incompatibility?

You know, sometimes a little backward incompatibility can be good. You can shoot yourself in the foot more by not breaking things when something needs fixing. As an example I point to VB.Net.

Back in the days of VB 6 and prior, VB was an entry sort of language that was designed with novice programmers in mind, and strayed quite a bit from normal programming constructs. With the advent of the .Net platform looming, Microsoft wanted to move VB to .Net, but didn't want to break existing code out in businesses. So they opted instead for kludges and workarounds in the language, and that's the reason I can't wait to get done with VB .Net project I am currently writing. If you are an experienced programmer in other languages such as Java, C++, etc, VB .Net can drive you mad. Here are just a couple examples:

1. Arrays. VB did not originally have 0 based arrays. Its arrays started at 1. In order to get VB to work with .Net, Microsoft really turned arrays into a kludge that is very difficult to remember (I almost always have to pull out an O'Reilly reference) and easy to get wrong. In a normal language, you declare an array of five integers and you get five slots numbered 0 through 4, right? Well not VB .NET! You declare an array of five integers, and you get SIX slots, numbered 0 through 5. And where most languages you would see "something - 1" when referencing array locations, in VB you often see "something - 2"! That really makes things annoying, especially if you've worked in .NET before with C#, because when you come over to VB you are almost guaranteed to biff things up.
2. Logical operators. In the original VB, "And" and "Or" operators did NOT do short circuit evaluation. To avoid breaking code, Microsoft introduced even more operators, "AndAlso" and "OrElse". Those two operators are just like "And" and "Or" except that they do do short circuit evaluation, but good luck trying to remember to use them when coding quickly. I know when I code and want to "And" things together, I think "X And Y", not "X AndAlso Y". Therefore, these new kludge operators certainly aren't the default operators most people think to use, so most end up using "And" and "Or" in the heat of coding, and VB code suffers and unecessary performance hit all over because short circuit evaluation isn't often used.

There are many other examples in VB like this, so I say this to the Python developers: If you have a good reason to break backwards compatibility with the language, then do it. Keep backwards compatibility whenever reasonably possible, but break it before accepting a kludge into your language. Your future coders will thank you, and will not run away screaming to the other, newer languages that will inevitably follow.

Re:Breaking backwards comp. in languages good

[Doctor Faustus (127273)]

Its arrays started at 1. In order to get VB to work with .Net
This was actually introduced prior to .Net. I'm not sure when, exactly, but VB6 is the same way. Actually, VB6 is pretty schizophrenic about 0 vs. 1 based indexes in general. I used to make a habit of just putting in a breakpoint and looking at the array while the program ran, instead of trying to remember which was which.

Personally, I like the way Turbo Pascal made you declare both upper and lower bounds. 0-based arrays make sense in a systems programming language when it's just a thin veneer for a pointer, but VB, C# and Java shouldn't force it on you.

In the original VB, "And" and "Or" operators did NOT do short circuit evaluation.
Beyond that, they were bitwise operators, not logical. The fact that 1 is true, 2 is true, but 1 AND 2 is false is far more perverse than the lack of short-circuiting. This is also why the canonical true value in VB is -1 (1111111111111111 binary) rather than the usual 1. This was appropriate in Bill G's 8K Altair BASIC, but it should have been fixed in MS BASIC well before 1980.

VB.Net Beta 1 made "And" and "Or" normal logical operators and introduced new bitwise operators for the old functionality, but too many people complained and they switched it back for Beta 2.

There are many other examples in VB like this
I don't know about that. Those are pretty much the worst parts that .Net didn't fix. There are some clunky IO libraries clearly designed to be used by the migration wizard, but they're strictly optional. Plus, I really like that VB.Net still has real optional arguments, as opposed to C# that goes the Java route and makes you declare a little stub version with fewer arguments that just calls the real one and fills in a default value.

Re:

[by Anonymous Coward]

Is Perl 6 backward-compatible with Perl 5?
No. [programmersheaven.com]

Re:It's a race

[IkeTo (27776)]

From Perl 6 FAQ (http://dev.perl.org/perl6/faq.html)

> Will I be able to convert my Perl 5 programs to Perl 6?

> Yes. Larry Wall and others are already working on a Perl 5 to Perl 6 translator, which will be able to translate (most) Perl 5 source code to the equivalent Perl 6 syntax.

> In addition, Perl 6 will provide a "Perl 5 compatibility mode", allowing the compiler to directly execute any code that it recognizes as being written in Perl 5.

Depending on who you talk to, this can be said to be backward compatible.

Re:It's a race

[Incongruity (70416)]

See, the funny thing is that the changes going into Python 3 are fixes for much of what people have complained about in Python 2.x and prior.

Moreover, every step of the way they've built translators to move code from 2.x compatibility to 3.0 compatible -- and it'll catch when it can't translate the code and tell you as much. It seems pretty slick from everything I've seen. In many cases the fixes are ones you could easily do yourself in seconds with a good text editor. This will be a minor speed-bump for most users

For more info, check out the recent Doctor Dobb's Journal interview (audio) with David Goodger [ddj.com] -- it's about PyCon 2008, but it also covers Python 3 as well.

Re:It's not like this is the first time.

[slackergod (37906)]

Indeed. You should read up on the changes they're planning to make.
Many of them are backwards-incompatible changes that will remove nasty warts,
realign certain parts of python's object structure (str vs unicode, int vs long)
in ways that are MUCH cleaner, MUCH harder to make mistakes with, etc.

This is not some Larry Wall day dream revision...
they aren't rethinking python from the ground up,
it's an evolutionary revision which is keeping the python 2.x codebase,
but changing some things they've been wanting to do for a long long time.
The 1.x -> 2.x transition is apt... the code will look almost the same,
only cleaner :)

If that's not good enough for anyone, automated tools are being developed
to convert python 2.6 -> 3.0. Yes, that's right, Python 2.6. It'll
be the last version of 2.x released, and be carefully designed to be
both compatible w/ 2.x, and still able to warn you about gotcha's
that would appear if you ran the code under 3.0.

Not only that, but 3.0 isn't projected to be in mainstream use
for at least 2 more years.

There's absolutely no need for the sensationalist panic this slashdot headline
annouced itself with. Rethinking a complex system from the ground up is always
a bad idea... but GVR knows that, and that's not whats happening here.

It's more akin to the linux kernel moving from 2.4 -> 2.6
Incompatible, yes, but a much better architecture.

Re:Python to not be backwards compatible

[Kadin2048 (468275)]

If it isn't backwards compatible, it isn't Python, as far as I'm concerned. I'm sure not going to be re-writing a couple hundred megabytes of code. They can take their incompatible new snakelike thing and smoke it. Foregoing backwards compatibility is about as dim a move as I've ever seen anyone outside of Microsoft pull (MS dropping VB underneath Access comes to mind as a comparable clueless act, with exactly the same consequences — it isn't the same product and is useless to me.)

I don't think anyone is really suggesting that a lot of code be rewritten. Python 2.x isn't going anywhere, for exactly the reasons you bring up. Python 3.0 is going to be almost a completely new product, and I expect it'll only be used for new projects. I'm not really sure why anyone would want to convert their old code, if it works for you the way it is.

The reason for breaking compatibility is to maintain the Python philosophy of not having a lot of ways to accomplish the same thing. Without periodic pruning, eventually you'd just have Perl with indents.

From what I've read, you'll be able to choose the version/style you want, and the 2.x and 3.x series will live alongside each other (I don't know whether the 3.x interpreter will have a compatibility mode, or if you'll have to keep the 2.x version around as a separate program, though).

Re:Python to not be backwards compatible

[timmarhy (659436)]

the problem is at some point they are going to drop support for it, and no modules will begetting written for it either. the fact that you can maintain it yourself is a retarded response to this, because people who say it have pretty much no idea how much extra work that puts back on you, and frankly no one wants that.

Re:It's a race

[BOFHelsinki (709551)]

15% of the google codebase consitutes a "small user base"? Why, yes. Google is a big user but still just one user. So if 15% of Google uses it, it's only 0.15 users. That's a very small user base. Even Mono has a larger user base (Miguel).

Re:It's a race

[Intron (870560)]

You could just ask python:

>>>"python" > "perl"
True

Whereas if you ask perl:
perl -e 'print "python" > "perl"'

It remains silent, see?

Just rename it.

[Besna (1175279)]

Call it "Cobra" or something. Too many kludges will confuse people. A new name and file extension will emphasize that this is "in with the new".

Re:Just rename it.

[Neil Hodges (960909)]

There are already other [wikipedia.org] languages [wikipedia.org] doing this. If you think naming closely-related languages the same thing is a kludge, what do you think of naming mostly-unrelated languages the same thing?

Re:Just rename it.

[Tritoch (989763)]

You could call it "Asp". Oh, wait...

Re:Just rename it.

[Bogtha (906264)]

The vast majority of the language and standard library will remain the same. This is just about tidying up some unfortunate warts that affect a lot of people, such as unifying the different string types. It remains Python in practically every way, and renaming it is simply unnecessary.

Re:Just rename it.

[Speare (84249)]

Python's named after the troupe Monty Python, not after the snake species. I don't think renaming it is a good idea, but suitable successors would be [Life of] Brian, [Fish Called] Wanda, Flying Circus, Holy Grail or perhaps start with sub-versions like Cleese, Chapman and Palin.

Alternatively, pick another troupe or favorite comedy show: Fry and Laurie, Mr. Bean, Fawlty Towers or Red Dwarf. Or my favorite, which brings back in the snake species AND British comedy into circular pun, Blackadder.

While talking about puns, snakes and coming full circle, I suggest Ouroboros.

Re:

[by Anonymous Coward]

> Otherwise, we'll be left with Python 1, Python 2, Cobra 1, Cobra 1.1, Cobra 1.3, Rattlesnake A, Rattlesnake SP1, Diamondback etc etc

"THAT'S IT! I have had it with these muthafuckin' snakes on the covers of these muthafuckin' O'Reilly books!"
- Guido L. Jackson.

Re:Just rename it.

[orclevegam (940336)]

Because whether or not it's a lie is largely a technical question that's somewhat open to interpretation. If by backwards compatible you mean is the old Perl5 syntax valid Perl6 syntax, then the answer is no. If however you mean will my old Perl5 scripts run in the Perl6 interpreter, then the answer is yes. Further more the old Perl5 modules (as per your link) will still be usable from Perl6, and there will be a converter to update the Perl5 scripts to Perl6 syntax. So, a better answer to the question of "Is Perl6 backwards compatible with Perl5?" might be "maybe".

Re:Damn right

[pthisis (27352)]

1. Python 2.6 is still due out and will be supported for years.
2. Python 3.0 includes a 2to3 script to convert existing Python code to Python 3
3. The incompatibilities are mostly mechanical, by far the largest is the change of "print" from a statement to a function (which simplifies the implementation and makes converting existing programs to log to files or whatever much easier). I've yet to find any of my code that 2to3 doesn't handle just fine.

Another Shock Story

[LowSNR (1138511)]

If the editors read the article rather than posting shock stories, Python 2.6 will also be released at the same time and will not break backwards compatibility. Python is not pulling the rug out from under its developers as the summary would lead you to believe.

Re:Another Shock Story

[betterunixthanunix (980855)]

No, but that means that everyone planning to run non-Python3 code will have to maintain two parallel Python installations. With package management that's not so bad, but it still puts a bit of pain on distro maintainers.

Yet everyone makes fun of me when I say that I am a C++ programmer.

Re:

[Schraegstrichpunkt (931443)]

No, but that means that everyone planning to run non-Python3 code will have to maintain two parallel Python installations. With package management that's not so bad, but it still puts a bit of pain on distro maintainers.

This is already done in the distros I've seen.

Re:Another Shock Story

[onion2k (203094)]

It is an issue though. PHP did exactly the same between version 4 and 5, and it crippled adoption of 5 because hosts refused to upgrade as it'd have broken too much code. It was a full 3 years or so before 5 was really considered the primary version amongst many developers and that took the announcement of 4.x support ending and the success of the GoPHP5 campaign.

Hopefully the Python team will learn from PHP's experience.

How did that Cripple PHP

[alexhmit01 (104757)]

PHP 5 was a MAJOR over-haul, biggest since PHP 4. When PHP 4 came out, hosts and OSes supported 3/4. PHP 3 has really cool, but extremely insecure, syntax things for web prototyping. When you posted a variable, put it on the query string, or passed it in a cookie, it just came into the script as ${name}. So if you needed to check for setting, it was isset($variable), using it was just using it, etc. This was dangerous because in PHP you don't declare your variables, so if you assumed that a variable you never zero'd/nulled before usage was null at the beginning, this could be exploiting by posting a value.

PHP 4 had a php.ini setting to use the old mode (I think register globals or something similar) that helped in the migration. Also, include_once/require_once massive simplified using libraries. Before that we had to do our own equivalent of ifdef/ifndef in our PHP code to avoid overwriting a function if two included libraries called it.

The PHP 3 -> PHP 4 transition took a while as well. What's the rush, my new systems all include PHP4 AND PHP5. All my new code is PHP 5. My PHP4 stuff happens to be in legacy mode, but if I needed to bring it will us, you put it on a machine and see what works, what doesn't, and add support for 4 AND 5, like we all used to do on PHP 3/4.

Better to introduce some source incompatibility for improvements then not being able to move forward, just make sure that there is a transition strategy. Three years isn't really that long... though when I started in web stuff, 3 years was an eternity, but 8 years later, eh, just a few more years.

Re:Another Shock Story

[pthisis (27352)]

Python2.5 and later let you write 3.0-compatible code. Just put parens around your print statements and you're probably fine to run in 2.5, 2.6, or 3.0. There are a few other changes which 2to3 and the 2.6 interpreter will warn about, but they're relatively rare. e.g. range will return an iterator instead of a list; code like:

for i in range(1000):
        # do something

will remain unaffected, but code that does something like:

a=range(1000)
a[37] = 2000
for i in a:
      # do something

will be affected.

The conversion tool and 2.6 interpreter will warn you about those things, and they're easy to fix ( "a=list(range(1000))" ) if 2to3 doesn't do them automatically. Which I'm not sure about, because they're highly unusual--2to3 has converted all of my personal code just fine.

Workaround...

[fahrbot-bot (874524)]

I think I have a Perl script that'll fix this...

Just kidding Python fanbois :-)
Chill, it's Friday.

Re:Workaround...

[tuffy (10202)]

Actually, Python2.6 is slated to include a tool which will update purely syntactic differences to Python 3 automatically. There are some issues it won't be able to fix, but Python2.6 will have a mode which will generate warnings about those so that they can be fixed well before Python 3's release.

print as function vs. keyword

[spookymonster (238226)]

The biggest incompatibility is how you output to stdout. Instead of doing

    print "This used to work"

You now have to do

    print("This is how 3.0 rolls")

There will be no grandfathering, so everything needs to be refactored accordingly.

A small, but significant change.

Re:print as function vs. keyword

[Bogtha (906264)]

Actually, the migration tool 2to3 performs this change automatically, and the function call approach works in both 2.5 and 3.0, so the incompatibility is greatly exaggerated.

Re:

[Wiseman1024 (993899)]

A good one though. Statements are stupid. Probably Python's biggest flaw. It's good that they're getting rid of a couple of them, though while, if and the like still remain.

If you're wondering why statements are stupid: they are not first class values you can pass, use and manipulate; they introduce a special, harder to learn and very quirky syntax; they cannot be used anywhere and thus subtract flexibility; and they are an extra form or feature that's actually unnecessary ("Programming languages should be

I don't see the problem

[Urger (817972)]

While it would be nice if it were otherwise, sometimes you need to break with the past to develop solutions to problems. It's an ugly, but very real truth. Thats not to say that my I will be rewriting my code to 3.0 immediately, but sometime in the next year or two I probably will.

We have options.

[Sludge (1234)]

I maintain tens of thousands of lines of Python... and I'm not worried. Why? Because I am sure they will continue to support security and bugs on the 2.x line for a long time to come.

It is not like your favourite Linux distro is just going to drop the 2.x series overnight, or like Python 3 will fight 2.x on your system.

Whiners

[MikeDataLink (536925)]

We whine when companies break compatability, yet we whine just as loud about bloated software when companies leave in compatibility.

Tell, me exactly what would satisfy you? How about we just take your computer away.

I'm running for president! FREE PACIFIERS to all Slashdotters. ;-)

Re:Whiners

[Sciros (986030)]

What color?

Re:Whiners

[19thNervousBreakdown (768619)]

It's almost like there's more than one person with an opinion on this!

meh

[seanyboy (587819)]

As long as you can run Python 2 & 3 in the same environment, this shouldn't be a big deal.
It'll just be a case of slowly moving code from one version to the next.

This is a brave move, but you've only got to see the mess you can get into trying to force backward compatibility for too long (Vista, anyone) to know it's the right move.

Of course, this being python, I fully expect some brainbox to come up with an automated conversion routine (v2 to v3) that "WAS WRITTEN IN ONLY 15 LINES OF CODE". etc, etc.

in fact, such a utility already exists

[Reality Master 201 (578873)]

http://svn.python.org/view/sandbox/trunk/2to3/ [python.org]

And, as others have stated, there'll be the 2.6 branch, which will be backwards compatible.

Or, in other words, the story is stupid and misleading.

Python is doomed

[Arthur B. (806360)]

Look at C++, they broke backwards compatibility with C ( malloc casting for example ) and because of that it never became mainstream.

Almost 2 years old?

[by Anonymous Coward]

People only reading the summary and shooting from the hip should realize:

• Python 3 (or Python 3000 as it was called) as a serious effort is more than a year old.
• There is already a working interpreter in its second alpha release.
• Final release is slated for August. (No infinite Perl 6 development.)
• Developers are working very hard to make the 2 to 3 transition as painless as possible.
• The Python team is committed to keeping the 2.x series going until 3.x has clearly been accepted.

You may now proceed to complain having been thus informed. :)

For more info, check out

[spookymonster (238226)]

The "What's New for 3.0?" article over on python.org:

http://docs.python.org/dev/3.0/whatsnew/3.0.html [python.org]

Not news in any way

[Bogtha (906264)]

This has been known for donkey's years. Guido has been talking about this compatibility break since the 90s. The changes were laid out in detail in PEP 3000 [python.org], first published in 2006. They have already released two alphas. A conversion tool to automatically make some of the required changes (such as changing print statements to print() function calls) already exists [python.org].

philosophy

[bcrowell (177657)]

IIRC, even dot-releases of python are not source-compatible. I assume that's why my install of Ubuntu has multiple versions of every library, e.g., /usr/lib/python2.4/smtplib.py and /usr/lib/python2.5/smtplib.py.

I think this is partly a matter of philosophy. The people in charge of a particular language tend to be computer language enthusiasts, and they like to tinker with them. They say things like, "The language has to be able to continue to evolve, or else it will die," or "We don't want to lock in things that, in retrospect, were really mistakes." But people actually using the language typically put a higher priority on not having their code break. Obviously we wouldn't all want to still be writing old-style fortran, with fixed columns, Hollerith codes, etc., but I also don't agree with the philosophy that "bit rot" is inevitable, and every piece of code you write is like a baby that you have to tend for the rest of your life. Personally, one of the things I really like about Perl is that it's got an excellent, mature implementation, and it's been mature for a long time. This is a lot less true for Ruby, for example, in my experience. It's true that Perl 6 is going to break backward compatibility with Perl 5, but the Perl 6 interpreter is going to automatically recognize Perl 5 code, and handle it correctly.

If this is news to you

[pembo13 (770295)]

&& you use python, please turn in your developer card

Can we put in requests?

[the-matt-mobile (621817)]

If we're breaking compatibility, can we put in requests? The 3 main things I'd like to see are:

1.) Ditch the silly ":" at the end of the line when starting a block. Why on earth is this needed? I almost always forget to use that darn colon.

2.) Ditch the bolted OO on "self" variable for object methods. Just make "self" a keyword and use it the way "this" works in Java/C#/C++.

3.) Allow for a whitespace agnostic mode with the end keyword the way Boo did for goodness sake! Why on earth is that so hard? If you want significant whitespace, it can still be the default. If significant whitespace drives you mad, then use a -wsa switch and a new "end" keyword and be done with it. It would end the flamewars and make Python a more acceptable language to those of us who prefer to have the option of aligning our code the way that makes the most sense, not the way the language Guido thought it should always be.

Re:

[pembo13 (770295)]

if it does, then it will be pointless. I like the whitespace thank you very much.

It does have block statements.

[Virak (897071)]

They're just delimited with indentation instead of braces. And it's not the language that is stupid, it's you, because you are apparently not indenting your code at all, or you'd realize that indented code in other languages "Fs up" (you do realize you can say "fuck" on Slashdot, right? Watch: Fuck, fuck, fuck, fuck. It's pretty cool.) stuff just as much as indented Python code does. Any decent editor can automatically indent code for you in any language, and in this case Python is even easier because all i

Re:So, will it FINALLY have block structures?

[msuarezalvarez (667058)]

Indentation is a derived piece of structure, but should never have semantic meaning in a language except as a separator.

Why, exactly, is it that it should never have semantic meaning? Is there a rule somewhere?

The Microsoft route

[mariox19 (632969)]

Exactly! Now they can force people to buy expensive IDE's and servers to run the new version of Python, and strong-arm people into purchasing licenses to use the language for commercial purposes. It's just a money making ploy.

Oh, wait a minute. All the old versions are going to continue to be available -- even the source code. And it will remain free for commercial use. Hmmmm.

Sorry, but I fail to see how what they are doing is at all like Microsoft.

Re:

[LaughingCoder (914424)]

You can slam Microsoft for a lot of things, but breaking backwards compatibility (B.C.) would be far, far down on that list. They bend over backwards (no pun intended) to maintain B.C. In fact, that compatibility oftentimes is the source of their security woes. And it is most certainly one of the major causes of code bloat, and buggy code. Personally, I think Microsoft should consider *reducing* their emphasis on B.C. in order to improve those other attributes I mentioned.

Re:#1 way to prevent adoption of new language vers

[Llywelyn (531070)]

Relax.

First of all, the changes are mostly simplifications to the core language (e.g., how to catch exceptions is currently a bit of a mess if you want to catch more than one exception). So for example, range and xrange are now one, iterators become more prevalent, "old-style" classes are going away and so new-style ones will become the standard, a lot of the things that have been deprecated now are being removed, etc. It isn't really a "new language" in any sense. This is far superior to Java's "always backwards compatible" approach which has lead to a lot of cruft building up over the years.

Next, it needs to be understood that 2.6 will be backwards compatible and include a warnings mode to highlight things that won't work in Python 3.0 to ease in the transition. There should be no problem supporting both on one system.

Finally, they are providing a 2.5->3.0 translation tool that runs in 2.5 and does most of the mechanical translation between the two for you.