Follow Slashdot blog updates by subscribing to our blog RSS feed

 


Forgot your password?
Close
typodupeerror
×
The Internet IT

First Look At the ACID3 Browser Test 133

Posted by kdawson from the turn-on-tune-in-render dept.
ddanier writes "Now that all major browsers have mastered the ACID2 test (at least in some preview versions), work on ACID3 has begun. The new test will focus on ECMAScript, DOM Level 3, Media Queries, and data: URLs. 100 tests will be put into functions each returning either true or false depending on the result of the test. The current preview of ACID3 is still missing 16 tests."
This discussion has been archived. No new comments can be posted.

First Look At the ACID3 Browser Test More

First Look At the ACID3 Browser Test

Comments Filter:

  • From the summary: (Score:5, Funny)

    by u-bend ( 1095729 ) on Friday January 11, 2008 @09:46AM (#21998784) Homepage Journal

    Te new test...
    Shouldn't that be Teh new test...?
  • Finally, the bigger browsers are ACID2 compatible now. But suddenly those fuckers release a new ACID test. Now everybody's standard incompatible again. Let's see who succesfully implements ACID3 first.

  • Various Scores (Score:5, Informative)

    by The MAZZTer ( 911996 ) <megazzt.gmail@com> on Friday January 11, 2008 @09:58AM (#21998970) Homepage

    Final scores of course are subject to change on the final test:

    • Firefox 3 beta 2 @ Windows XP: 62%*
    • Internet Explorer 7 @ Windows XP: Dear God... you need to try it yourself. Viewing the generated source is needed to see the result is 24%
    • Opera 9.5 build 9721 @ Windows XP: 65%
    • lynx and elinks @ Windows XP: No JavaScript support. :(
    • Opera 9.3 @ Wii: 61%
    • Opera 8.5 @ Nintendo DS: 1%

    * - script takes long enough to run that browser prompts you to kill it.

    • Re:Various Scores (Score:5, Informative)

      by mzs ( 595629 ) on Friday January 11, 2008 @10:01AM (#21999030)
      Safari 3.0.3 on Mac OS X 10.5.1 does 50%. It does not have the little colored squares as in the reference though.
    • IE 6 supports 100% if the standards, or so it claims.

      And the IE6 version, you don't even have to compare it to the reference rendering.

    • Re:Various Scores (Score:5, Informative)

      by Laebshade ( 643478 ) <laebshade@gmail.com> on Friday January 11, 2008 @10:21AM (#21999326)
      lynx and elinks @ Windows XP: No JavaScript support. :(

      I don't know what versions you're using, but at least for elinks (and links), they both support javascript. Just has to be compiled in.

      eix elinks
      * www-client/elinks
                Available versions: 0.11.2 0.11.2-r1 0.11.3 {X bittorrent bzip2 debug finger ftp gopher gpm guile idn ipv6 javascript lua nls nntp perl ruby ssl unicode zlib}
                Homepage: http://elinks.or.cz/ [elinks.or.cz]
                Description: Advanced and well-established text-mode web browser

      eix ^links$
      [I] www-client/links
                Available versions: (2) 2.1_pre26 2.1_pre28-r1
                      {X directfb fbcon gpm javascript jpeg livecd png sdl ssl svga tiff unicode}
                Installed versions: 2.1_pre28-r1(2)(21:18:19 11/07/07)(javascript ssl tiff unicode -X -directfb -fbcon -gpm -jpeg -livecd -png -sdl -svga)
                Homepage: http://links.twibright.com/ [twibright.com]
                Description: links is a fast lightweight text and graphic web-browser

      So while they do support javascript, they don't support iframes, and the test uses 3 of those.

    • Re: (Score:1, Informative)

      by Anonymous Coward
      I ran the newest versions of IE, Opera, and FireFox, same scores as you but w/o any hanging... might want to fix your computer!
    • Firefox 2.0.0.11 on Windows XP: 59%
    • Generic Web Browser @ PS3: 34% ... no little reference boxes though, so YMMV.
    • Well, for IE6 it reports 100% of the standards as I can prove here [imageshack.us]! In your face Firefox!
    • Opera 9.12 on OLPC (One Laptop Per Child): 55%
    • # Firefox 3 beta 2 @ Windows XP: 62%*
      Same here, although it always says it should be 'smooth', and I'd say it's nowhere close that.

      # Internet Explorer 7 @ Windows XP
      My eyes! My eyes!

    • Re:Various Scores (Score:4, Informative)

      by Dak RIT ( 556128 ) on Friday January 11, 2008 @11:20AM (#22000188) Homepage

      Safari 3.0.4 on Windows using WebKit-r29380 (today's nightly build), Safari scores a 70/100.

    • > Firefox 3 beta 2 @ Windows XP: 62%*

      Firefox 3 nightly @ Linux: 63 %

    • Re: (Score:2)

      by WK2 ( 1072560 )
      Iceweasel 2.0.0.11-1 @ Debian Linux: 59%

      Firefox probably has similar results. It displays a red box with a weird bunny creature inside. I looked at the "reference redering" and laughed. It's supposed to be a series of boxes, each a different rainbow color.

      It should be noted that the ACID3 test is not ready, and should not really be used for comparison, and certainly should not be used by browser developers yet.
    • Camino nightlies: 58/100

  • Safari 3.0.4 (Windows) hangs at 60, Internet Explorer 7.0.5730.11 messes up so badly the result can't be read...

    The test looks interesting, for sure. And it's going to raise the game for standards compliance!

    • ... and looks somewhat like the reference image...

      Opera 9.2.4 (Windows) reaches 55 (but looks horrible)...

      Firefox 3 looks like the best shot at it so far.

    • The test looks interesting, for sure. And it's going to raise the game for standards compliance!

      Also, with the emphasis on ECMA script and animation, it'll raise standards and compliance for games.
    • Funny, in the version of Acid 3 I tested a week ago. Konqueror got to 85%. Konqueror 4.0 now stops on some weird embedding of text/plain assumption the test makes.

      More fun acid hacking for me :)
      • I just noticed this now, I tried to view it but it stopped and said 1/100 passed, though i was supre Konqueror couldn't be that wrong sicne KHTML was first engine to display Acid2 correctly (though not in Konqueror).

        I'm wondering whether the Konqueror devs will notice and fix this, since a drop of 84% in that amount of time is definitely messed up.

    • And it's going to raise the game for standards compliance!

      Look, I don't mean to be getting you down about this, but I'm fairly certain this is the only time such a phrase has been uttered with such enthusiasm. Ever. Congratulations of a sort are in order.
  • Swell, better than nothing, but tests miss the point. It was pointed out by Dykstra i think, that tests can reveal the presence of errors, but never their absence. So testing is in some sense a pointless pursuit. Being able to display a smiley face tests something like 1 googletillionth of the phase space.

    • Re: (Score:2, Insightful)

      by roggg ( 1184871 )

      It was pointed out by Dykstra i think, that tests can reveal the presence of errors, but never their absence. So testing is in some sense a pointless pursuit.
      I got your missing the point right here. It's not necessary to prove the absence of errors. Developers use the presence of errors (and knowledge of those errors) to direct efforts at improving products. In what sense is discovering errors a pointless pursuit?
      • >In what sense is discovering errors a pointless pursuit?

        Because it focuses on the tip of the iceberg, the symptoms, not the disease. I wonder how many of the browsers have been tweaked to pass certain tests, instead of being engineered to meet the specs.

        • Re: (Score:3, Insightful)

          by bunratty ( 545641 )
          That would be pointless. If browsers are getting mere hacks to display the specific acid test page correctly, the jig will be up when web developers start using the features tested by that acid test and discover that the features don't really work. I suspect that no browsers have been tweaked to pass certain tests, as that tweaking wouldn't fool web developers for any significant period of time.

        • Re: (Score:1)

          by roggg ( 1184871 )

          Because it focuses on the tip of the iceberg, the symptoms, not the disease. I wonder how many of the browsers have been tweaked to pass certain tests, instead of being engineered to meet the specs.

          Your medical analogy is apt. Black-box testing is pointless in exactly the same way that going to a doctor when you are sick is pointless. The symptoms are all you have initially to indicate there is any disease or what it's nature might be. Those with clear-box knowledge (doctors or developers) can use the symptoms as a starting point fur further inquiry to diagnose the underlying problem.

          I know what theory says about testing and correctness and all that. It doesn't work that way in the real world.

    • Yes, testing can never prove a program correct. On the other hand, do you think you'd get anywhere trying to prove that anything about any browser is correct using formal methods? Especially when the source code for most browsers is not even publicly available.

      The Acid tests are also not really about finding obscure bugs, but about demonstrating which basic features work and which ones do not work. After all major browsers pass an Acid test, web developers can attempt to use the features tested by the Acid

  • Where is the reference image from? (Score:5, Interesting)

    by Aaron Isotton ( 958761 ) on Friday January 11, 2008 @10:08AM (#21999116)
    Something I always wanted to know (applies to the older Acid tests, too): how do they render the reference image? Is someone creating them by hand? How do we know no mistake was made when creating the reference image?

    • Re:Where is the reference image from? (Score:5, Funny)

      by patio11 ( 857072 ) on Friday January 11, 2008 @10:19AM (#21999300)
      It's created by an advanced, custom-built browser which, for certain input, correctly renders a perfectly standards-compliant reference image. Just don't ask to use the browser on any other input.

      • Re: (Score:3)

        by dpilot ( 134227 )
        I'm glad I don't have mod points, at the moment.

        I don't know if you're trying to be Funny(sarcastic) or Informative.

        • Re: (Score:2)

          by Arimus ( 198136 )
          He has a point - most browsers around are designed to cope with the non-standard/flawed/broken crap that passes for some websites these days. Use a standards compliant browser which does not make a best guess at working out what was really meant and rejects any errors you'll have problems with alot of sites.

          • Re: (Score:2)

            by Dan Ost ( 415913 )
            Is there a plugin for Firefox that will tell me when a page is being rendered using these non-standard coping methods?
            • I don't know if there is a plugin, but you can just right-click the page in Fx and select "View Page Info" and then it will either say "Render Mode: Standards compliance mode" or "Render Mode: Quirks mode".

              Oddly enough, I had trouble finding a site that renders in quirks mode so I could get the exact message that the dialog displays. I tried about a half dozen sites in my bookmarks and they were all rendered in standards compliant mode. So then I tried microsoft.com [microsoft.com] and bang, I had a site that rendere
      • Can I use the browser on other input?

    • Re:Where is the reference image from? (Score:5, Interesting)

      by thue ( 121682 ) on Friday January 11, 2008 @10:30AM (#21999442) Homepage
      How do we know no mistake was made when creating the reference image?

      You don't

      I remember an article by the Apple guy who made ACID2 work on Safari (I think this was the first browser to make it work). One of the steps to get it working was to fix a bug in the test, when he couldn't make the reference result fit with what the test HTML said.

      • Re:Where is the reference image from? (Score:5, Informative)

        by gsnedders ( 928327 ) on Friday January 11, 2008 @10:42AM (#21999654) Homepage
        Yeah, it was David Hyatt who was working on getting Saf to pass (and got it to be the first browser to pass in any build, and the first to have a generally available release (i.e., a non-development build, even if public) -- the latter being the only thing that truly counts for passing the test).

        http://weblogs.mozillazine.org/hyatt/archives/2005_04.html#008011 [mozillazine.org] details the bug (in this case, it was the test itself that was wrong -- not the reference). The reference rendering for Acid3 is likely correct as the actual rendering isn't overly complex (the complexity is in the ECMAScript and DOM support), though with the complexity of some tests there could easily be bugs in the test again.
        • There ARE bugs in the test. Hopefully, when they are finished with it, there won't be.
          • In my view a test isn't a test until it is finished and can be used as a test, so it isn't yet a test (which means any current bugs are irrelevant).

            • In my view a test isn't a test until it is finished and can be used as a test, so it isn't yet a test

              Your view dramatically reduces the value of a test. It's very normal in any test process that every "failure" has to be evaluated to determine if the error is in the implementation, the test or even the specification. Building and running tests is how you find the issues that need to be resolved. Saying they're not tests until both the test and the specification have been proved correct is just a word game that makes it necessary to come up with some other term for the "tests" that aren't quite yet "te

      • I know this may sound little "preachy"(?) but wouldn't that be one of the biggest benefits to open standards? People reviewing the standards, and fixing possible bugs or "workarounds" the original standard missed.

        I just wish we could get that into the heads of the big software shops... you know who you are. It's unfortunate that money turns against a good idea.

      • Re: (Score:1)

        by Val314 ( 219766 )
        Well.. Safari may pass the original ACID2, but it fails the ACID2 (no Data) test from http://hixie.ch/tests/evil/acid/002-no-data/#top [hixie.ch]
        In fact, it always downloads a file "data006" when openig this page.

        (tested with Safari 3.0.4 (5523.10.6) on Leopard)

        Firefox 3 Beta passes this test.

    • Re: (Score:2)

      by Tribbin ( 565963 )
      That is a question of Faith

      You're not supposed to question Faith

    • Re: (Score:2)

      by hixie ( 116369 )
      I'm not sure how Todd did it for Acid1 -- I think he may have worked it out by hand and drawn it in photoshop.

      For Acid2, I made a second version of the test that worked around all the bugs in Firefox, and then took a screenshot of Firefox.

      For Acid3, I actually made the background of the reference rendering first as a simple HTML file, took a screenshot of that, made that the background of the reference.html file, and then added some text to the reference file and used absolute positioning to get the text wh

  • So.... (Score:2, Flamebait)

    by JediTrainer ( 314273 )
    Are you telling me Firefox isn't a major browser? I just tried Acid2 on my FF 2.0.0.1.1 on Windows and it still looks like crap. How far behind is it?

    • Re:So.... (Score:5, Informative)

      by Arimus ( 198136 ) on Friday January 11, 2008 @10:47AM (#21999734)
      Firefox is a major browser, however the version which passes ACID2 is Firefox 3, I think the first build which passed was around this time last year so either go with the development release (FF3 is currently in Beta).

  • ECMAScript, DOM Level 3 (Score:3, Informative)

    by QuietLagoon ( 813062 ) on Friday January 11, 2008 @10:55AM (#21999836)
    Excellent. These two, especially, need to be tightened up (and in some cases, fixed) across the browsers.
  • Konqueror 3.5.8 on KDE 3.5.8 (Debian Sid, AMD64; packages from Debian repository) fails with a segmentation fault.

    But there's a newer version in the repository, so I'm going to upgrade and see what happens.

    • Re: (Score:2)

      by ajs318 ( 655362 )
      Interesting. The "new" version of Konqueror also claims to be 3.5.8.

      This time, it asks me what to do with a file it is trying to download, then crashes (with a different-looking requester; maybe that also changed between versions).

      • Re: (Score:2)

        by ajs318 ( 655362 )
        Tits [slashdot.org]. Missed something important.

        KDE4 isn't in Sid yet (it's a credit to Debian that "unstable" is actually so stable, that they have to have an "experimental" distribution). I'd grab a Kubuntu liveCD and try that, but it wouldn't necessarily prove anything: crashing in Kubuntu is hardly news.
  • Safari displays the reference rendering jes' fine!

    Oh, wait...

  • The Acid Test is all about seeing if browsers can properly render intentionally mis-written, broken code, including things that I find it hard to believe that anybody would do on propose on a real-world page. The important question isn't whether or not browsers can render the Acid Test correctly, but whether or not they can render 99.44% of all pages out there correctly. Personally, I don't care if my browser can render such malformed pages correctly or not, or if any other browser can. If somebody's rea

    • Re: (Score:2)

      by hixie ( 116369 )
      According to my studies, about 93% of all pages out there are syntactically invalid in some way. So to render 99.44% of all pages out there correctly, a browser has to be able to handle syntactically invalid pages. That's why it's important to test handling of correct markup as well as incorrect markup.

      (Based on a study I did at Google using several billion pages.)

      • Re: (Score:2)

        by WK2 ( 1072560 )
        According to my (informal) studies, 90% of all web pages that display the "(X)HTML X.X" compliant logo are non-conformant in some way.
      • The point is that most pages are either correct, or have only minor errors, such as not properly closing a paragraph tag before opening a new one. The Acid Tests, from what I gather, do everything their authors can think of to break the standard. I think it's all well and good that browsers aren't too strict in following the standards and have a little wiggle room included for accidental errors, but to me, the Acid Test goes far beyond that and encourages sloppy coding. In the long run, the Acid Test is

        • Re: (Score:2)

          by hixie ( 116369 )
          Oh no, the errors on Web pages are all kinds of things. One of the most common errors was bogus content inside tables, for example (27% of pages had this error). (Like, <table><font><tr><td>...</table>)

          The authors of the Acid2 test (primarily me) didn't actually include any HTML4 parsing error handling tests. There were some CSS ones, but a far cry from all the ones I could think of. (Acid3 has even fewer.) The CSS and HTML5 standards define how you handle errors, by the way --

    • Re: (Score:2)

      by Kelson ( 129150 ) *

      The Acid Test is all about seeing if browsers can properly render intentionally mis-written, broken code, including things that I find it hard to believe that anybody would do on propose on a real-world page.

      This misconception seems to come up every time Acid2 gets mentioned on Slashdot.

      Broken code was part of the Acid2 test, but far from the primary focus. Read the test guide [webstandards.org] sometime. The first Acid test [cleverchimp.com] didn't use broken code at all; it simply tested implementation of the box model.

  • Since when is vaporware acceptable as proof? (Score:3, Interesting)

    by jbn-o ( 555068 ) <mail@digitalcitizen.info> on Friday January 11, 2008 @08:13PM (#22008838) Homepage

    Now that all major browsers have mastered the ACID2 test (at least in some preview versions) [...]

    When Firefox makes news on this there are daily builds to test, source code to inspect and compile. One can see the progress first-hand.

    There is no build of Microsoft Internet Explorer 8 to test. You are accepting something unverifiable as reality and thus talking about these browsers as if they're all on the same level. This suggests a new low: believing the illegal monopolist who tells you that their vaporware behaves in accordance with published publicly-implementable standards.

Slashdot Top Deals

Never test for an error condition you don't know how to handle. -- Steinbach

Close