Forgot your password?
typodupeerror
Bug Education Software IT

"ExamSoft" Bar Exam Software Fails Law Grads 100

Posted by timothy
from the until-it-happens-to-you dept.
New submitter BobandMax writes ExamSoft, the management platform software that handles digital bar exam submissions for multiple states, experienced a severe technical meltdown on Tuesday, leaving many graduates temporarily unable to complete the exams needed to practice law. The snafu also left bar associations from nearly 20 states with no choice but to extend their submission deadlines. It's not the first time, either: a classmate of mine had to re-do a state bar exam after an ExamSoft glitch on the first go-'round. Besides handling the uploading of completed exam questions, ExamSoft locks down the computer on which it runs, so Wikipedia is not an option.
This discussion has been archived. No new comments can be posted.

"ExamSoft" Bar Exam Software Fails Law Grads

Comments Filter:
  • Really? (Score:5, Funny)

    by nospam007 (722110) * on Thursday July 31, 2014 @05:55AM (#47572937)

    "leaving many graduates temporarily unable to complete the exams needed to practice law."

    And that's a bad thing, because ... ?

    • by Anonymous Coward

      Because they would like to practice law and earn money, presumably.

    • Re:Really? (Score:5, Informative)

      by fiziko (97143) on Thursday July 31, 2014 @08:01AM (#47573325) Homepage

      The business next door proctors these and similar exams. They are expensive and not available in every community, so the test takers have often paid a relatively large amount of money at this stage of their lives, not just several hundred to take the test, but also travel, accommodations, missing day(s) of work, etc. to be where the test is available. The proctoring company does not charge them for the second attempt, but all of the expenses needed to be there get doubled.

      • by nealric (3647765)
        Depending on your state, the all in cost of taking the bar exam can be $4,000 when you include test prep materials, fees, and travel. The fee just for taking the exam can be almost $1,000 and is not refunded if you have to retake.
        • I'm almost certain that a company which just screwed over a bunch of protolawyers will allow free re-testing for those involved. It would probably turn very, very ugly for them if they didn't. Test takers will have to pay for travel again, which is probably significant for many of them, but they won't have to pay for test prep and fees again.
    • by nealric (3647765)
      Whatever you feel about the necessity of lawyers in society, many of these graduates are out $150,000 of tuition and are $200,000+ in student loan debt. They are prohibited by law from working in their profession until they pass the bar, which is only offered twice a year. So yeah, it's a pretty huge deal to be sentenced to 6 months of unemployment when you are in deep debt because of a software glitch.
      • I don't know about that. Say the average first year lawyer makes $60,000 (pulled directly from my butt; I have no idea what the actual number is and don't care to look). Suppose that 80% of bar takers pass the exam. That means the expected income for the next six months of a random person taking the bar is 60K * .8 * .5 = 24K. This is the number that a good lawyer could convince a judge (who is a lawyer) that these young, brilliant, aspiring lawyers should be compensated by the testing firm (who is not a la

        • by nealric (3647765)

          I'm going to go ahead and say you have no idea what you are talking about.

          First: Law graduate salaries are heavily bi-modal. While the average salary is around $60,000, that average is heavily skewed by high-earners. My starting salary upon law school graduation was $160,000. Most large law firms (500+ lawyers), which employ approximately 10% of new graduates, pay exactly that salary to first-years. My offer was explicitly contingent on passing the bar. If I had failed, there would have been approximatel

      • So yeah, it's a pretty huge deal to be sentenced to 6 months of unemployment...

        Wait - you can still work as a paralegal or in a similar support role until you pass the bar as a full-blown lawyer, no?

        • by nealric (3647765)
          In theory yes. In practice, probably not. Law firms don't want to hire people with law degrees as paralegals since they are a high flight risk once they pass the bar. Non-legal employers don't like hiring law graduates for the same reason.
    • Students weren't unable to complete exams; they were unable to upload the exams, which you need to do after you get home (or to a hotel) after the exam. It gets stores on your laptop (presumably with public key encryption) in the meantime. Examsoft's servers ran at least 50% slower than they had in the past; the company hasn't announced why.

      The only trick is that some jurisdictions required you to upload the exam within a few hours, so Examsoft had to contact those jurisdictions and get them to extend the

    • by Dabido (802599)
      World without Lawyers [youtube.com]
  • Lockdown (Score:4, Insightful)

    by GigaplexNZ (1233886) on Thursday July 31, 2014 @06:08AM (#47572979)

    Besides handling the uploading of completed exam questions, ExamSoft locks down the computer on which it runs, so Wikipedia is not an option.

    Yeah, that'll work, because nobody has internet capable cellphones, secondary machines or even Virtual Machines.

    • Re: (Score:2, Insightful)

      by Anonymous Coward

      Besides handling the uploading of completed exam questions, ExamSoft locks down the computer on which it runs, so Wikipedia is not an option.

      Yeah, that'll work, because nobody has internet capable cellphones, secondary machines or even Virtual Machines.

      and exams are never held in controlled conditions on known hardware with invigilators..... but sure lets let law students BYOD to there exams....

      • Re:Lockdown (Score:4, Insightful)

        by Chelloveck (14643) on Thursday July 31, 2014 @10:46AM (#47574321) Homepage
        RTFA:

        The digital system for the exam works on usersâ(TM) personal laptop, which they bring to the testing facility, where they download the company's application to the computers they use to take the tests. At the end of the exam, the file closes and locks. When the user is able to connect to the Internet, the file uploads. Users cannot make changes to the file after the conclusion of the test.

        You have to drive to the exam site but you're expected to bring your own equipment? Who thought that up? Rather than trying to intrusively lock down everyone's machine it would be far better to simply issue everyone a cheap tablet or netbook on which to take the exam. Controlled hardware, no need to try to "lock down" innumerable variations of BYOD. The ExamSoft web site says the software runs on "any modern machine", defined as Windows, Mac, or iPad purchased in the past 3-4 years. But disable any anti-virus, and no VMs. They're basically trying to secure any random machine off the street to prevent cheating. That's a very fine example of "doing it wrong".

        • by Anonymous Coward

          Even better, when I took the exams, they didn't even provide internet access to upload. You had to wait until you got home or other internet access to upload it. Goodness only knows what could have happened in the mean time. Plus, a guy in my class ran it in a virtual machine as all he had was a Linux machine. Apparently, running dmidecode and vboxmanage was enough to do so.

        • by buhusky (3064123)
          You've never asked a just-graduated law student to take a 6-hour type as fast as you can essay exam on a keyboard they aren't intimately familiar with. Death may ensue to those who ask such a question.
    • by pla (258480)
      Yeah, that'll work, because nobody has internet capable cellphones, secondary machines or even Virtual Machines.

      I had wondered about that myself... Do they seriously not require taking the Bar on controlled hardware? Hell, a bunch of geeks should take and "ace" the Bar just for the sake of making fun of it.

      "Oh, JD huh? Yeah, I have one of those too, figured I'd just drop by in my spare time and take a go at it, and whaddya know, perfect score. Oh, sorry about all those 100 hour weeks of study you pu
    • Re:Lockdown (Score:5, Informative)

      by Anonymous Coward on Thursday July 31, 2014 @07:17AM (#47573157)

      For exam takers, a secondary machine and internet capable cellphone is not an option. The exam doesn't take place in your living room but at a monitored location. In my state, the exam proctors don't even let you bring in your own pencil. Pulling out a cellphone would be a great way to be kicked out and never be allowed to take the exam again.

      I am not sure the current state of virtual machines and ExamSoft, but at least a few years ago the ExamSoft software would not run when a virtual environment was present. While I am sure people have found workarounds, the point of the software (and why most law schools and state bars use it) is to avoid that result.

      • by Anonymous Coward

        I wonder what all of these skill based systems will do when Neural interfaces and cybernetic coprocessors become possible. do courtrooms even have faraday cages? Else lawyer dude could link in to Lexus Nexus and a legal expert system on the fly and eat the other lawyer for breakfast. Same for doctors linking in to a medical expert system, the AMA datastore, and others on the fly.

        is it really cheating then, or a hidebound insistence on doing things the way they have always been done, linked with a large a

      • there exam taking best practices (including disabling antivirus programs) and NO VM??? on your own hardware?

    • Re:Lockdown (Score:4, Informative)

      by sandytaru (1158959) on Thursday July 31, 2014 @09:41AM (#47573841) Journal
      Last time I took a test (CAPM), the testing place gave me a temporary locker to put my stuff, and also requested that I turn my pockets inside out to show I didn't have a tiny cell phone or something hidden in them. They take it pretty seriously.

      I noticed that the exam software we used was running on XP and appeared to have been originally programmed for Windows 98. I wonder if they ever upgraded those boxes to Win7...?
    • by Etherwalk (681268)

      Besides handling the uploading of completed exam questions, ExamSoft locks down the computer on which it runs, so Wikipedia is not an option.

      Yeah, that'll work, because nobody has internet capable cellphones, secondary machines or even Virtual Machines.

      It doesn't need to be perfect, just decent enough to make it harder to cheat. Things like the consequences of getting caught also apply--law is a highly regulated profession, and getting caught would keep a person from ever becoming a lawyer. Failing the bar exam generally just means you retake it six months later and study more.

      Cellphones are not permitted in the exam room; so are second computers; and I believe the software is designed not to run on at least some class of virtual machines.

  • by Mr D from 63 (3395377) on Thursday July 31, 2014 @06:19AM (#47573003)
    Sue the bastards... but they might need to hire a lawyer to do it.
  • "...leaving many graduates temporarily unable to complete the exams needed to practice law."

    I guess that's what we call Artificial Intelligence.

  • I hope they have a good EULA ;-)

  • Trust (Score:5, Insightful)

    by DoofusOfDeath (636671) on Thursday July 31, 2014 @08:00AM (#47573323)

    If we can't trust these applicants to take the test honestly, how can we trust them to act as officers of a court?

    • I took the bar exam this week. My experience was that there are lots of restrictions on what you're allowed to bring in with you (in a clear ziplock bag), but they are not cavity searching you for violations, or even really looking at your bag at all. I.e. to a large degree they are trusting us to take the test honestly.

      Now of course the nerds among us (you, me, most everyone else here) can think of clever ways someone could conceivably cheat—a tiny scroll in 4pt font rolled up into your pen, rul
      • by mjwalshe (1680392)
        But when you take a relatively less important exam such as a ccna you are not allowed to do it on your own equipment even more so for the CCIE.
        • I can't speak to those tests or their security procedures, but it may have less to do with the importance of the test itself than the power of the testmaker. Cisco is not an arm of my state's Supreme Court, and they don't have the power to e.g. look through all my civil and criminal records (sealed, juvenile, expunged, all of them), bring me up on perjury charges, or permaban me from practicing my profession if I try to hoodwink them. The state bar is, and they do have those powers.

          Not defending ExamSo
          • by mjwalshe (1680392)
            Quite so the exams for the state bar exam should have more security than a CCIE hell my piddling Security+ exam was locked down tighter
    • by Calinous (985536)

      Those are preparing to be lawyers, not judges or prosecutors.

      • Those are preparing to be lawyers, not judges or prosecutors.

        I thought that even civil and defense lawyers are considered officers of the court.

        I also think that even they are given certain powers not available to regular citizens, such as issuing subpoenas. I thought that was one of the reasons for requiring even them to be of good character.

        • Correct, every sworn-in attorney is an officer of the court, with heightened ethical obligations. I don't think the subpoena power specifically is reserved for lawyers; if you represent yourself you can still subpoena witnesses.

          I know there's some special stuff attorneys get to do by virtue of being officers of the court, but the main thing is actually practice law (represent clients) and keep nearly unassailable confidentiality (atty-client privilege). One of the gotchas is that your duty to the cou
    • by TubeSteak (669689)

      how can we trust them to act as officers of a court?

      You shouldn't.
      Any system based on trust will be abused.

    • By that logic, if all you need is 'trust,' why have trials? Can't we 'trust' that the police got the right man? Can't we 'trust' that the DA has his facts in order?
  • Oh wait... they haven't passed the bar yet.

  • by Anonymous Coward on Thursday July 31, 2014 @09:12AM (#47573617)

    So I took the NY Bar yesterday and can validate this is all true.

    This thread has seen a lot of jokes, but just to put things in perspective:

    This software costs $100 for a one time use accounting for two 3 hour sessions. Furthermore, it is bound to your computer, so if you need to transfer it, you need to pay for an additional license.

    The software is not complex. It has exactly 3 main functions:
    1. Provide limited word processing functionality
    2. Lock the user out of other programs
    3. Automatically upload the answer files to Examsoft servers upon closing.

    At my administration of the exam, there were perhaps 1000 people. Mine was in Albany, but there were also administrations in NYC (the largest), Buffalo, and I think one other location. So say 5000 people. Several other states were offering their bar exam with Examsoft on the same day, so lets be generous and say there were 50,000 students who needed to upload files over the course of an evening. The files uploaded consisted of 2 250k files in their proprietary format. So we are talking 25 whopping Gigabytes TOTAL being thrown at them. And they were paid roughly $5M for one day of testing. Also, bear in mind, that there is a winter administration as well, albeit with fewer candidates, and that Examsoft is used for many other types of test as well, so their yearly gross is probably well into the 8 figures.

    Yet somehow, they didn't have the bandwidth/and/or the server capacity to handle the connections or puny amount of data being thrown at them? This is unacceptable by any business standard. And they have been the exclusive provider of this service for years, so it's not like they had no notice of what kind of volume they should be prepared for!

    I personally paid $300 just to take this exam ($200 to NY for the exam itself, easily one of the cheapest fees among the states, + $100 for Examsoft). I also spent about $1000 in travel expenses. After many tries, I apparently managed to upload the files (my software said they failed, but I received confirmation emails saying they had succeeded). Examsoft can't confirm either way whether data corruption has occurred. If they did not manage to upload in one piece, my exam will have automatically failed by the Bar's standards, a decision which is unappealable. So I will be out my financial investment, close to two months of study time, as well as 6 months in lowered earning potential SINCE I WON'T BE LICENSED.

    This was a massive, massive failure, and I will frankly be shocked if multiple lawsuits aren't filed against Examsoft over this.

    • by Shimbo (100005)

      This was a massive, massive failure, and I will frankly be shocked if multiple lawsuits aren't filed against Examsoft over this.

      However, those that haven't passed the exam won't be allowed to sue; Tisias must be laughing in his grave. Seriously, though, the whole online examination business needs a shakeup.

      • by nealric (3647765)
        You don't need to BE a lawyer to sue and you don't even need to hire one. You can sue on your own behalf without a license.
      • by tlhIngan (30335)

        However, those that haven't passed the exam won't be allowed to sue; Tisias must be laughing in his grave. Seriously, though, the whole online examination business needs a shakeup.

        No, those students will hire a lawyer to sue.

        Remember the whole self-representation thing (as in - you don't do it. "A man who is his own lawyer has a fool for a client".

        And it'll be a lawyer representing soon-to-be lawyers against a company that isn't a bunch of lawyers.

        • by Shimbo (100005)

          No, those students will hire a lawyer to sue.

          Yes, of course. I was making a joke. Obviously not a very good one.

    • So if they gave you the software that can be used on any computer, why force people to travel? And if you are going to be forced to travel, why not use one of the hundreds of existing test centers around the country? I took every major test since I started college at the same Prometric test center on campus, because they were licensed to handle almost anything.
  • by Anonymous Coward

    ... So the end result is fewer lawyers in the world?

    That's not a bug. It's a feature.

  • Shows you just how much of an unjust, deeply shackled, and corrupt society you live in. At least I can find state law online these days. I can go through the thousands upon thousands of pages myself if I need to.

  • by Anonymous Coward

    My wife was one the exam takers affected by the examsoft upload issue. I'm not sure that most people understand how much stress is on this test. My wife literally spend 9 hours a day, 7 days a week for 10 weeks studying for this test. This is the second time she's taken it because she went to school part time and has had two kids during that time and she just barely missed the pass line on her first go around. The stress is beyond immense because 3-5 years of school are rendered meaningless unless she can p

    • My wife's a doctor and we recently moved to a new state with very protectionistic licensing policies. For example, you're required to have passed the medical boards within the last ten years. Doesn't matter if you're a professor of medicine at Harvard: you had to have passed the boards recently. You know, the ones new doctors take in their senior year of med school when they've been doing nothing but studying for the last for years straight and it's still fresh in their minds. So my wife, who's owned a succ

  • It's actually interesting that the bar exam is administered using software running on somebody's personal computer. All the computer based tests I've taken (GRE, various vendor certification tests) have been at a Prometric or similar testing facility on their hardware. They're actually pretty strict -- no personal items of any kind allowed in the room, the only scratch paper you get is a whiteboard and marker, etc. I know the bar exam isn't a multiple choice test you can memorize the answers to, but even so

  • Until someone pushes a shopping cart into their minivan. Then it's "Kill that motherfucker bring me his head on stick!!!"

A CONS is an object which cares. -- Bernie Greenberg.

Working...