Catch up on stories from the past week (and beyond) at the Slashdot story archive

 


Forgot your password?
Close
typodupeerror
×
IBM IT

IBM Open Sources UIMA 23

Posted by ScuttleMonkey from the more-the-merrier dept.
psykocrime writes "Line56 is reporting that IBM has open-sourced core components of their UIMA project. UIMA is (or was) an IBM Research Project for managing "unstructured information" such as free-text. IBM describes UIMA as: 'an open, industrial-strength, scalable and extensible platform for creating, integrating and deploying unstructured information management solutions from combinations of semantic analysis and search components. [...] and makes the core Java framework available as open source software to provide a common foundation for industry and academia to collaborate and accelerate the world-wide development of technologies critical for discovering the vital knowledge present in the fastest growing sources of information today.' The newly opened source is available at SourceForge."
This discussion has been archived. No new comments can be posted.

IBM Open Sources UIMA More

IBM Open Sources UIMA

Comments Filter:

  • Translation... (Score:2, Interesting)

    by IAAP ( 937607 )
    an open, industrial-strength, scalable and extensible platform for creating, integrating and deploying unstructured information management solutions from combinations of semantic analysis and search components.

    FTFA: UIMA is about text analytics, which encompasses functions like search and pattern analysis. "It's able to create sophisticated solutions that extract insight from a large amount of unstructured data stored in computers," says Dr. Nelson Mattos, IBM VP of Information and Interaction. "The goal i

    • Re:Translation... (Score:3, Interesting)

      by Eightyford ( 893696 )
      In other words, it's now an opensource way of picking out themes in text. So, I guess you can (or the company you work for) have a version of Carnivore

      And so can China. Woohoo...
    • "semantic analysis" and "themes in text"? I see a great context based spell checker.

      • Sign me up, man! (Score:3, Insightful)

        by IAAP ( 937607 )
        Because, I can't make head nor tail of all this marketing speak. That's what I got out of reading TFA.

        If you have to rely on buzz words and jargon when communicating to the popular press, then you are saying nothing, or you don't understand what the fuck you're talking about.

        • Re:Sign me up, man! (Score:3, Interesting)

          by drinkypoo ( 153816 )
          That, or maybe you're just trying to get the word out to developers. Since the mainstream press will basically just print anyone's press releases as articles without revision, you have a pretty good chance of getting your word out there verbatim these days. It doesn't matter if they can understand it if they don't even try to understand it.
        • If you have to rely on buzz words and jargon when communicating to the popular press, then you are saying nothing, or you don't understand what the fuck you're talking about.

          Did it occur to you that it might be you, rather than the IBM researcher, who doesn't know what the IBM researcher is talking about? Pardon my being blunt, but what he was saying wasn't buzz word driven at all, and when you deal with topics this far removed from natural language, it's either communication in jargon or no communication
    • So, I guess you can (or the company you work for) have a version of Carnivore

      Or Google.

  • I admit it. (Score:4, Insightful)

    by jd ( 1658 ) <imipakNO@SPAMyahoo.com> on Wednesday January 25, 2006 @02:32PM (#14560913) Homepage Journal
    This is one of the very few times I'm actually more confused by TFA than I am by the Slashdot summary. So it's a text object with a semantic grep function? A collection of objects that can be used in a semantic wiki? A variant of Xanadu? A Java version of "Ask Jeeves"?

    • Re:I admit it. (Score:3, Informative)

      by blamanj ( 253811 )
      It's a framework for running custom analyses. For example, you could write a piece of code that looks for items like "VP" or "Director" or "CEO" next to peoples' names and then tag those as "Corporate executives". Running several different sets of analyses on your text increases the semantic information. Ultimately, you can then search not just for text (a la Google) but also on these advanced semantics, e.g., find all Corporate executives that occur near "convicted".
      • I suppose you could also use it to create a system log miner to figure out what happened and what it's related to. Possibly, enough intelligence could be put into it so that the thing only paged you when something was ACTUALLY wrong rather than whenever something SEEMS to be wrong.
    • From the IBM UIMA SDK Page ( http://www.alphaworks.ibm.com/tech/uima/ [ibm.com])

      Unstructured information management (UIM) applications are software systems that analyze unstructured information (text, audio, video, images, etc.) to discover, organize, and deliver relevant knowledge to the user. In analyzing unstructured information, UIM applications make use of a variety of analysis technologies, including statistical and rule-based Natural Language Processing (NLP), Information Retrieval (IR), machine learning, an

    • What is the UIMA framework? (Score:5, Informative)

      by Coruscater ( 736540 ) on Wednesday January 25, 2006 @03:27PM (#14561475)

      I am a UIMA user within IBM (but not an official spokesperson), so I can take an unofficial stab at answering that question. The UIMA framework (now on Sourceforge) is primarily a Software Developers Kit. It is not really intended to "do" anything out of the box; it does come with a handful of illustrative example applications but does not really provide direct end-user functionality (any more than a Java SDK does). Instead it provides a variety of interfaces and framework code to enable programmers to build applications that analyze unstructured information such as text (or images, audio, video, etc.).

      Some key capabilities provided by the UIMA framework include:

      1) A data structure, known as the Common Analysis Structure (CAS), that holds the results of analysis. It is essentially a big bag structure that has a lot convenient methods for using that bag to hold a "subject of analysis" (e.g., a text string) and information about that subject of analysis (e.g., annotations over character spans in a text string). Also included are methods for serializing and deserializing the CAS into an XML format.

      2) A set of interfaces for components that manipulate CAS's. For example, an "Annotator" takes a CAS that contains an existing subject of analysis and adds more information about that subject of analysis into that CAS. Developers are expected to implement those interfaces to build analysis capabilities.

      3) An XML format for providing metadata about those components and aggregates of those components.

      4) A "collection processing manager" that takes a metadata specification of an aggregate component and executes those components, either locally or remotely, via SOAP or some other protocol.

      So why use UIMA? If you are building some specialized analysis component, the UIMA framework provides common structures and interfaces for building and deploying these components. If you are building an end user application, basing the application on the UIMA framework allows you to draw on whatever "best of breed" UIMA components are available for providing the various elements of functionality that you want.

      The range of applications that can potentially benefit from a combination of analysis components is virtually limitless. The parent post mentions a few (e.g., semantic grep, natural-language question answering). There are many others such as machine translation, summarization, etc. Many of these applications can benefit from common primitive subcomponents such as being able to identify the subject and verb of an English sentence. A common framework like UIMA can help developers to build, share, and reuse these components.

    • Their 340 page user manual [ibm.com] (just under 4MB) is actually quite readable. Chapter 2 gives you a conceptual summary, and the rest of it is focused on actually using it.

      What is it in a nutshell? Unstructured data -> UIMA -> Structured Data. It's a means of converting unstructured, or more likely semi-structured data into what appears to be relational tables (with indices).

      UIMA is really a collection of analysis engines, which you can write, and tends to specialize in some kind of knowledge extraction,

  • IBM is open sourcing all its abandoned garage projects.

    Is it good? Is it bad? Who knows.

    • Re:Translation for common people: (Score:5, Insightful)

      by TheRaven64 ( 641858 ) on Wednesday January 25, 2006 @02:48PM (#14561077) Journal
      If they're abandoned, then it can only be good. Either someone will pick them up and do something useful with them, or they will be ignored. If they remain closed, then only the latter of these is an option.
    • I think it's a good thing. One of the great strengths of Opensource is doing cool things which products which aren't really initially commercially viable, becuase it opens up the code to an audience where there is a lack of profit motive.
    • IBM sells servers, DB2, and WebSphere (among other products). If more apps use this framework, it would probably help them sell more of all three.

    • IBM is open sourcing all its abandoned garage projects.

      If you stop and think about it for a moment, this would be stupid for IBM to do, and it's obviously not what they are doing.

      Paradoxically, it costs money to open source something (at least for a big company like IBM). Getting the package ready for the light of day, getting all the approvals and sign-offs, and staffing the resource to keep working on it, all take money.

      IBM wouldn't do that for something it doesn't consider valuable in some way

  • When will IBM opensource UFIA?
  • This should have been in the summary, but I didn't put it in, because I did not realize this had been discussed a few months ago:

      http://slashdot.org/article.pl?sid=05/08/08/233723 6&from=rss [slashdot.org].

    Not exactly a dupe, since at the time of that story, the code wasn't yet available as open source. So consider this a follow-up to the above story.

Slashdot Top Deals

"I'm a mean green mother from outer space" -- Audrey II, The Little Shop of Horrors

Close