Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 


Forgot your password?
Close
typodupeerror
×
Programming IT Technology

Open Source Message Queuing System 350

Posted by Zonk from the web-services-with-a-smile dept.
psicode writes "John Davies has announced AMQ, an effort at JPMorgan Chase & Co. to create an open-source message queuing system that can compete with proprietary message systems like IBM MQSeries and Tibco/RV. The announcement was made at the annual conference Web Services on Wall Street during Davies' presentation on February 1. eWeek has an article today with more details and some funny statements about Red Hat, SuSE and Sun possibly integrating AMQ into their "kernel". If JPMorgan Chase & Co. follows through with their announcement and they come up with a suitable open-source license, AMQ could become the Apache of messaging systems."
This discussion has been archived. No new comments can be posted.

Open Source Message Queuing System More

Open Source Message Queuing System

Comments Filter:

  • Web services (Score:4, Funny)

    by WesG ( 589258 ) on Tuesday February 08, 2005 @04:12PM (#11610942)
    I think web services are going to be the next big thing.

    yay

    • Re:Web services (Score:5, Informative)

      by Anonymous Coward on Tuesday February 08, 2005 @05:40PM (#11612128)
      You are indeed quite funny. However, the total tonnage of posts later in other threads asking about how MQ-Series competes with HTTP, InstantMessanger, J2EE containers has just made it clear to me that Slashdot hasn't been particularly illuminating to nerds - it's spent far to much time on MPAA/RIAA.

      Message-oriented middleware (MOM)is broard term encompasing a messaging system that may be sychronous or asynchronous. MQ-Series runs on over 30 platforms and allows developers to integrate apps on wildly differnet systems. How would you get an app on a Tandam-Hymalaya nonstop machine to talk to an AS400, then pass the results off to Linux box as well as supplying an audit trail to a Z/OS Mainframe?

      Remember that some of these platforms might not even support TCP/IP at your site.

      Hint: Yahoo Instant Messanger will be viewed as a sub-optimal solution.

      AC

      p.s. I was just made redundant at work yesterday. (The company closed - everyone was let go) Time to stop posting and get another job....

  • Active MQ (Score:5, Informative)

    by LordMyren ( 15499 ) on Tuesday February 08, 2005 @04:14PM (#11610958) Homepage
    I thought that's what CodeHaus's ActiveMQ [codehaus.org] was for.

    CodeHaus rocks, even if they are overly java biased for my tastes.

    • Re:Active MQ (Score:4, Informative)

      by jdray ( 645332 ) * on Tuesday February 08, 2005 @04:22PM (#11611081) Homepage Journal
      I think the point behind AMQ (at least it was heavily-pointed-out in the article) is that it supports bindings for C, C++, C#, Java, etc., etc., etc. ActiveMQ seems to only support bindings for Java.

      • Re:Active MQ (Score:4, Informative)

        by Anonymous Coward on Tuesday February 08, 2005 @04:35PM (#11611281)
        IBM MQ is extremely useful becuase it goes across platforms (Windows, any type of Unix, mainframe) as well as languages (C, C++, C#, Java, perl, Cobol, etc).

        Any Java-centric system is useful to tie Java programs together, but not for firm-wide integration. The same story goes for platform-specific products. Unless your firm uses only one langauge/platform, in which case you're not likely to need queueing...

      • Re:Active MQ (Score:2, Informative)

        by clockwise_music ( 594832 )
        At an old place of work we used SonicMQ [sonicsoftware.com] for our messaging.

        Now to be fair, the broker itself was quite stable, and if you were just using java, I assume that everything would be ok. However we were using a bunch of different languages (legacy systems) and some of the adapters for those languages were a joke.

        It became our developer in-joke. "Why did the system go down today? Sonic". Memory leaks, random crashing, threading problems, you name it. Caused us months of bug fixing. I'm sure (hope) that things

    • Re:Active MQ (Score:3, Interesting)

      by psicode ( 857306 )
      AMQ is not to be confused with ActiveMQ. ActiveMQ is an open-source Java JMS implementation. Sun's JMS API defines a unified interface to a messaging system. The problem with JMS is that it is Java centric and Sun did not specify a wire protocol or the binary message format. AMQ is an implementation of a messaging system that is cross language/platform, something that is, from what I know, currently only povided by commercial messaging implementations which are quite pricey.

  • I agree with the FP ?!?!?! (Score:4, Interesting)

    by MajorDick ( 735308 ) on Tuesday February 08, 2005 @04:18PM (#11611014)
    The trick is in Web Services, I was lead developer on a LARGE project during the DotCom Era (2000 ish) that used Vitria, a great system and its interop capablities blew my mind at first but if we could have utilized web services we would have halved developemnt time, not to mentio forgoing a 1 Million $ Vitria Liscence.

    It was cool though, we took about 6 different Apps on TOTALLY Different Platforms, 1 only HP UX, 1 , Two on Solaris ( on on 7 one on 2.51) One on DOS (Yes DOS it only ran there) and a couple on Windows, tied them all together through Vitria then hooked it up to a Web Front end, It was sooo slick the first VC that came to see our Proof of Concept (which was totally functional )gave us 15 Mil in VC

    But alas another DotCom fatality, we went from 12 employees to 165 in a year......can you say BAM

  • Article summary (Score:5, Funny)

    by wdd1040 ( 640641 ) on Tuesday February 08, 2005 @04:18PM (#11611016)
    Buzzword buzzword buzzword, incorrectly used terminology, buzzword buzzword.

  • Or ObjectWeb? (Score:3, Interesting)

    by wickedhobo ( 461297 ) on Tuesday February 08, 2005 @04:18PM (#11611022)
    How about Joram [objectweb.org] which is supposed to be pretty good.

    • Re:Or ObjectWeb? (Score:3, Informative)

      by eyeball ( 17206 )
      Joram is beautiful. Unfortunatly it's entirely java based. Sure you can interface to native C applications, but you still end up with a huge runtime, and sometimes you need really lightweight processes.

  • "What Is Message Queuing?" (Score:5, Informative)

    by modifried ( 605582 ) on Tuesday February 08, 2005 @04:19PM (#11611040) Homepage
    "Message queuing is a communication tool that allows applications to reliably interconnect in a distributed environment where one of the applications may or may not be available at any given time. The queue acts as a holding container for messages as they are sent between applications. The applications send messages to and read messages from queues to communicate back and forth. An application writes a message to a queue, which will then be received and processed by another application at some point determined by the receiving application. This type of communication is designed for asynchronous use where the applications involved are not waiting for an immediate response from the other end."

    From http://www.developer.com/net/asp/article.php/32979 11 [developer.com]

    • Re:"What Is Message Queuing?" (Score:4, Informative)

      by Profane MuthaFucka ( 574406 ) <busheatskok@gmail.com> on Tuesday February 08, 2005 @04:25PM (#11611122) Homepage Journal
      That little word "reliably" is more important than the blurb above states. Unlike some messaging systems, MQ will either deliver your message, or will let you know that it did not deliver your message. There is no such things as sending a note off into the wild blue yonder, only to have it mysteriously disappear without a trace.

      • <sarcasm>Really hard to do, if it is meant in the MQSeries way.</sarcasm> If MQSeries can't deliver a message, it puts it on a dead.letters.queue. I am not impressed.

    • Re:"What Is Message Queuing?" (Score:2, Insightful)

      by Anonymous Coward
      How is this different from e-mail? It sounds the just like e-mail that happens to be carrying messages that are intended to be parsed by applications, rather than humans. What's the magic? Is this just a more secure, reliable mail system?
    • So how is this different from email?

  • POPFile::MQ (Score:3, Funny)

    by JohnGrahamCumming ( 684871 ) * <slashdot.jgc@org> on Tuesday February 08, 2005 @04:22PM (#11611078) Homepage Journal
    You mean they can't just fork POPFile::MQ [sourceforge.net] and use it :-)

    John.

  • Open Source Honesty and Ethics (Score:2, Funny)

    by Anonymous Coward
    Wall Street is also looking for a new Open Source Ethics and Honesty implementation as current proprietary implementations seem to be faulty, buggy, and expensive to maintain.

  • MQSeries (Score:2, Troll)

    by sapped ( 208174 )
    We have had MQSeries at each IBM implementation that I have worked at (SAP software).

    My perception of MQ is that we would be better off throwing it away and FTPing files back and forth while using SAP's job control to kick things off. I cannot imagine how many weeks we have lost in a production environment because of some fiddly piece of junk involved in the MQ process decided to hiccup.

    • Re:MQSeries (Score:5, Informative)

      by azaris ( 699901 ) on Tuesday February 08, 2005 @05:01PM (#11611643) Journal

      My perception of MQ is that we would be better off throwing it away and FTPing files back and forth while using SAP's job control to kick things off.

      Good luck with writing ACID transaction support, rerouting in case of failure, dead letter handling and enterprise scalability features. Then get busy convincing dozens of sysadmins to bust their firewalls open to allow the antiquated piece of crap that is the FTP protocol to get further than your office router.

      Trying to replicate enterprise functionality with naive hacks makes no sense whatsoever. I pity the fool who gets to administer the kludge you leave behind when you switch jobs.

  • "they come up with a suitable open-source license, AMQ could become the Apache of messaging systems."

    If they want to be the Apache, then they'll have to use the Apache license too. Effects have causes.
    -russ

  • I bet this really pisses off the copyright expansi (Score:3, Insightful)

    by ShatteredDream ( 636520 ) on Tuesday February 08, 2005 @04:33PM (#11611255) Homepage
    As a voting libertarian, I whole-heartedly encourage companies to actively develop whatever technical solution that best fits their needs. This is one example of how open source software really fits in with the libertarian world view. It's not the "free software versus only commercial software," but a mix and match that lets both coexist. If someone wants to develop their own software, then let them and don't even begrudge them the right to give it away.

    This is what really pisses me off about the copyright expansionists. They don't want people to be able to easily develop software for free. Copyright expansionism, as pushed by groups like the "Progress and Freedom Foundation" is not about freedom, it's about protecting business against hobbyists and other "asymmetric competition." I bet it makes such groups' heads spin to think that one of the biggest corporations on Wall Street is now about to dive head first into open source development and that the result of this development could send shockwaves through a segment of the commercial software industry.

    The freedom to write open source software is the same freedom to write closed source software. If you erode the foundation for the former, then you have no right to demand respect for the latter. That is why when the big copyright cartels lobby Congress, I see nothing wrong in doing things like buying academic licenses for software and taking them out into the real world. The copyright holder makes no profit on the academic license, only the seller does.

    • Re:I bet this really pisses off the copyright expa (Score:2, Insightful)

      by Anonymous Coward
      >This is what really pisses me off about the copyright expansionists.

      I really appreciate source code licenses that actually let you use the source code permanently such as BSD.

      Copyright expansionist vehicles, such as GPL, end up costing too much in the long run.

  • a camel (Score:4, Insightful)

    by St. Arbirix ( 218306 ) <matthew.townsendNO@SPAMgmail.com> on Tuesday February 08, 2005 @04:36PM (#11611290) Homepage Journal
    an open-source message queuing system that can compete with proprietary message systems like IBM MQSeries

    Somewhere in IBM's headquarters there is a camel. A straw has just been placed on its back.

  • Check out TIP (Score:4, Interesting)

    by AaronW ( 33736 ) on Tuesday February 08, 2005 @04:36PM (#11611299) Homepage
    They should check out tipc for message queueing. TIPC is a reliable messaging protocol designed for clusters. TIPC is fairly transparent whether a task is local or somewhere else on the network.

    -Aaron
    • I hate to respond to my own post, but I forgot to include the link. TIPC is available under a dual license, both BSD and GPL. It can be found at <a href="http://tipc.sourceforge.net/">http://tipc.so urceforge.net/</a>. We're planning on using it in an embedded project. It has good performance and reliability capabilities.

  • Savings (Score:4, Insightful)

    by blackmonday ( 607916 ) on Tuesday February 08, 2005 @04:38PM (#11611318) Homepage
    Some years ago a company I worked for was quoted one hundred thousand dollars as the cost to license IBM MQ Series messaging. There could be some serious savings in using an OSS product like this.

    • I find it ironic, how IBM claims to be huge backers of open source. Yet Wall Street with all of their influence could not even convince IBM to open source the MQ Series product. They had to resort to building their own from scratch. The truth is, IBM only open source products where they have competitors they want to hurt, by commotitizing that type of software (i.e. supporting Linux & Eclipse to fight Microsoft over Windows & VS.Net).

      Not saying there is anything wrong with, they are a company that
      • The truth is, IBM only open source products where they have competitors they want to hurt, by commotitizing that type of software

        Not true. Big blue has open sourced bits that act as a loss leader, much like the 'free' items at a store, if it will make them cash longer term. I'd watch for IBM to make a low end 'open source' MQ, and have the connecting libraries match up perfectly with the WebSphere MQ. Much like what they are trying to do with Apache derby (formerly Cloudscape) using the same JDBC dri

  • Flabbergasted (Score:4, Insightful)

    by Anonymous Coward on Tuesday February 08, 2005 @04:39PM (#11611328)
    By the technical ignorance of some of the posters here... Before you post, please make sure you have at least an inkling about what you're talking about! A Message-Oriented Middleware system has absolutely nothing to do with IM, or Web Services (except in a very indirect way), or even REST systems. /. seems to be in dire need of real geeks, it seems...

    • Re:Flabbergasted (Score:4, Insightful)

      by slcdb ( 317433 ) on Tuesday February 08, 2005 @04:53PM (#11611515) Homepage
      As one other Slashdotter commented about the article itself:
      Buzzword, buzzword, buzzword, incorrectly used terminology, buzzword, buzzword, buzzword...
      The same could be said to summarize most of the Slashdot readers' comments.

      I guess somebody just needs to say it: the article was crap, and Slashdot was the wrong forum for discussing it.
  • Not that more options are not a good thing, but the Java Messaging Service (JMS) built into Jboss rocks. It is LGPL as well, which is one of the better open source licenses out there in my book.
    • JBoss uses both JMS and JGroups [javagroups.org], an excellent open source reliable multicasting library/protocol stack.

      According to the project plan [jboss.org] the goal is to have JMS for client-server structure communication, and use JGroups for peer to peer structure communication.

      I highly recommend JGroups for your intermachine networking needs! :-)
  • I think they're talking about OS queue stack for transaction process (i.e Stratus VOS queues).

    If a company was going to contribute that type of technology to the kernel, that would be awesome.
  • I work for a major airline (and the only one that has always been profitable =)) Because of the nature of our business, we have to use both Tibco and MQSeries for our communications. We must use both because of the transactional nature of running an airline (communication to / from the aircraft) and the non-transactional (Sabre / Ticketing). If there was a product that gave us the ability to do guaranteed / clustered messaging in both a synchronous and synchronous environments we would be all over it. In th

  • Horus/Ensemble? (Score:2, Interesting)

    by putko ( 753330 )
    Horus and Ensemble [cornell.edu] allow for process groups. Doesn't that do a superset of this stuff? It is open source.

    Horus is/was used to run the NYSE, and a prototype radar for the Aegis battlecruisers.

    Why is it not good enough?
    • I don't know much about Horus at all, but I would guess that its either a) too high-level and abstract of a solution to be of any practical use, or b) it doesn't natively support a number of environment like Java, C/C++, C#, etc.

  • #include <queue>
    #include <string>
    #include <iostream>
    using namespace std;

    int main()
    {
    queue<string> messageQueue;
    string message;

    cout << "Enter a message: ";
    cin >> message;
    messageQueue.push(message);
    cout << "Your message has been queued." << endl;
    }

    Licensed under the GPL! Feel free to use it to queue all the messages you want!

    (I hate it when people use totally generic terms like "message queuing" when they are actually referring to some much mor

  • please someone explain. (Score:4, Insightful)

    by Lord Bitman ( 95493 ) on Tuesday February 08, 2005 @05:07PM (#11611751)
    What is a message queue, why should I care, what does this have to do with anything, what are they talking about? In short: what kinds of messages are we talking about, what are sending them to what, and why do we need to queue them?

    "A message queue is a queue onto which messages can be placed."
    thanks a fucking lot google
    • I imagine JP Morgan developed their message queue to send stock or other financial transactions from one place to another, for example from customer, to broker, to exchange and back again. I imagine if you've bought stocks online it was all done through message queue transactions and its unlikely there was ever a human in the loop. If that is the context the environment they are using it in it has to both be extremely reliable, since large quantities of money are involved, and it has to handle potentially

  • MQ Advantage Is Mainframe Messaging! (Score:3, Interesting)

    by Black-Man ( 198831 ) on Tuesday February 08, 2005 @05:11PM (#11611800)
    It is the best way to get data from distributive systems to/from the mainframe. So... are they going to write a COBOL interface from 3270? Without IBM's help, I don't see that happening.

    For distributive->distributive, web services is all you need.

  • Queue Redux (Score:5, Insightful)

    by Doc Ruby ( 173196 ) on Tuesday February 08, 2005 @05:12PM (#11611816) Homepage Journal
    Keep in mind that Wall Street was probably *the* major engine driving Web applications into the mainstream in the 1990s. One reason Wall Street went crazy touting the Web as an unlimited business panacea was that it was actually like that for its own industry. By the time they promoted it in 1995, Wall Street shops had been churning out httpd patches, CGI apps, Perl revisions and code, DB connection software and techniques, TCL patterns, and all kinds of R&D that underpinned much of the development momentum. Message Queueing has a similar arc, because Wall Street both inhabits the bloodiest edge on which they can survive, and has the least tolerance for failure. The flakiness of network messaging is the root of some of the worst inhibition on network growth among users and deployers - Wall Street has been there, and has answers. Let's use them.
  • For the clueless among us (eg me) WTF is a 'message queueing system' and what would one use it for? Something to do with email? Kernel API calls? Tech support ticket system?

    • Re:And that would be what, exactly? (Score:5, Informative)

      by easter1916 ( 452058 ) on Tuesday February 08, 2005 @05:20PM (#11611919) Homepage
      No, it's akin to JMS (Java Messaging System) or, say, the TIBCO/Rendezvous system used to broadcast trading information to NASDAQ clients.

      Here at my workplace, we use "guaranteed messaging" to integrate disparate systems such as SAP R/3, as iSeries/AS400 running BPCS, a bespoke warehousing system. Messages (i.e., details of a transaction, for example an XML doc. containing details of a sales order) are transmitted between systems to keep them in sync. If the target system is down the messages are queued for delivery in the messaging system itself, and delivered later.

      Not a great explanation, but I hope it helps.
    • The need for a messaging system is hard to understand if you aren't used to working with/in hideously large, heterogeneous networks and groups of affiliated companies.

      What does it do? Virtually nothing. Just takes a few buytes of data and sends it from here to there. Thats the easy part - the hard part is that it does so with absolutely guaranteed results - if you send it, you know they will get it. Imagine chains of more or less reliable machines, software stacks and networks all linking together sender

  • My company built and sold a message-oriented messaging product that merged the pub/sub and transactional models several years back, and it included guaranteed delivery. And yes, there was a lot of discussion about open sourcing the code base. The system is still in daily production in a bunch of Wall St. firms.
    What these firms all wanted as a key requirement was clean integration with C/C++, and extreme performance. We had to handle thousands of messages (ranging into megabytes each) per second, with full g
  • The article has a "related item" link in it to this article about Oasis opting for UDDI [eweek.com] for publishing web services on a directory server. Although UDDI has been around for a while, this bit of push might help make it more common.

    I have always thought that Discovery is the most interesting area of research on the Internet. The idea of applications seeking out each other, learning about their resources and interfaces, and hooking up and communicating, without any human intervention is fascinating.

  • What about XML Blaster? (Score:3, Informative)

    by vegetasaiyajin ( 701824 ) on Tuesday February 08, 2005 @05:26PM (#11611990)
    XML blaster [xmlblaster.org] is an open source message oriented middleware that has bindings for multiple languages (C,C++, Java, PHP, Python, Perl).
    I have never used it, but it's been available since a long time.

  • Heh (Score:3, Interesting)

    by roman_mir ( 125474 ) on Tuesday February 08, 2005 @05:30PM (#11612031) Homepage Journal
    I wrote my own AMQ implementation about 4 years ago for a project that I was working on. The project was done in Java on Weblogic 5.1 (flashback - I am working on W5.1 on my current contract today,) and message beans were just introduced but we did not have them in our implementation.

    The project was for life insurance industry (for a little known company named Worldinsure that spent over 2.5*10^7 dollars on Aeron chairs and plasma monitors and - surprise, surprise - went belly up.) The app. would ask hundreds of questions that needed to be populated page by page. On some page the app would have to generate a PDF file, and since the Adobe pdf generator would crash the JVM after a couple of uses, we desynchronized it.

    I wrote a message queue that was generic enough to start whatever action that was appropriate for the message type. It was not too difficult, a relational db (Oracle) was used as the message storage mechanism and I had 2 different options for message retrieval - message polling from the java code and Oracle triggers. The tricky part was the multithreaded message queue processor that could start any action - a java class in a seperate safe thread. Every message had a status - New, InProgress, Failed. Messages could be setup to be retried with a retry-decrimenting counter, so a message would become 'Failed', once the retry counter reached 0. Message types had priorities assigned to them, so some messages would take priority over others. To make sure that all messages eventually were drained, there was also a priority timeout. Messages could be 'peeked at' and retreived. Once a message queue processor decides to retrieve a message, this message is not deleted, but marked with status 'InProgress', so that other processors would not pick it up as a new message.

    The interesting side-effect of this design was that it was possible to run as many message queue processors as one wished, thus an application could spread the load onto multiple computers (this adds redundancy and scalability.)

    All of this can be implemented quite easily to follow JMS standard, but JMS standard is even simpler (in terms of functionality) than what this designed allowed.

    • Forgot to notice that after a while we just rewrote the PDF generator (FDF populator) ourselves and it became stable and usable, but we left the MQ in anyway.

  • The article does not mention a web page for this project and googling did not help me. So, is this a "secret" open source project, or what?
  • ... the glue that holds the worlds together.

    Seriously, I would have to think a while to count the projects that I have used ISIS, JMS, etc. for. Asynchronous messaging makes architectures simpler.

    ISIS supported "virtual synchrony": guaranteed order of deliver also.
  • So this is quite interesting; I'll be very interested to see their implementation. With IBM's MQ, their big thing was that there was something like 8 function calls, tops. I can still see the IBM guy talking about how simple it was, listing out the functions.

    What he neglected to mention was that each of those function calls takes *structures* as arguments...some of those structures had a dozen+ members.

    To be fair, MQ is pretty slick; I wrote a connector between the webserver and an ES/9000 mainframe using

  • open-source message queuing system (Score:3, Insightful)

    by Scott7477 ( 785439 ) on Tuesday February 08, 2005 @05:47PM (#11612233) Homepage Journal
    What JP Morgan and other Wall Street firms use message queuing software for is trading in the financial markets. So if the software bugs out, they stand to lose millions of dollars per minute or more depending on the size of the trades they are executing. So I see this story as a positive for open source software because if Wall Street people think they can get a usable mission critical piece of software out of the open source community then no other potential target for OSS is invulnerable.

  • Why? (Score:3, Insightful)

    by Ratbert42 ( 452340 ) on Tuesday February 08, 2005 @07:00PM (#11613048)
    Everyone's asking "Why?" and debating the technical bits. I'll tell you why. So they can get a deep discount on MQSeries from IBM.

  • I bet guys who make MQSeries must be nervous (Score:3, Insightful)

    by Donny Smith ( 567043 ) on Wednesday February 09, 2005 @04:36AM (#11616492)
    I bet it'll take at least 5 years before this new thing makes any inroads in financial customers, but this must be really scary for IBM's MQSeries folks.

    Things like MQ Series are bread-and-butter of many sales reps as there's no serious competition to their solution - I haven't talked to many banks, but those to which I did all use MQSeries.

    OSS is moving from the edge to the data center, it's already half-way there. Five more years and banks will be able to get 80% of their apps under GPL license.

Slashdot Top Deals

"What the scientists have in their briefcases is terrifying." -- Nikita Khrushchev

Close