Matahari Project

Matahari 0.5.0 released

2011-10-21T00:00:00-07:00

New release of Matahari now in circulation. Read the release note here: https://fedorahosted.org/pipermail/matahari/2011-October/002141.html

Automated Testing of Matahari

2011-10-05T00:00:00-07:00

We recently made some upgrades to our automated build and test setup. Russell posted some information about it over on his personal blog, russellbryant.net.

New version (0.4.2) and updated developers manual

2011-08-29T00:00:00-07:00

We've been busy testing out the agents that will be made available in Fedora 16 and RHEL 6.2. So far we are down to only a handful of bugs to squash and have new builds already in place for Fedora 15/16/rawhide. The Matahari team urges you to test out these new builds and if you get stuck setting up the software please take a look at Matahari getting started wiki page.

The developer's manual also got some attention and was updated with an API and Security section. The API section covers the existing agents that will officially be in Fedora 16. The Security section covers configuring your brokers and agents with Kerberos and SSL. Please head on over to the developers manual for more information.

Matahari to support DNS SRV records

2011-06-16T00:00:00-07:00

From wikipedia

A Service record (SRV record) is a specification of data in the Domain Name System defining the location, i.e. the hostname and port number, of servers for specified services. It is defined in RFC 2782. Some Internet protocols such as the Session Initiation Protocol (SIP) and the Extensible Messaging and Presence Protocol (XMPP) often require SRV support by network elements. An SRV record has the form: _service._proto.name TTL class SRV priority weight port target

What does this mean for Matahari? We've now got the ability to reorganize, prioritize, and balance our Matahari brokers. Patch Pending API Tests

To get a feel for DNS SRV records we've got a README for those interested in testing out the patches and providing some feedback.

In the next couple of weeks we are hoping to have an in-depth developers guide and and administrators guide. These manuals are outlined to cover creating agents, unit testing, functional testing, and managing it all. The developers guide has been started and hosted on one of the developer's forked repo's. Yes it is written using Latex, although, Publican is looking like a pretty good alternative.

How-to create a Matahari agent

2011-06-01T00:00:00-07:00

The matahari team recently published an excellent wiki page on how to create an agent. What is great about this article is that the developer is taken through every step from defining a QMF schema to how to build the library in CMAKE. In addition, we are presented with example code for the running agent and a library that the agent can interface with.

This is a highly recommended read for developers who are interested in a cross-platform, guaranteed delivery, and secure messaging system.

Also have a look at the coding guidelines for upstream development expectations.

Matahari upcoming actions items

2011-02-11T00:00:00-08:00

We've moved Matahari over to github.com.

Packages will officially be released in Fedora 15 (matahari, mingw32-matahari) and will be making their way down into Red Hat Enterprise Linux.

If interested please see the getting started page to familiarize yourself with the project.

Quick setup for having Matahari interact with federated brokers

2010-11-10T00:00:00-08:00

One of the things with matahari is that we didn’t want our agents to be tied down to just 1 broker. With qpid we can setup broker federation and squash any of the use case scenarios that may involve differences in location, etc.

To start setup 2-3 brokers, in this writeup there are 2 brokers running on one machine and a 3rd on a second machine.

BrokerA has ip 192.168.1.3 and a port 10001
BrokerB has ip 192.168.1.3 and a port 10002
BrokerC has ip 192.168.1.5 and a port 10001

Startup all three brokers and for the 2 that are on the same machine some options will need to be set

BrokerA
# qpidd -p 10001 --pid-dir /tmp/brokera --data-dir /tmp/brokera --auth no
BrokerB
# qpidd -p 10002 --pid-dir /tmp/brokerb --data-dir /tmp/brokerb --auth no
BrokerC
# qpidd -p 10001 --pid-dir /tmp/brokerc --data-dir /tmp/brokerc --auth no

Now we need to link all 3 together (federated) into a broker exchange. To do so run the following on any of the machines with brokers to be linked or a machine with no broker at all. The next tools being listed do not require a broker to be running in order to network the brokers together.

qpid-route is the utility being used and for simplicities sake we will be setting up dynamic routes (described in section 1.4.3.2 of the previous link)

# qpid-route dynamic add 192.168.1.3:10001 192.168.1.3:10002 amq.direct
# qpid-route dynamic add 192.168.1.3:10002 192.168.1.3:10001 amq.direct
# qpid-route dynamic add 192.168.1.3:10001 192.168.1.5:10001 amq.direct
# qpid-route dynamic add 192.168.1.5:10001 192.168.1.3:10001 amq.direct

# qpid-route dynamic add 192.168.1.3:10001 192.168.1.3:10002 qmf.default.direct
# qpid-route dynamic add 192.168.1.3:10002 192.168.1.3:10001 qmf.default.direct
# qpid-route dynamic add 192.168.1.3:10001 192.168.1.5:10001 qmf.default.direct
# qpid-route dynamic add 192.168.1.5:10001 192.168.1.3:10001 qmf.default.direct

# qpid-route dynamic add 192.168.1.3:10001 192.168.1.3:10002 qmf.default.topic
# qpid-route dynamic add 192.168.1.3:10002 192.168.1.3:10001 qmf.default.topic
# qpid-route dynamic add 192.168.1.3:10001 192.168.1.5:10001 qmf.default.topic
# qpid-route dynamic add 192.168.1.5:10001 192.168.1.3:10001 qmf.default.topic

Now I know this looks like a lot of repetition but the above is required since we are creating a bidirectional route. Looking carefully we see that the brokers are being flipped an added to the same exchange.

Speaking of broker exchange you’ll notice amq.direct, qmf.default.direct, qmf.default.topic. These exchanges are the default exchanges that need to be supported in any AMQP broker. To get a better idea of these routing algorithms there is a blogpost that covers it pretty clearly.

Moving on we need to do a quick check on the topology which can be accomplished with the following

# qpid-route route map 192.168.13:10001

Finding Linked Brokers:
 192.168.1.3:10001... Ok
 192.168.1.5:10001... Ok
 192.168.1.3:10002... Ok

Dynamic Routes:

Exchange qmf.default.topic:
 192.168.1.5:10001 &lt;=&gt; 192.168.1.3:10001
 192.168.1.3:10002 &lt;=&gt; 192.168.1.3:10001

Exchange qmf.default.direct:
 192.168.1.5:10001 &lt;=&gt; 192.168.1.3:10001
 192.168.1.3:10002 &lt;=&gt; 192.168.1.3:10001

Exchange amq.direct:
 192.168.1.5:10001 &lt;=&gt; 192.168.1.3:10001
 192.168.1.3:10002 &lt;=&gt; 192.168.1.3:10001

Static Routes:
none found

The brokers are now federated (networked together) so lets do something useful. We will connect one of matahari’s core agent to any of the brokers on both machines.

BrokerA
# matahari-hostd --port 10001 --broker 192.168.1.5

BrokerC
# matahari-hostd --port 10001 --broker 192.168.1.3

Above we’ve just connected the agent on one machine to the broker on the other and vice versa. So now if we bring up qpid-tool on BrokerB we should see that 2 agents are connected within this broker network and we will be able to interact with those agents no matter where we are.

BrokerA
# qpid-tool 192.168.1.3:10002

qpid: agents
Agent Name                         Label QMF version
1.0 BrokerAgent                      QMFv2
1.redhat.com:matahari:4d3d4442-562a-4514-a639-b366ef17e306 QMFv2 Agent 2
1.redhat.com:matahari:fb7584d4-8d10-4cea-ab30-ae4afaea1060 QMFv2 Agent 2

With both agents connected we can access their methods and pull some data from it.

qpid: list host
Object Summary:
 ID  Created  Destroyed Index
 =====================================================
 161 22:03:47 -     com.redhat.matahari:host:
 101 21:05:51 -     com.redhat.matahari:host:

qpid: show 161
Object of type: com.redhat.matahari:host:_data(7297d90c-5e2a-557c-7b58-90cdd4d916f2)
 Attribute     161
 ===================================================================
 uuid       ec742c182da05427605f96b300000014
 hostname     im.gangstar.com
 is_virtual    False
 operating_system Linux (2.6.34.7-61.fc13.i686)
 memory      3057352
 swap       2047996
 arch       i686
 hypervisor    
 platform     32
 processors    2
 cores       4
 model       Intel(R) Core(TM)2 Duo CPU   T9600 @ 2.80GHz
 last_updated_seq 83
 last_updated   Fri Nov 12 22:10:37 2010
 load_average_1  0.000000
 load_average_5  0.020000
 load_average_15  0.000000
 memFree      186788
 swapFree     1987552
 procTotal     394
 procRunning    1

qpid: show 101
Object of type: com.redhat.matahari:host:_data(7297d90c-5e2a-557c-7b58-90cdd4d916f2)
Attribute     101
===================================================================
uuid 3d4c71ac60c0b113d8ce73b700000016
hostname im.gangster-twice.com
is_virtual False
operating_system Linux (2.6.34.7-61.fc13.i686)
memory  2036688
swap       4095996
arch       i686
hypervisor    
platform     32
processors    2
cores       4
model       Intel(R) Core(TM)2 Duo CPU   T7500 @ 2.20GHz
 last_updated_seq 795
 last_updated   Fri Nov 12 22:12:03 2010
 load_average_1  0.120000
 load_average_5  0.120000
 load_average_15  0.110000
 memFree 168912
 swapFree 4091164
 procTotal 384
 procRunning 4

Pretty simple and really cool :D

Matahari Development screencasts

2010-11-03T00:00:00-07:00

Put up some screencasts to give everyone an idea of its capabilities. This is being shown using the development version of qpid 0.7. The next stable release is going to be 0.8 and that should be out very soon. Once 0.8 is released work will continue on making sure the windows ports are updated to the new codebase. Matahari shouldn't require any changes as the cross platform code is independent of the qpid changes.

I'm not that great with taking screencasts so forgive me if they are hard to see

matahari network agenthttp://www.youtube.com/watch?v=iA-k5pTWfGA&hd=1 matahari host agenthttp://www.youtube.com/watch?v=Q_WgwQkJMN8&hd=1