User Tools

Site Tools


documentation:rubypingpong

Mace Interoperability with Other Systems (Mace Wire Protocol)

Suppose you like Mace, but want to write your front-end in some other language, or simply want to include some distributed nodes written in other languages. This is not hard, as the example code below shows. The easiest way to implement this is by getting the non-Mace code to speak using the Mace wire protocol, which aside from a few quirks, is just a straightforward binary protocol.

Note: this example does not cover other alternatives, such as using XML-RPC, or specialized transport services, to allow processes to interact. Mace does include an HTTP server, and an XML-RPC compiler, but this is not well documented. For examples, see the dht application directory. The XML-RPC code also is believed to have some bugs that still need ironing out—it is not commonly used or maintained.

The working example code below is committed to the Mace trunk, and available at these links (below). There is no automation to keep the code in the repository in sync with the code on this Wiki, so no promises. But the repository's history is available, if you need to see what changes occurred or why. This example doesn't do much, just illustrates a simple Pong message bouncing back and forth between Mace and Ruby implementations. Some things would make this more complex (multiple “services” in Mace, multiple “transports” in Mace, or more than one message type), but it should be a sufficient starting point such that anyone comfortable with Ruby should get a feel for how to get going.

For the most recent version of this example, see: http://www.macesystems.org/svn/mace/trunk/docs/ruby/PingPong.rb

For comparison, the Mace PingPong service is at: http://www.macesystems.org/svn/mace/trunk/services/PingPong/PingPong.mac

For a more advanced PingPong example ruby code, that is faster, uses EventMachine, and supports modularity and registration uids, see: http://www.macesystems.org/svn/mace/trunk/docs/ruby/PingPong2.rb

Mace PingPong Code

Before diving into the Ruby code, first let's look at the Mace code for the PingPong service:

service PingPong;
 
provides Null;
 
trace=med;
 
constructor_parameters {
  MaceKey remote = MaceKey::null;
}
 
services {
  Transport t = auto(shared,[reliable],[]);
}
 
messages {
  Pong {
    int counter;
    uint64_t initTime;
  }
}
 
transitions {
  downcall maceInit() {
    if (!remote.isNullAddress()) {
      downcall_route(remote, Pong(0, curtime));
    }
  }
 
  upcall deliver(s,d,m) {
    downcall_route(s, Pong(m.counter+1, m.initTime));
  }
}

Importantly, the code defines one message, the Pong message, a deliver handler for responding to a Pong message, and an initialization function that starts the ping-pong if the constructor parameter “remote” is set when the service is created. (This is done in our example by setting the runtime parameter “ServiceConfig.PingPong.remote” to the address of the remote PingPong node).

The Pong message has two fields - a counter to increment and a timestamp to distinguish different ping pong chains. The upcall deliver handler uses Mace shortcuts to define parameters s (source), d (destination), and m (the message type – Pong). It simply responds to the source, incrementing the counter and echoing the timestamp.

Network Messages

For this Ruby example, we must first have message formats compatible with Mace. This is accomplished using the bindata ruby gem.

require "rubygems"
require "bindata"
 
class SockAddrBin < BinData::Record
  endian :big
 
  uint32 :addr 
  uint16 :port
end
 
class MaceAddrBin < BinData::Record
  endian :big
 
  sock_addr_bin :local
  sock_addr_bin :proxy
end
 
class TransportMessage < BinData::Record
  endian :big
 
  mace_addr_bin :src
  mace_addr_bin :dest
  uint32        :registration_uid
  bit8          :flags
  uint32        :len, :value => lambda { data.length }
  string        :data, :read_length => :len
end

In the above code, we set all messages to big endian. Mace, like most binary network protocols, adopts big endian for wire protocols. All messages Mace sends on a transport have a common header: two MaceAddr objects, followed by a registration UID, a set of flags, and a string, encoded by first including the length of the string.

The two MaceAddr objects are used for source and destination. (Destination is included to support proxying messages, not discussed in this tutorial – note that in general, the destination is not needed because the message has already arrived at its destination.) Each MaceAddr includes two SockAddr objects, the first is the “local” address and the second is the “proxy” address. Most of the time, only the “local” address is used, and represents the IP address and port of the first server socket listening at that client. In some cases, a client will run multiple transports, each one will run on a port at a given offset from the base port, which is normally assigned sequentially. More will be said later on the ports and addresses. In general, the proxy address (when provided) represents where messages can be sent such that they will be proxied to the given host, such as through firewall port forwarding, or a proxy process. NAT support is completely ignored by the example Ruby script, but you should be aware that in addition to a proxy address, a special flag might be set to tell the recipient that the host is behind a NAT with no proxy, in which case special handling must be used.

PingPong Service Message

class Pong < BinData::Record
  endian :big
 
  uint8  :message_type, :value => 1
  uint32 :counter
  uint64 :timestamp
end

Next we show the Pong message from the PingPong service. Note that in the transport message, this message is encoded in the data field. In this example, we do not show how to decipher and handle multiple message choices, as there is only one message type.

The Pong message has three fields (though in the Mace version, you can only see two). The first field is the message_type, which identifies this is message type number 1, corresponding to the Pong message. The messages in Mace are uniquely numbered between 1 and 255 (to fit in an unsigned 8 bit integer), no message can have more messages than that. The messages type numbers are typically assigned sequentially, but can be manually tweaked using the number attribute in the Mace syntax.

The other two fields are the counter, which is incremented each time the message is received, and a timestamp, which can be used in practice to distinguish multiple different Ping Pongs. The Mace client assigns it from the current time; any scheme may be used – it is simply echoed on each rebound.

Main Execution Loop

require "ipaddr"
require "socket"
 
myserver = TCPServer.new('127.0.0.1',10203)
sockaddr = myserver.addr
 
puts "Server running on #{sockaddr.join(':')}"
 
while true
  Thread.start(myserver.accept) do |sock|
    puts "#{sock} connected at #{Time.now}"
 
    message = TransportMessage.read(sock);
 
    remote_ip = IPAddr.new(message.src.local.addr, Socket::AF_INET)
    proxy_ip = IPAddr.new(message.src.proxy.addr, Socket::AF_INET)
    puts "Got connection from #{remote_ip}:#{message.src.local.port}/#{proxy_ip}:#{message.src.proxy.port}, size #{message.len}, regid #{message.registration_uid}"
 
    #assumes proxy not used.
    sock2 = TCPsocket.open("#{remote_ip}", "#{message.src.local.port}");
 
    begin
 
      while true
        msg = Pong.read(message.data)
        puts "Got Pong message! (timestamp: #{msg.timestamp} counter: #{msg.counter})"
 
        response = Pong.new
        response.timestamp = msg.timestamp
        response.counter = msg.counter+1
        puts "Response message message_type: #{response.message_type} counter: #{response.counter} timestamp: #{response.timestamp}"
 
        response_msg = TransportMessage.new
        response_msg.src = message.dest
        response_msg.dest = message.src
        response_msg.registration_uid = message.registration_uid
        response_msg.flags = message.flags
 
        response_msg.data = response.to_binary_s
 
        response_msg.write(sock2)
 
        message = TransportMessage.read(sock);
      end
 
    rescue Exception => e:
      puts "Error handling PingPong client: #{e}"
    ensure
      sock2.close
      sock.close
    end
  end
end

Socket Design and Port Offsets

The code starts by creating a server socket, and accepting connections. Each connection is handed off to a new thread, and loops on reading messages. Once it reads the first transport header, it opens a connection to the sender, for later use in sending Pong messages. One oddity in Mace is that TCP connections are unidirectional, that is–messages only flow in one direction along the socket. The reasons are historical, and had to do with complexities dealing with clients simultaneously connecting or disconnecting from each other. For now – just accept it as the way it works. To send a message back to the sender, you connect to the sender's address in the MaceAddr, unless a Proxy address is set, and then you'll need special NAT handling (not covered here). Our script assumes no proxying or NAT handling. Our script also ignores the port offset, since with a single transport, the port offset is typically 0. The port offset is not included in the transport header since it is supposed to match at the sender and receiver. To compute the port to connect to, you should take your own port offset, and add it to the port number in the MaceAddr. Connecting to the wrong port might confuse network messaging. Moreover, since sockets are unidirectional, you should not respond with messages on the socket you accepted.

The rest...

The rest of the code is pretty straightforward. A transport message is read from a socket, and then a Pong message is read from the data. A response Pong message is constructed, and then assigned to the field of the constructed transport message, which is then written to the socket. On any exception, both sockets are closed.

Running the Example

Run this script using ruby in one shell

$ ruby PingPong.rb

Run unit_app with the Mace PingPong service, specifying the remote service to be ruby script. This assumes you have built “unit_app” in mace/application/unit_app, and are in that directory.

$ ./unit_app -run_time 30 -service PingPong -MACE_ADDRESS_ALLOW_LOOPBACK 1 -MACE_LOCAL_ADDRESS localhost:5700 -MACE_LOG_AUTO_ALL 1 -ServiceConfig.PingPong.remote IPV4/localhost:10203
documentation/rubypingpong.txt · Last modified: 2012/07/18 19:02 by ckillian