ldipd worker model + protocol
=============================
ABSTRACT
Transport is separate from the master <-> worker relationship. By default, ldipd will
attempt to communicate via SSH with a worker node, to establish / start up worker processes.
One could easily use xinetd/inetd, rsh, or whatever else, the processes would just be
started by it's respective transport provider. Encryption is the responsibility of the
transport, LDIP does not encrypt any data it sends over the wire.
perl -MLDIP::Worker::AutoStart
AutoStart's BEGIN { } block creates a new LDIP::Worker object and calls start on it.
i.e:
my $worker = LDIP::Worker->new();
$worker->start();
Workers can be configured at constructor time, or using ldipp (the ldip protocol).
my $worker = LDIP::Worker->new(config => '/path/to/ldip.yaml');
or start a "dumb" worker (a worker that doesn't know what it's doing) and configure it:
perl -MLDIP::Worker::AutoStart
Protocol information follows, this is the proposed protocol flow. All data sent from the
client is prefixed by a >, all the non-prefixed data is from the worker itself.
LDIP::Worker (0.04) Startup
NOT_READY:CONFIG
>CONFIG
Please send configuration data (YAML) terminated by a . on it's own line.
>...LDIP CONFIG DATA...
>.
READY
>HANDLERS
HANDLERS:*:01noHandlers
>JOB 203021402
>Please send LDIP job data (LDIF) terminated by a . on it's own line.
>...LDIP JOB DATA...
>.
ERROR:QUEUE "<queue name>" not defined!
>QUEUE <queue name> [05someHandler:<md5sum>,10someOtherHandler:<md5sum>]
QUEUE:<queuename> defined, please load some handlers
>HANDLER <queue name>:05someHandler
>Please send LDIP handler code (Perl) terminated by a __END__ on it's own line.
>...LDIP HANDLER CODE...
>__END__
HANDLER:<queue name>:05someHandler loaded
>HANDLER <queue name>:10someOtherHandler
>Please send LDIP handler code (Perl) terminated by a __END__ on it's own line.
>...LDIP HANDLER CODE...
>__END__
HANDLER:<queue name>:10someOtherHandler loaded
>JOB 203021402
JOB:203021402:RESPONSE:[PASSED:Processing complete, and other stuff...]
>RESULTS 203021402
RESULTS:203021402:[ak1520_203021402.ldif,ak1520_203021402_<queue name>_metadata.ldif]
...RESULTS DATA...
.
...RESULTS DATA...
.
>DELETE 203021402
DELETE:203021402:Job Removed
...
>QUEUE <queue name> [05someHandler:<md5sum>,10someOtherHandler:<DIFFERENTmd5sum>]
QUEUE:HANDLER_CHANGED:10someOtherHandler md5sum changed, please re-send!
HANDLER <queue name>:10SomeOtherHandler
>Please send LDIP handler code (Perl) terminated by a __END__ on it's own line.
>...LDIP HANDLER CODE...
>__END__
HANDLER:<queue name>:10someOtherHandler loaded
... other jobs, etc ...
>BYE
__EOF__
Config Stuffs
=============================
Queues should be able to be configured as "not executable by remote workers" i.e., they
need more than access to state data to successfully work. For example: server startup
or shutdown, local account creation, other system administration tasks.
<queue_name>_remote : FALSE
other config stuffs:
local_workers : 4
worker_nodes :
- node : erie.mg2.org
workers : 4
transport : ssh
- node : t2000.wayne.edu
workers : 64
transport : socket
port : 9800
Startup w/ Workers
=============================
Upon ldipd startup, local worker processes should be started first. If there are remote
nodes, each node & socket is connected and cached. Opening of the sockets and ssh
commands should take mere seconds on a decently fast ldipd server (between 2 and 8 seconds).
Workers will be individually configured and primed for work, handlers uploaded and cached.
ldipd's IPC will be non-blocking so workers will be dealt with when data is received from
them and data will be sent immediately back. It there is work to do, it will begin to be
dispatched when the first node becomes ready, be it local or remote. It will probably be a
local worker that comes back with READY first.
Fundamental Changes To Structure
=============================
Queue code will be uploaded the first time an entry for a particular queue is uploaded. The
LDIP::Worker class is based off of default_handler.pl. default_handler.pl will be the default
behavior for ALL LDIP interactivity. This means that distinction will be made between
destination queues and other queues. The current destination queues will be ported to become
monolithic subhandlers. to_ldap_handler.pl for example can become the to_ldap queue's
01toLdapHandler handler. Note there will be NO HANDLER SCRIPTS in the bin/ directory of LDIP.
Bytecode caching will become /the only/ way to write code for LDIP. This means that for now
it has to be written in perl5. As perl6 becomes available, we can move to support sub handlers
written in other languages, we can also work toward a c-based worker that links against php,
python, and ruby libs. But that's further off.
One way around bytecode caching is to include your language's code in the LDIP data (LDIF file),
and compile it and execute using a simple wrapper handler.
e.g.
seqno: 404040404
uniqueid: hank
code:: cHJpbnQgIkhlbGxvIFdvcmxkISI=
02runPythonCode
...
my $code = decode_base64($entry->code);
`python -c '$code'`;
...
This is just for "Hello World", for whitespace delimited languages, its probably best to include
the whole script MIME encoded, decode it, write it to a temp file, execute it, and get it's
output. Regardless, executing any type of code that either compiles or runs on any of the worker
nodes is still possible. It also will have a similar performance profile to the "old way". APIs
may be written in these languages that allow easy integration with LDIP's API, and perhaps even
auto-generate the perl-code necessary to encode, transport, decode, and execute them in a given
worker.
The new model separates the worker from the control functionality completely. Much like the
original model (pre bytecode-caching). The hybrid model will be thrown out in favor of the "one
true way". The LDIP daemon will do no caching of any code itself nor will it run any itself. It
will check timestamps, and keep workers' cached bytecode up to date. It will send and receive data
and update local storage and the backend database accordingly.
Handlers will get simpler. Right now we have the following code at the front of almost every
handler:
use vars qw($this_handler);
use lib($ENV{LDIP_ROOT} . "/lib");
BEGIN {
use LDIP::HandlerRegistry::Handler;
$this_handler = LDIP::HandlerRegistry::Handler->new(
handler_name => 'add_krb5_attributes',
priority => '50',
);
if ($ARGV[0] eq "register") {
print $this_handler->registration;
exit();
}
}
# everything else goes here..
use LDIP::Entry;
use LDIP;
# clean up!
use strict;
my $ldp = LDIP->new();
my $file_name = $ARGV[0];
# make sure we have a file name
unless ($file_name) {
print "FAILED:no file name specified!\n";
exit();
}
# create the entry object for this piece of data.
my $entry;
# catch errors
eval {
$entry = LDIP::Entry->new($file_name);
};
if ($@) {
print "FAILED:entry creation error $@\n";
exit();
}
... then the meat and potatoes
if ($entry->exists('krb5PrincipalName')) {
# you could optionally verify that they were correct here or replace them.
print "PASSED:record " . $entry->seqno . " already appears to have krb5 attributes!\n";
exit();
}
That's 31 lines of setup for 5 lines of code.. That's too much. With the new standard
both the $entry and $ldip objects will be provided to all subhandlers, and will refer to
the entry currently being processed and the configuration of the current LDIP worker.
The new code will look like this:
use LDIP::Handler::Interface qw($entry $ldip); # will fail with setup errors if there are
# setup problems
if ($entry->exists('krb5PrincipalName')) {
# you could optionally verify that they were correct here or replace them.
print "PASSED:record " . $entry->seqno . " already appears to have krb5 attributes!\n";
exit();
}
... Much cleaner.