[PLUG] LAN broadcast?

[Tim]

Hi Wayne,

> I have a question about user applications using broadcast on a LAN.
> 
> I usually have several processes that connect via TCP/IP to a central 
> server that solves some optimization problems. In more demanding project 
> that I am working on now, I hope to distribute the optimization problem 
> between 24 to 36 helper processes.
> 
> At the start of each optimization job, I will have a few data files that 
> need to be distributed to each of the helper optimization processes. 
> Rather than have the central optimizer / server send these files 24 
> times, some way to broadcast these files to all machines / processes at 
> the same time might be nice.
> 
> Is that the kind of thing that broadcasting could accomplish?
> 
> And if so, and in a big picture way, how would one do it?


You could use broadcast, yes.  Most likely you would want to use UDP
for this, since TCP handshakes and broadcast doesn't really mix.  To
accomplish this, you would simply have each helper process listen for
the broadcast messages and then send results back to the central
server at the server's unicast address.  

However, a "sexier" way of accomplishing the same thing would be to
use multicast.  This would allow you to distribute the broadcast
messages scalably across router boundaries.  Multicast is really
powerful for specific applications like this and I hope it catches on
some day, since it can completely replace distribution models like
bittorrent.

Finally, instead of implementing all of this distributed
computation/communication yourself, you might consider using an MPI
library or even building a BOINC application.  Or maybe you don't need
to go distributed at all for the scale of your project by instead
leveraging the massively parallel processor you probably have in your
system (a.k.a. your video card).  See OpenCL.

HTH,
tim