[Ns-developers] Questions about Packet, Buffers, and zero payloads

[Mathieu Lacage]

On Thu, 2009-06-11 at 19:12 +0000, Tom Henderson wrote:

> >After quite a bit of trying various methods, the code ended up looking  
> >like this:
> >
> >  // Serialize the packet
> >   i->buffer = p->Serialize();
> >   // Add the dest node and dest device
> >   i->buffer.AddAtStart(sizeof(node) + sizeof(dev));
> >   Buffer::Iterator it = i->buffer.Begin();
> >   it.WriteU32(node);
> >   it.WriteU32(dev);
> >
> >   // Find the system id for the destination node
> >   Ptr<Node> destNode = NodeList::GetNode(node);
> >   uint32_t nodeSysId = destNode->GetSystemId();
> >   MPI_Isend((void*)i->buffer.PeekData(), i->buffer.GetSize(),  
> >MPI_CHAR, nodeSysId,
> >             0, MPI_COMM_WORLD, &(i->request));
> >   cout << "MPI Packet sent" << endl;
> >
> >This seems to work; but one question is does the "AddAtStart" after  
> >the serialization
> >cause a buffer re-alloc and copy?  If so this is not acceptable.   
> 
> It seems to depend on whether there is space in the buffer; if you
> look at the implementation, there is an isDirty flag that might return
> true.

If the goal is to minimize the number of data copies and make sure that
you do just one buffer copy, I think that the following API would be
needed instead of the current API:

uint32_t Packet::GetSerializedSize (void) const;
// returns zero if input buffer was too small.
uint32_t Packet::Serialize (uint8_t *buffer, uint32_t size) const;
// returns zero if input buffer did not contain a complete message.
uint32_t Packet::Deserialize (uint8_t *buffer, uint32_t size);

At least, the above allows you to precisely control when that copy
happens and ensures that there is only one. You need to get hold of a tx
buffer big enough but, at least, you know when and where it is allocated
and you can implement a per-mpi-request allocation policy if you wish
to.

> >Second question is
> >that the i->buffer.GetSize() on the MPI_Isend returns the size of the  
> >buffer including
> >the zero-byte (and non-existent) payload.  Shouldn't we only send the  
> >actual packet data
> >without the zero payload?  How do I get the zero payload back on the  
> >receiving end?
> >I did not see where this is handled in Packet::Serialize() and  
> >Packet::DeSerialize().
> 
> I believe that these are presently converted to real data bytes upon
> serialization, and not restored to their previous state upon
> deserialization.  Whether we should or shouldn't do this, I think that
> it is safest to do it that way but maybe it could be optimized for MPI
> if needed.

I think that Packet::Serialize should call the currently non-existent
Buffer::Serialize which should not write the zero area content, but,
instead, which should write information about it. I was planning to work
on that after the release.

Mathieu