Progressively decode a message (and other questions)

Hi, I'm a Netty noob and I have some basic questions. I'm still exploring
the API so be gentle :)

Is there a way to incrementally decode a message from a ChannelBuffer? Like
this:

   1. Read some key integer fields
   2. If not interested, discard message - do not decode the rest of the
   message. Just discard it. How do I cancel further downstream codecs?


There are some other restrictions:

   - I do not want to do a getBytes(xx) on the message and make a copy of
   the remaining message from the buffer to a byte[] . I know people would
   suggest this because it's the easiest workaround by doing the decode on my
   own from a byte[]. I want to avoid copying the bytes again and again from
   layer to layer
   - If I used slice() I suppose it would help avoid a copy?
   - Are these channelbuffers pooled? How long can I hang on to the
   MessageEvent, Channel or Channelbuffer instance? Would I be starving some
   other component/connection?
      - FYI, I had asked a similar question on the Google Protobuf forum -
      http://groups.google.com/group/protobuf/brows...
   - If I want to re-route my server request from 1 server to another server
   - read some bits and then decide to "forward" the message to another server,
   can I just do a zero-copy of the original bytes and send them to another
   server? You see - like a Hibernate disconnected session or a SQL
   Disconnected row set?
   - The server now becomes a client to another server, so can I just use
      the same buffers and "transfer" them to the other handler that
is talking to
      the final server?

Confused? :)

Super clean APIs though, I'm impressed.

Regards,
Ashwin Jayaprakash.

Hi Ashwin,On 11/15/2010 04:16 AM, Ashwin Jayaprakash wrote:
> Hi, I'm a Netty noob and I have some basic questions. I'm still
> exploring the API so be gentle :)
> 
> Is there a way to incrementally decode a message from a ChannelBuffer?
> Like this:
> 
>    1. Read some key integer fields
>    2. If not interested, discard message - do not decode the rest of the
>       message. Just discard it. How do I cancel further downstream codecs?You need to maintain additional state to discard the remainder.
LengthFieldBasedFrameDecoder does exactly what you want:


http://docs.jboss.org/netty/3.2/xref/org/jbos...

You can optimize it even further by not extending FrameDecoder if you
really want, but I think it will perform just well enough.> There are some other restrictions:
> 
>     * I do not want to do a getBytes(xx) on the message and make a copy
>       of the remaining message from the buffer to a byte[] . I know
>       people would suggest this because it's the easiest workaround by
>       doing the decode on my own from a byte[]. I want to avoid copying
>       the bytes again and again from layer to layer

>     * If I used slice() I suppose it would help avoid a copy?Yes.>     * Are these channelbuffers pooled? How long can I hang on to the
>       MessageEvent, Channel or Channelbuffer instance? Would I be
>       starving some other component/connection?
>           o FYI, I had asked a similar question on the Google Protobuf
>             forum -
>             http://groups.google.com/group/protobuf/brows...ChannelBuffers are not pooled.  By default, Netty relies on JVM's GC and
it seems to scale pretty well.  However, under some constraints, you
could write a very efficient buffer pool which might beat JVM, but I'm
not sure it will be a generic enough solution, and that is why Netty
does not ship a buffer pool at the moment.  If relatively generic
solution is found, I'd love to ship it so that many applications reap
the benefit.>     * If I want to re-route my server request from 1 server to another
>       server - read some bits and then decide to "forward" the message
>       to another server, can I just do a zero-copy of the original bytes
>       and send them to another server? You see - like a Hibernate
>       disconnected session or a SQL Disconnected row set?
>           o The server now becomes a client to another server, so can I
>             just use the same buffers and "transfer" them to the other
>             handler that is talking to the final server?Reading some bits involve copying them from kernel socket buffer to user
heap, so there's no way to do a zero copy.  What you can do is always
'minimize' the number of copies, and you can avoid unnecessary memory
copy as much as you can with Netty as you write your own NIO application.

> Confused? :)

Good luck! :-)

> Super clean APIs though, I'm impressed.

Thanks.  Any suggestion for improvements?

Thanks for the reply.

FYI, check this out when you have the time - http://jfs.des.udc.es/ and
http://torusware.com/

Ashwin.-- 
View this message in context: http://netty-forums-and-mailing-lists.685743....
Sent from the Netty User Group mailing list archive at Nabble.com.

Hi Trustin,>http://docs.jboss.org/netty/3.2/xref/org/jbos...

>You can optimize it even further by not extending FrameDecoder if you
>really want, but I think it will perform just well enough.How and why can the LengthFieldBasedFrameDecoder perform better by not
extending FrameDecoder? Just remove the "extends FrameDecoder" part
and put the decode logic in a MessageReceived method?On Sat, Jan 22, 2011 at 1:15 PM, AshwinJay  wrote:
>
> Thanks for the reply.
>
> FYI, check this out when you have the time - http://jfs.des.udc.es/ and
> http://torusware.com/
>
> Ashwin.
>
> --
> View this message in context: http://netty-forums-and-mailing-lists.685743....
> Sent from the Netty User Group mailing list archive at Nabble.com.
> _______________________________________________
> netty-users mailing list
> 
> https://lists.jboss.org/mailman/listinfo/nett...
>-- 
Timmy