|
FAST Protocol
< Previous Next >
Re: Fast SCP
Jean-Marie Sulmont / RTS <>
1 Aug 2008 1:34AM ETThank you David -- this is what I thought. If this is acceptable for receiver, then I see indeed no reasons to bother with virtual synchrony etc.
> When the decoder sees a tid it hasn't seen before, it must resolve that
> tid into a template name and further resolve that template name into the
> actual template definition before it can continue.
>
> The exact steps involved in the resolution are dependent on how the
> particular service is setup. In a pure multicast scenario I guess you'd
> have a "template beacon" periodically scrolling through the current set
> of templates. Then you'd have to suspend your main decoder until the
> template decoder sees the requested template definition pass by if it
> hasn't seen it already.
>
> In other setups you might have a TCP back channel or FTP or web site
> where you can request new templates (think this was what Darshan
> recommended).
>
> /David
>
> > Cheers for this.
> >
> > As long as there is no compatibility between template version (which
> > boils down to Hanno's question), then I don't understand how what you
> > describe works.
> >
> > Say template foo, version associated with tid=7 and version 2
> > associated with tid=27, as per your example. If I receive a message
> > with tid=27 and I have not build the corresponding codec, then I'm
> > doomed. Unless I know that 27 the id map to the next incarnation for
> > template_id=7
> >
> >
> >
> > > I was maybe a bit terse.
> > >
> > > By using immutable template ids (in the short term) you can relax
> > > the requirements on synchrony between template definitions and
> > > messages using template ids. As a receiver, you still need a new
> > > template definition before you can decode a message with the new
> > > template id, but the existing definitions don't change (within a
> > > time window).
> > >
> > > This allows for a lower overhead template distribution solution,
> > > which is what Dimitry was asking about, I think.
> > >
> > > An example; Let's assume you have a template named "foo/1" where 1
> > > is the version of the template. Further let's assume the Template
> > > Name "foo/1" is associated with the Id 7. Now, if you want to modify
> > > this template (extend or change), you can create "foo/2" which
> > > differs in some aspect from "foo/1" and, depending on other Ids in
> > > use your first unused Id is 27, so you associate the name "foo/2"
> > > with the id 27.
> > >
> > > Now receivers may see different representations of a message type.
> > > Template id 7 will still refer to the old layout and id 27 will be
> > > unknown until receivers see / retrieve the definition for id 27. The
> > > important difference is that you can relax strict ordering as id 7
> > > is immutable for some period of time before it may be reused. Not
> > > seeing the new definition of id 27 can be detected.
> >
> > > You still need a level of synchronization between templates and
> > > messages in order to consume messages with newly created templates,
> > > but you don't need a strict ordering; new definitions can be sent
> > > out earlier as there is no (short-term) change to the meaning of an
> > > existing template.
> > >
> > > There needs to be additional synchronization if several sources are
> > > allowed to allocate template ids, but that is a separate issue.
> > >
> > > /Rolf
> > >
> > > >> The remaining problem is then to make sure that a receiver
> > > >> recognizes and acts on template id rollover. Implementors need to
> > > >> make sure that senders receivers agree on the current meaning of
> > > >> a template id (ie. the current association between a template
> > > >> name and a template id). It is still an instance of the reliable
> > > >> messaging problem, but with drastically lower latency
> > > >> requirements.
> > > >
> > > > That's precisely the hard part: IMHO this is more than an instance
> > > > of the Reliable messaging problem -- because it involves some
> > > > ordering considerations. Given a sender and a possibly large
> > > > number of receivers: no receiver should deliver a message prior to
> > > > delivering its related template.
> > > > e.g., ... [T_i] [M_i_1] ... [M_i_j] ... [T_(i+1)] [M_(i+1)_1] ....
> > > >
> > > > Not sure that's an instance of the distributed consensus, but
> > > > rather instance of the virtual synchrony problem. That is,
> > > > something (SCP) should guarantee that template changes within a
> > > > group of receiver, are observed in the same order by all
> > > > receivers. Furthermore, template changes in SCP must be totally
> > > > ordered with respect to all regular FAST messages.
> > > >
> > > > BTW there is a typo in Fast SCP draft 1.1: page 15 section 7.3
> > > > third item one should read "...defined in 6.1" and not "defined in
> > > > 5.1"
> > > >
> > > > I couldn't find mention of this in the SCP spec.
Fast SCP Dimitry London / Morgan Stanley 30 Jul 2008 10:16PM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 31 Jul 2008 3:28AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 31 Jul 2008 6:17AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 31 Jul 2008 9:05AM ET Re: Fast SCP Hanno Klein / Deutsche Börse Systems 31 Jul 2008 10:06AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 31 Jul 2008 10:15AM ET Re: Fast SCP Greg Orsini / Cameron Systems 31 Jul 2008 11:44AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 2:06AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 2:39AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 3:13AM ET Re: Fast SCP Hanno Klein / Deutsche Börse Systems 1 Aug 2008 3:33AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 3:57AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 4:10AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 3:06AM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 1 Aug 2008 6:28AM ET Re: Fast SCP Jacob Northey / The LaSalle Technology Group 1 Aug 2008 8:21AM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 1 Aug 2008 9:33AM ET Re: Fast SCP Greg Orsini / Cameron Systems 1 Aug 2008 10:01AM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 1 Aug 2008 10:16AM ET Re: Fast SCP Greg Orsini / Orc Software w/CameronFIX 1 Aug 2008 10:32AM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 1 Aug 2008 10:58AM ET Re: Fast SCP Jean-Marie Sulmont / RTS 31 Jul 2008 10:19AM ET Re: Fast SCP Jean-Marie Sulmont / RTS 31 Jul 2008 10:41AM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 31 Jul 2008 11:10AM ET Re: Fast SCP
Jean-Marie Sulmont / RTS 1 Aug 2008 1:34AM ET Re: Fast SCP Dimitry London / Morgan Stanley 7 Aug 2008 10:53PM ET
|
