|
FAST Protocol
< Previous Next >
Re: Fast SCP
David Rosenborg / Pantor Engineering AB
31 Jul 2008 11:10AM ETWhen the decoder sees a tid it hasn't seen before, it must resolve that tid into a template name and further resolve that template name into the actual template definition before it can continue.
The exact steps involved in the resolution are dependent on how the particular service is setup. In a pure multicast scenario I guess you'd have a "template beacon" periodically scrolling through the current set of templates. Then you'd have to suspend your main decoder until the template decoder sees the requested template definition pass by if it hasn't seen it already.
In other setups you might have a TCP back channel or FTP or web site where you can request new templates (think this was what Darshan recommended).
/David
> Cheers for this.
>
> As long as there is no compatibility between template version (which
> boils down to Hanno's question), then I don't understand how what you
> describe works.
>
> Say template foo, version associated with tid=7 and version 2 associated
> with tid=27, as per your example. If I receive a message with tid=27 and
> I have not build the corresponding codec, then I'm doomed. Unless I know
> that 27 the id map to the next incarnation for template_id=7
>
>
>
> > I was maybe a bit terse.
> >
> > By using immutable template ids (in the short term) you can relax the
> > requirements on synchrony between template definitions and messages
> > using template ids. As a receiver, you still need a new template
> > definition before you can decode a message with the new template id,
> > but the existing definitions don't change (within a time window).
> >
> > This allows for a lower overhead template distribution solution, which
> > is what Dimitry was asking about, I think.
> >
> > An example; Let's assume you have a template named "foo/1" where 1 is
> > the version of the template. Further let's assume the Template Name
> > "foo/1" is associated with the Id 7. Now, if you want to modify this
> > template (extend or change), you can create "foo/2" which differs in
> > some aspect from "foo/1" and, depending on other Ids in use your first
> > unused Id is 27, so you associate the name "foo/2" with the id 27.
> >
> > Now receivers may see different representations of a message type.
> > Template id 7 will still refer to the old layout and id 27 will be
> > unknown until receivers see / retrieve the definition for id 27. The
> > important difference is that you can relax strict ordering as id 7 is
> > immutable for some period of time before it may be reused. Not seeing
> > the new definition of id 27 can be detected.
>
> > You still need a level of synchronization between templates and
> > messages in order to consume messages with newly created templates,
> > but you don't need a strict ordering; new definitions can be sent out
> > earlier as there is no (short-term) change to the meaning of an
> > existing template.
> >
> > There needs to be additional synchronization if several sources are
> > allowed to allocate template ids, but that is a separate issue.
> >
> > /Rolf
> >
> > >> The remaining problem is then to make sure that a receiver
> > >> recognizes and acts on template id rollover. Implementors need to
> > >> make sure that senders receivers agree on the current meaning of a
> > >> template id (ie. the current association between a template name
> > >> and a template id). It is still an instance of the reliable
> > >> messaging problem, but with drastically lower latency requirements.
> > >
> > > That's precisely the hard part: IMHO this is more than an instance
> > > of the Reliable messaging problem -- because it involves some
> > > ordering considerations. Given a sender and a possibly large number
> > > of receivers: no receiver should deliver a message prior to
> > > delivering its related template.
> > > e.g., ... [T_i] [M_i_1] ... [M_i_j] ... [T_(i+1)] [M_(i+1)_1] ....
> > >
> > > Not sure that's an instance of the distributed consensus, but rather
> > > instance of the virtual synchrony problem. That is, something (SCP)
> > > should guarantee that template changes within a group of receiver,
> > > are observed in the same order by all receivers. Furthermore,
> > > template changes in SCP must be totally ordered with respect to all
> > > regular FAST messages.
> > >
> > > BTW there is a typo in Fast SCP draft 1.1: page 15 section 7.3 third
> > > item one should read "...defined in 6.1" and not "defined in 5.1"
> > >
> > > I couldn't find mention of this in the SCP spec.
Fast SCP Dimitry London / Morgan Stanley 30 Jul 2008 10:16PM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 31 Jul 2008 3:28AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 31 Jul 2008 6:17AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 31 Jul 2008 9:05AM ET Re: Fast SCP Hanno Klein / Deutsche Börse Systems 31 Jul 2008 10:06AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 31 Jul 2008 10:15AM ET Re: Fast SCP Greg Orsini / Cameron Systems 31 Jul 2008 11:44AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 2:06AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 2:39AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 3:13AM ET Re: Fast SCP Hanno Klein / Deutsche Börse Systems 1 Aug 2008 3:33AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 3:57AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 4:10AM ET Re: Fast SCP Rolf Andersson / Pantor Engineering 1 Aug 2008 3:06AM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 1 Aug 2008 6:28AM ET Re: Fast SCP Jacob Northey / The LaSalle Technology Group 1 Aug 2008 8:21AM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 1 Aug 2008 9:33AM ET Re: Fast SCP Greg Orsini / Cameron Systems 1 Aug 2008 10:01AM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 1 Aug 2008 10:16AM ET Re: Fast SCP Greg Orsini / Orc Software w/CameronFIX 1 Aug 2008 10:32AM ET Re: Fast SCP David Rosenborg / Pantor Engineering AB 1 Aug 2008 10:58AM ET Re: Fast SCP Jean-Marie Sulmont / RTS 31 Jul 2008 10:19AM ET Re: Fast SCP Jean-Marie Sulmont / RTS 31 Jul 2008 10:41AM ET Re: Fast SCP
David Rosenborg / Pantor Engineering AB 31 Jul 2008 11:10AM ET Re: Fast SCP Dimitry London / Morgan Stanley 7 Aug 2008 10:53PM ET
|
