Barrier with compare?
I've got N Udops processing data.
The all spit out their best guess.
I'm trying to synchronize their outputs via Barrier:
Example:
stream clusterResultsSync(schemaFor(clusterResultsObj)) := Barrier ( clusterResults1; clusterResults2 ) []
{
}
I don't see a good way to make a UDOP either as the number of operators is adjustable making the schema and coding difficult.
Is something like the following possible?
stream clusterResultsSync(schemaFor(clusterResultsObj)) := Barrier ( clusterResults1; clusterResults2 ) []
{
if $2.x < $1.x $1.x = $2.x
if $2.y < $1.y $1.y = $2.y
// or maybe this? $1.x = min(clusterResults1.x,clusterResults2.x); $1.y = min(clusterResults1.y,clusterResults2.y);
}
What I really want to is something like
Message was edited by: kjerick - Missing end code tag in first code snippet.
Hi Mr. Black,
I am not sure what happened to your post after "What I really want to is something like" but there was nothing there.
To answer your question, if you look in the "Programming Model and Language Reference" for the Barrier operator, the output section does not allow any expressions, it only has selectors. The statement syntax you are using won't work either, even in output sections of operators that allow expressions. For your application, you can use a Barrier, but send that stream to a Functor to do the "math".
I came up with a working example application (I called it qt for quick test) in qt.dps using yours as a base and expanding from there:
[Application] qt debug [Program] vstream clusterIntegerObj( x : Integer, y : Integer ) vstream clusterSyncObj( x1 : Integer, y1 : Integer, x2 : Integer, y2 : Integer ) stream clusterResults1(schemaFor(clusterIntegerObj)) := Source()[
"file:///CartesianIn1.dat", csvFormat, nodelays]
{
} stream clusterResults2(schemaFor(clusterIntegerObj)) := Source()[
"file:///CartesianIn2.dat", csvFormat, nodelays]
{
} stream clusterSyncResult(schemaFor(clusterSyncObj)) := Barrier ( clusterResults1; clusterResults2 ) []
{ $1.x, $1.y, $2.x, $2.y
} stream clusterCompareResults(schemaFor(clusterIntegerObj)) := Functor(clusterSyncResult) []
{ COND(x2 < x1, x1, x2), COND(y2 < y1, y1, y2)
} Nil := Sink(clusterCompareResults)[
"file:///CartesianOut.dat", csvFormat, nodelays]
{
}
Hope this helps.
Best regards,
Kevin
Thanks a bunch..this idea worked wonderfully in my more complex problem.
I was thinking it could be done in the Barrier but the Functor works fine.
Bad part is making this dynmamic is going to make for some ugly loops due to the syntax I think.
Is there a prettier way to do this? Same goes for defining the other objects too.
stream clusterResultsSync(schemaFor(clusterResultsObj)) := Barrier ( for_begin @L 1 to N-1 clusterResults@L; for_end for_begin @L N to N clusterResults@L for_end ) []
{
}
Hi Mr. Black,
I would like to say there was something prettier, but I think what you have coded is what you need to do.
Best regards,
Kevin
Michael, if you use a dmm file rather than a dps file then you could use some embedded perl to write the loop for you (see the documentation for Mixed Mode SPADE). So I think your example could be coded something like:
stream clusterResultsSync(schemaFor(clusterResultsObj)) := Barrier ( <% my @a = ();
for(my $i=1;$i<=$n;$i++)
{ push @a, clusterResults.$i
} %> for_begin @L 1 to N-1 clusterResults@L; for_end for_begin @L N to N clusterResults@L for_end ) []
{
}
Michael, if you use a dmm file rather than a dps file then you could use some embedded perl to write the loop for you (see the documentation for Mixed Mode SPADE). So I think your example could be coded something like:
stream clusterResultsSync(schemaFor(clusterResultsObj)) := Barrier ( <% my @a = ();
for(my $i=1;$i<=$n;$i++)
{ push @a, clusterResults.$i.
" ";
} print join(
";",@a); %> ) []
{
}
You'll have to define $n somewhere, maybe something like:
<% my $n = 10; %>
"Pretty" is in the eye of the beholder. Opinion on the beauty of Perl varies widely, but I like this solution. Thanks for your contribution Andy.
Best regards,
Kevin