Digital Developer Conference: a FREE half-day online conference focused on AI & Cloud – North America: Nov 2 – India: Nov 9 – Europe: Nov 14 – Asia Nov 23 Register now

Close outline
  • United States
IBM?
  • Site map
IBM?
  • Marketplace

  • Close
    Search
  • Sign in
    • Sign in
    • Register
  • IBM Navigation
IBM Developer Answers
  • Spaces
    • Blockchain
    • IBM Cloud platform
    • Internet of Things
    • Predictive Analytics
    • Watson
    • See all spaces
  • Tags
  • Users
  • Badges
  • FAQ
  • Help
Close

Name

Community

  • Learn
  • Develop
  • Connect

Discover IBM

  • ConnectMarketplace
  • Products
  • Services
  • Industries
  • Careers
  • Partners
  • Support
10.190.13.195

Refine your search by using the following advanced search options.

Criteria Usage
Questions with keyword1 or keyword2 keyword1 keyword2
Questions with a mandatory word, e.g. keyword2 keyword1 +keyword2
Questions excluding a word, e.g. keyword2 keyword1 -keyword2
Questions with keyword(s) and a specific tag keyword1 [tag1]
Questions with keyword(s) and either of two or more specific tags keyword1 [tag1] [tag2]
To search for all posts by a user or all posts with a specific tag, start typing and choose from the suggestion list. Do not use a plus or minus sign with a tag, e.g., +[tag1].
  • Ask a question

Barrier with compare?

270002CGRE gravatar image
Question by MichaelBlack  (1) | Mar 30, 2011 at 06:27 PM streamsdev

I've got N Udops processing data.
The all spit out their best guess.
I'm trying to synchronize their outputs via Barrier:
Example:


stream clusterResultsSync(schemaFor(clusterResultsObj)) := Barrier ( clusterResults1; clusterResults2 ) [] 
{ 
}

I don't see a good way to make a UDOP either as the number of operators is adjustable making the schema and coding difficult.
Is something like the following possible?


stream clusterResultsSync(schemaFor(clusterResultsObj)) := Barrier ( clusterResults1; clusterResults2 ) [] 
{ 

if $2.x < $1.x $1.x = $2.x 

if $2.y < $1.y $1.y = $2.y 
// or maybe this? $1.x = min(clusterResults1.x,clusterResults2.x); $1.y = min(clusterResults1.y,clusterResults2.y); 
}


What I really want to is something like

Message was edited by: kjerick - Missing end code tag in first code snippet.

People who like this

  0
Comment
10 |3000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster

6 answers

  • Sort: 
060001Q1QN gravatar image

Answer by kjerick (31) | Mar 31, 2011 at 12:59 AM

Hi Mr. Black,

I am not sure what happened to your post after "What I really want to is something like" but there was nothing there.

To answer your question, if you look in the "Programming Model and Language Reference" for the Barrier operator, the output section does not allow any expressions, it only has selectors. The statement syntax you are using won't work either, even in output sections of operators that allow expressions. For your application, you can use a Barrier, but send that stream to a Functor to do the "math".

I came up with a working example application (I called it qt for quick test) in qt.dps using yours as a base and expanding from there:


[Application] qt debug   [Program]   vstream clusterIntegerObj( x : Integer, y : Integer ) vstream clusterSyncObj( x1 : Integer, y1 : Integer, x2 : Integer, y2 : Integer )   stream clusterResults1(schemaFor(clusterIntegerObj)) := Source()[
"file:///CartesianIn1.dat", csvFormat, nodelays] 
{ 
}   stream clusterResults2(schemaFor(clusterIntegerObj)) := Source()[
"file:///CartesianIn2.dat", csvFormat, nodelays] 
{ 
}   stream clusterSyncResult(schemaFor(clusterSyncObj)) := Barrier ( clusterResults1; clusterResults2 ) [] 
{ $1.x, $1.y, $2.x, $2.y 
}   stream clusterCompareResults(schemaFor(clusterIntegerObj)) := Functor(clusterSyncResult) [] 
{ COND(x2 < x1, x1, x2), COND(y2 < y1, y1, y2) 
}   Nil := Sink(clusterCompareResults)[
"file:///CartesianOut.dat", csvFormat, nodelays] 
{ 
}


Hope this helps.

Best regards,
Kevin

Comment

People who like this

  0   Share
10 |3000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
270002CGRE gravatar image

Answer by MichaelBlack (1) | Mar 31, 2011 at 09:17 AM

Thanks a bunch..this idea worked wonderfully in my more complex problem.
I was thinking it could be done in the Barrier but the Functor works fine.

Bad part is making this dynmamic is going to make for some ugly loops due to the syntax I think.
Is there a prettier way to do this? Same goes for defining the other objects too.


stream clusterResultsSync(schemaFor(clusterResultsObj)) := Barrier ( for_begin @L 1 to N-1 clusterResults@L; for_end for_begin @L N to N clusterResults@L for_end ) [] 
{ 
}

Comment

People who like this

  0   Share
10 |3000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
060001Q1QN gravatar image

Answer by kjerick (31) | Mar 31, 2011 at 02:58 PM

Hi Mr. Black,

I would like to say there was something prettier, but I think what you have coded is what you need to do.

Best regards,
Kevin

Comment

People who like this

  0   Share
10 |3000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
27000197V5 gravatar image

Answer by RockClimb (1) | Mar 31, 2011 at 04:40 PM

Michael, if you use a dmm file rather than a dps file then you could use some embedded perl to write the loop for you (see the documentation for Mixed Mode SPADE). So I think your example could be coded something like:


stream clusterResultsSync(schemaFor(clusterResultsObj)) := Barrier ( <% my @a = (); 

for(my $i=1;$i<=$n;$i++) 
{ push @a, clusterResults.$i 
} %> for_begin @L 1 to N-1 clusterResults@L; for_end for_begin @L N to N clusterResults@L for_end ) [] 
{  
}

Comment

People who like this

  0   Share
10 |3000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
27000197V5 gravatar image

Answer by RockClimb (1) | Mar 31, 2011 at 04:44 PM

Michael, if you use a dmm file rather than a dps file then you could use some embedded perl to write the loop for you (see the documentation for Mixed Mode SPADE). So I think your example could be coded something like:


stream clusterResultsSync(schemaFor(clusterResultsObj)) := Barrier ( <% my @a = (); 

for(my $i=1;$i<=$n;$i++) 
{ push @a, clusterResults.$i.
" "; 
} print join(
";",@a); %> ) [] 
{  
}


You'll have to define $n somewhere, maybe something like:


<% my $n = 10; %>

Comment

People who like this

  0   Share
10 |3000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
060001Q1QN gravatar image

Answer by kjerick (31) | Mar 31, 2011 at 05:23 PM

"Pretty" is in the eye of the beholder. Opinion on the beauty of Perl varies widely, but I like this solution. Thanks for your contribution Andy.

Best regards,
Kevin

Comment

People who like this

  0   Share
10 |3000 characters needed characters left characters exceeded
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster

Follow this question

No one has followed this question yet.

Answers

Answers & comments

Related questions

CGI ?, or PDF .. 1 Answer

newbie ? - instance started, but AAS,SRM not started and SWS not configured 1 Answer

extract data from record which has no delimiters 3 Answers

UDOP ByteList troubles 5 Answers

Use of Sort operator 0 Answers

  • Contact
  • Privacy
  • IBM Developer Terms of use
  • Accessibility
  • Report Abuse
  • Cookie Preferences

Powered by AnswerHub

Authentication check. Please ignore.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • API Connect
  • Analytic Hybrid Cloud Core
  • Application Performance Management
  • Appsecdev
  • BPM
  • Blockchain
  • Business Transaction Intelligence
  • CAPI
  • CAPI SNAP
  • CICS
  • Cloud Analytics
  • Cloud Automation
  • Cloud Object Storage
  • Cloud marketplace
  • Collaboration
  • Content Services (ECM)
  • Continuous Testing
  • Courses
  • Customer Experience Analytics
  • DB2 LUW
  • Data and AI
  • DataPower
  • Decision Optimization
  • DevOps Build
  • DevOps Services
  • Developers IBM MX
  • Digital Commerce
  • Digital Experience
  • Finance
  • Global Entrepreneur Program
  • Hadoop
  • Hybrid Cloud Core
  • Hyper Protect
  • IBM Cloud platform
  • IBM Design
  • IBM Forms Experience Builder
  • IBM Maximo Developer
  • IBM StoredIQ
  • IBM StoredIQ-Cartridges
  • IIDR
  • ITOA
  • InformationServer
  • Integration Bus
  • Internet of Things
  • Kenexa
  • Linux on Power
  • LinuxONE
  • MDM
  • Mainframe
  • Messaging
  • Node.js
  • ODM
  • Open
  • PartnerWorld Developer Support
  • PowerAI
  • PowerVC
  • Predictive Analytics
  • Product Insights
  • PureData for Analytics
  • Push
  • QRadar App Development
  • Run Book Automation
  • Search Insights
  • Security Core
  • Storage
  • Storage Core
  • Streamsdev
  • Supply Chain Business Network
  • Supply Chain Insights
  • Swift
  • UBX Capture
  • Universal Behavior Exchange
  • UrbanCode
  • WASdev
  • WSRR
  • Watson
  • Watson Campaign Automation
  • Watson Content Hub
  • Watson Marketing Insights
  • dW Answers Help
  • dW Premium
  • developerWorks Sandbox
  • developerWorks Team
  • Watson Health
  • More
  • Tags
  • Questions
  • Users
  • Badges