6093690339_a09493f126_b1

Contents


Overview

Skill Level: Beginner

Intermediate

Ingredients

Eclipse IDE

Java

Bluemix Account

Step-by-step

  1. Create a Bluemix Application

    Actually, this step is described in the tutorial “Bluemix Watson APIs Quickstart using Java SDK” (https://developer.ibm.com/recipes/tutorials/bluemix-watson-apis-quickstart-using-java-sdk/ )

    The only difference is that, instead of adding the service “Speech to Text”, we’re going to add the “Alchemy API” service.

  2. Retrieving News Data using Alchemy API

    Let’s suppose we’re interested in the Office Depot stocks.

    ""

    We can see two main events here. First, the stocks are slowly going down, when, suddenly, stocks go up quickly on March 17th.

    Maybe Watson can tell us a little bit about it.

     CredentialUtils.setServices(FileUtils.readFileToString(new File("/home/leoks/git/newsmood.1/vcap.json")));
    AlchemyDataNews service = new AlchemyDataNews();
    service.setApiKey(CredentialUtils.getAPIKey("alchemy_api"));

    Map<String, Object> params = new HashMap<String, Object>();

    String[] fields =
    new String[] {"enriched.url.title", "enriched.url.url", "enriched.url.author",
    "enriched.url.publicationDate", "enriched.url.enrichedTitle.entities",
    "enriched.url.enrichedTitle.docSentiment"};
    params.put(AlchemyDataNews.RETURN, StringUtils.join(fields, ","));
    params.put(AlchemyDataNews.START, "1450267200");//mar 1st 2016
    params.put(AlchemyDataNews.END, "1458129600");//mar 10th 2016
    params.put("q.enriched.url.enrichedTitle.entities.entity","|text=Office Depot,type=company|");
    params.put("q.enriched.url.enrichedTitle.taxonomy.taxonomy_.label","business and industrial");
    DocumentsResult result = service.getNewsDocuments(params);

    System.out.println(result);

     

    Notice that the contents of VCAP_SERVICES was saved in the “vcap.json” local file.

    Alchemy API uses seconds from 1970 for dates. This is how we get the numbers in Java

     Calendar d = new GregorianCalendar();
    d.set(Calendar.YEAR, 2016);
    d.set(Calendar.MONTH, Calendar.MARCH);
    d.set(Calendar.DAY_OF_MONTH, 16);
    d.set(Calendar.HOUR, 0);
    d.set(Calendar.MINUTE, 0);
    d.set(Calendar.SECOND, 0);
    d.setTimeZone(TimeZone.getTimeZone("GMT-12"));
    System.out.println(d.getTimeInMillis() / 1000);

     

    The results are printed in the JSON format. Because Alchemy API access is limited by day, it’s always a good idea to cache the results.

     (...)

    {
    "id": "NDU1MzQ0NTU5NnwxNDU4MDg2NDAw",
    "source": {
    "enriched": {
    "url": {
    "author": "Marcia Heroux Pounds",
    "enrichedTitle": {
    "entities": [
    {
    "count": 1,
    "disambiguated": {},
    "knowledgeGraph": {
    "typeHierarchy": "/organizations/agencies/ftc"
    },
    "quotations": [],
    "relevance": 0.330000013,
    "sentiment": {
    "mixed": "0",
    "score": 0.0,
    "type": "neutral"
    },
    "text": "FTC",
    "type": "Organization"
    },
    {
    "count": 1,
    "disambiguated": {
    "dbpedia": "http://dbpedia.org/resource/Office_Depot",
    "geo": "",
    "name": "Office Depot",
    "subType": [],
    "website": "http://www.officedepot.com"
    },
    "knowledgeGraph": {
    "typeHierarchy": "/organizations/businesses/retailers/office depot"
    },
    "quotations": [],
    "relevance": 0.330000013,
    "sentiment": {
    "mixed": "0",
    "score": 0.206799001,
    "type": "positive"
    },
    "text": "Office Depot",
    "type": "Company"
    }
    ],
    "docSentiment": {
    "mixed": "0",
    "score": 0.0,
    "type": "neutral"
    }
    },
    "publicationDate": {
    "date": "20160316T000000",
    "confident": "true"
    },
    "title": "FTC: Staples, Office Depot must spin off operational company to get merger OKu0027d",
    "url": "http://www.sun-sentinel.com/business/fl-staples-office-depot-ftc-pre-trial-20160316-story.html"
    }

    (...)

     

    One important thing here is the publication date. When “confident” is “true”, it means that Anchemy API is sure about the document publication date. Otherwise, a timestamp is used (indicating the date when the document was added to Alchemy data news repository). So we’re going to filter all the entries where “confident” is “false”.

     

  3. Data Analysis

    Now we can iterate on the results and check if we have any relevant information.

     SimpleDateFormat sdf = new SimpleDateFormat("yyyyMMdd");
    Date start = sdf.parse("20151215");
    Date end = sdf.parse("20160317");
    Gson gson = new GsonBuilder().create();
    DocumentsResult res = gson.fromJson(new FileReader("/home/leoks/git/newsmood.1/office.json"), DocumentsResult.class);
    for(int i=0;i < res.getDocuments().getDocuments().size();i++){
    Document doc = res.getDocuments().getDocuments().get(i);
    Article article = doc.getSource().getEnriched().getArticle();
    if (article.getPublicationDate().getConfident()){
    for(int j=0;j Entity entity = article.getEnrichedTitle().getEntities().get(j);
    if (
    entity.getText().contains("Office Depot")
    && article.getPublicationDate().getDate().before(end)
    && article.getPublicationDate().getDate().after(start)){

    System.out.println(article.getTitle());
    System.out.println(article.getPublicationDate().getDate());

    System.out.println(entity.getSentiment().getType());
    System.out.println(entity.getSentiment().getScore());
    }
    }
    System.out.println();
    }
    }

    which brings (results may vary)

    FTC: Staples, Office Depot must spin off operational company to get merger OK'd
    Wed Mar 16 00:00:00 BRT 2016
    POSITIVE
    0.206799001

    Office Depot : Survey Finds Less Than 50 Percent of Shoppers Shred Paper during Tax Season | 4-Traders
    Tue Mar 15 00:00:00 BRT 2016
    NEGATIVE
    -0.530317008

    Office Depot : Best of Our Blogs: Shopping center sold, Related project draws critics | 4-Traders
    Mon Mar 14 00:00:00 BRT 2016
    NEUTRAL
    0.0

    Staples Bets on Rebranding as an Office Depot Merger Seems Increasingly Unlikely
    Tue Mar 15 00:00:00 BRT 2016
    NEGATIVE
    -0.721242011

    See how much Office Depot in Royal Palm Beach fetched | Real Time
    Tue Mar 15 00:00:00 BRT 2016
    NEUTRAL
    0.0

    Office Depot, Inc. Survey Finds Less Than 50 Percent of Shoppers Shred Paper during Tax Season | Business Wire
    Tue Mar 15 00:00:00 BRT 2016
    NEUTRAL
    0.0

    See how much Office Depot in Royal Palm Beach fetched | Real Time
    Tue Mar 15 00:00:00 BRT 2016
    NEUTRAL
    0.0

    Staples calls FTC 'incredibly narrow' on Office Depot acquisition
    Fri Mar 04 00:00:00 BRT 2016
    NEUTRAL
    0.0

    Office Depot Inc (ODP) Stake Raised by Neuberger Berman Group LLC
    Fri Mar 11 00:00:00 BRT 2016
    NEUTRAL
    0.0

    So here’s what we have here

    • March 04th: 1 NEUTRAL
    • March 11th: 1 NEUTRAL
    • March 14th: 1 NEUTRAL
    • March 15th: 2 NEGATIVE, 3 NEUTRAL
    • March 16th : 1 POSITIVE

6 Comments on "Using Watson as a Cognitive Stock Market Advisor"

  1. Hi Leo, first of all, thanks for sharing. Have you tried to backtest the results to see if the algorithm would actually be profitable?

    • I am afraid it’s not that simple 🙂 but I have a feeling that Alchemy API tried to use the merge and acquisition information from the news for this purpose. Not really a real information here, just a feeling, from what I’ve saw from the API parameters.

  2. Hi Leo, Thank you for the article. I established a Start Up her in Israel, we focus on Text mining for mid range Stocks trade. This system does not let me sebd you an email. can you please send me your address to amnon_e@oshyot.co.il ?
    sincerely,
    amnon

  3. Hi Leo, I am a little new to all this so sorry if my questions may actually be very trivial. I followed you other tutorial on using API’s with SDK at this link: https://developer.ibm.com/recipes/tutorials/bluemix-watson-apis-quickstart-using-java-sdk-version-2016/#r_overview

    However, when trying to go from that one to this (using alchemyAPI), I am having trouble understanding where exactly you input the code to implement the API in Eclipse. I am trying to make a simple act like the one above that can read multiple news sources and look for their effects (+ or – like above) on cheap stock volatility, more so based on keywords in the articles. How would I first go about doing that in the most simple way?

    • Hi. In these example, all the code runs inside the main() method of an arbitrary class (I am not sure if this is exactly what you want to know)

Join The Discussion