Nutch 2.3 + Elasticsearch / results are not rendered in Kibana

Question

Nutch 2.3 + Elasticsearch / results are not rendered in Kibana

FYI, this has been crossed out on the Apache Nutch mailing list.

I'm really not sure where the issue with this issue is, whether it is Nutch, Kibana or Elasticsearch issue. I am using Nutch 2.3, HBase 0.94.14 and Elasticsearch 1.6 with Kibana 4.1.0 for crawling, archiving, and indexing.

Basically I followed the tutorial below, the only exception is the upgrade to ES 1.6 from training version 1.4 (to which I am now wondering if this is the problem).

https://gist.github.com/xrstf/b48a970098a8e76943b9

Following this tutorial, I am using the / bin / nutch script.

Most work; Nutch follows my url, HBase stores downloads, and Elasticsearch seems to be indexing content, however I can't get Kibana to render content coming from Nutch. Kibana recognizes the index and its fields, but does not display content. I have loaded the index into Kibana with and without time based events to no avail.

I have other indices and "types" on this Elasticsearch instance that Kibana can render, and I can query Elasticsearch with cURL and get nutch results just fine, I just can't get Kibana to render specific content from Nutch.

I tried two different ES + Kibana settings and was just redirecting the Nutch indexing output and I have the same problem for both. I also tried dropping the index and starting over by first creating the index and then running "nutch index -all" and trying a clean install of Elasticsearch / Kibana.

I even went as far as deploying Elasticsearch 1.4, however this requires downgrading Kibana to v3 and I am having a hard time getting this to work, but I have confirmed (again) that the content is IN IN elasticsearch via cURL.

My guess is that there is something different in the ES version, although if there was a problem with that, wouldn't it lose the transport client on insert?

Below are the logs from Kibana that don't seem to show anything interesting.

{
  "name": "Kibana",
  "hostname": "VirtualBeast",
  "pid": 6695,
  "level": 30,
  "req": {
    "method": "POST",
    "url": "\/elasticsearch\/_msearch?timeout=0&ignore_unavailable=true&preference=1434483458287",
    "headers": {
      "host": "localhost:5601",
      "connection": "keep-alive",
      "content-length": "732",
      "accept": "application\/json, text\/plain, *\/*",
      "origin": "http:\/\/localhost:5601",
      "user-agent": "Mozilla\/5.0 (X11; Linux x86_64) AppleWebKit\/537.36 (KHTML, like Gecko) Chrome\/43.0.2357.125 Safari\/537.36",
      "content-type": "application\/json;charset=UTF-8",
      "referer": "http:\/\/localhost:5601\/",
      "accept-encoding": "gzip, deflate",
      "accept-language": "en-US,en;q=0.8"
    },
    "remoteAddress": "127.0.0.1",
    "remotePort": 51632
  },
  "res": {
    "statusCode": 200,
    "responseTime": 12,
    "contentLength": 4992
  },
  "msg": "POST \/_msearch?timeout=0&ignore_unavailable=true&preference=1434483458287 200 - 12ms",
  "time": "2015-06-16T19:39:57.372Z",
  "v": 0
}