[Yanel-dev] Elasticsearch REST client [WAS: Re: Upgrading Lucene

[Michael Wechner]

Hi

I finally managed to get an elasticsearch client connection running,
whereas the Java code is as follows:

+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.Response;
+import org.apache.http.HttpHost;
+import org.apache.http.Header;
+import org.apache.http.message.BasicHeader;

+                    // TODO: Make sure to configure certificate inside
truststore, see http://www.yanel.org/en/documentation/security/ssl.html
+                    String host = "MY_HOST_ID.us-east-1.aws.found.io";
+                    int port = 9243;
+                    String protocol = "https";
+                    RestClient restClient = RestClient.builder(new
HttpHost(host, port, protocol)).build();
+                    String username = "elastic";
+                    String password = "MY_PASSWORD";
+                    Header header = new BasicHeader("Authorization",
getBasicAuthHeaderValue(username, password));
+
+                    log.debug("Access Elasticsearch ...");
+                    Response response =
restClient.performRequest("GET",
"/yulup/_search?q=repository-url:'hugo'", header);
+                    //Response response =
restClient.performRequest("GET",
"/yulup/_search?q=repository-url:'https\\:\\/\\/github.com\\/wyona\\/yanel'");
+                    restClient.close();
+                } catch(Exception e) {
+                    log.error(e, e);
+                }

+    private String getBasicAuthHeaderValue(String username, String
password) {
+        String auth = username + ":" + password;
+        byte[] encodedAuth =
org.apache.commons.codec.binary.Base64.encodeBase64(auth.getBytes(java.nio.charset.Charset.forName("ISO-8859-1")));
+        return "Basic " + new String(encodedAuth);
+    }

whereas re SSL one has to configure a truststore, whereas first one
needs to get the SSL certificate:


openssl s_client -showcerts -connect
MY_HOST_ID.us-east-1.aws.found.io:9243 </dev/null 2>/dev/null|openssl
x509 -outform PEM >elasticsearch.pem

and then create the truststore file

keytool -importcert -v -alias elasticsearch -file elasticsearch.pem
-storepass SOME_PASSWORD -keystore truststore.jks

and then configure this according to

http://www.yanel.org/en/documentation/security/ssl.html

If I find the time, then I will make this available inside Yanel itself.

Hope that helps

Michael

Am 15.08.17 um 17:35 schrieb Michael Wechner:
> Hi Balz
>
> Thanks very much for your feedback!
>
> I just found an alternative Elasticsearch client, which has no Lucene
> dependencies
>
> https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/_initialization.html
>
> but I think we should upgrade Lucene anyway and your suggestions below
> make a lot of sense!
>
> Thanks very much
>
> Michael
>
> Am 15.08.17 um 17:02 schrieb basZero:
>> Hurray!!!
>> I proposed the same thing in 2013:
>> https://github.com/wyona/yarep/issues/6
>>
>> It would be great to see Lucene 6 in yanel.
>>
>> Regarding source code repository of yanel/yarep etc., here my thought:
>> - I would create a new branch "lucene2.9.2" in all repositories of
>> the current latest status. This way, people who want to work with
>> this latest version and lucene 2.9.2 can still get the code very easily.
>> - The master branch would then be upgraded to use Lucene 6.
>>
>> Some other remarks:
>> - Lucene 6 runs on Java 1.8 or later. So make sure all old
>> dependencies are upgraded as well! (in all repositories like yarep,
>> security etc.)
>> - The index of 2.9.2 won't be compatible, Lucene only guarantees
>> compatibility within the same major release (e.g. 6.0.0 and 6.6.0
>> will have the same index format).
>>
>> The upgrade must be done carefully, because I currently don't see
>> what libraries Lucene is depending on (e.g. Log4j, apache commons
>> stuff...), I assume that many libs currently in yanel (which are out
>> of date) will have to be upgraded as well. Example: yanel uses log4j
>> 2.0 beta8 ... Lucene 6 will most probably not use a beta version :-)
>> so an upgrade to log4j 2.5 is most probably required.
>>
>>
>>
>> Cheers, Balz
>>
>> On Tue, Aug 15, 2017 at 4:45 PM, Michael Wechner
>> <michael.wechner at wyona.com> wrote:
>>
>>     Hi Together
>>
>>     I have recently tried to integrate Elasticsearch into Yanel, but have
>>     trouble doing this, because Elasticsearch requires a more recent
>>     Lucene
>>     version than Yarep is using (2.9.2).
>>
>>
>>     Lucene already offers version 6.6.0
>>     (https://lucene.apache.org/core/downloads.html
>>     <https://lucene.apache.org/core/downloads.html>) and I think it
>>     would be
>>     good to upgrade Lucene finally. I am not sure whether search
>>     indices are
>>     still compatible, but one could re-generate a search index.
>>
>>     What do you think? Any objections?
>>
>>     Thanks
>>
>>     Michael
>>
>>
>>     --
>>     Yanel-development mailing list Yanel-development at wyona.com
>>     http://mx2.wyona.com/cgi-bin/mailman/listinfo/yanel-development
>>     <http://mx2.wyona.com/cgi-bin/mailman/listinfo/yanel-development>
>>
>>
>>
>>
>
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mx2.wyona.com/pipermail/yanel-development/attachments/20170818/9205e7c2/attachment.html>