Node-Angular and Nosql App Part 2

We are adding full text searching functionality to our application.This tutorial assumes that you already completed part 1.Please refer to here for part 1.

Unfortunately,Couchbase does not have full text search feature as of version 2.2.0. So,it is advised to integrate Couchbase with elasticsearch to provide this.

Elastic search is a search analysis tool and a key value store which is getting more and more popular.

Here are the steps:

    1. Install Elasticsearch version 0.90.5 since this is the one compatible with Couchbase 2.2.0.Installation available here.
    2. Install Couchbase Elasticsearch plugin version(assuming you have Couchbase 2.2.0 installed) 1.2.0 following instructions here.Be careful with the version numbers since it does not work but with compatible version.Below is the version compatibility matrix:couchbaselabs-elasticsearch-transport-couchbase · GitHub
    3. Verify that you can see administration console of elasticsearch upon starting on http://localhost:9200/_plugin/head/ .
    4. Install the Couchbase template using  curl -XPUT http://localhost:9200/_template/couchbase -d @plugins/transport-couchbase/couchbase_template.json
    5. Create an index guardian on elasticsearch admin console http://localhost:9200/_plugin/head/
    6. elasticsearch-head
    7. Configure Couchbase to replicate data to elasticsearch:
      1. Navigate to the Couchbase Server admin interface.
      2. Select the Replications tab.
      3. Press the button labeled “Create Cluster Reference”
      4. Choose a name for your ElasticSearch cluster
      5. In the IP/Hostname and field provide an address and port of one of the nodes in your ES cluster (127.0.0.1:9091)
      6. Enter the Username and Password corresponding to your “couchbase.username” and “couchbase.password” settings in ElasticSearch. Refer to bullet no 2 for this.
      7. Press the “Save” buttonCouchbase Console (2.2.0)
      8. Press the button labeled “Create Replication”
      9. Select “guardian” bucket from source cluster
      10. Next select the cluster you defined in previous step
      11. Click advanced settings and set XDCR Protocol to version 1.
      12. Press the button labeled “Replicate”CouchbaseReplicate
      13. Verify on elasticsearch admin console that replication took place.
        elasticsearch-data
    8. Now that you completed the installation and configuration,it is time to run the code and try it out. Go to github and download the code available at https://github.com/selmantayyar/guardian-news-content/tree/version2 . This is the same github repository as the part one but on a different branch.To avoid installing couchnode again,just override your current local code copy with this.If this is the first time you are running this code,follow the instructions on readme file.Otherwise just type node app.js and enjoy it.Guardian Newspaper Football Content
    9. But how does the searching work?Application does a REST call to elastic search server sending the keyword.A very simple post request,which makes elastic search really easy to use.By the way,a very nice introduction to elasticsearch is here.
   $scope.searchNews = function () {
 console.log('scope keyword: '+$scope.form.keyword);
 var queryStr={
 'query': {
 'query_string': {
 'query': $scope.form.keyword,
 'fields': ["trailText"]
 }
 }
 };
 $http.post('http://localhost:9200/guardian/_search', queryStr).
 success(function(data) {
 console.log(data);
 searchResultService.setSearchresult(data);
 $location.url('/readNews');
 });
 }; 

Results & Analysis

To sum up, noticing that Couchbase(as of version 2.2.0) does not have any full text search feature is a bit disappointing.You need to install elasticsearch,replicate the data to it and handle all the data transfer issues,indexing issues(updating data sometimes could require indexes and mappings to be updated ..etc) and other potential issues.This is an additional operational burden. Couchbase is really good for storing data and retrieving it by key or by some other indexes.It is also easier to manage replicas.However, i still would expect full text search functionality here.

Advertisements

Node-Angular and Nosql App

I have created a simple web application using Node.js(Expres) on server-side.DB is Couchbase and using client as Angular.js

Basically,it is displaying data stored on couchbase using different views(queries if we speak in relational db terms).Data comes from The Guardian Open Data Platform via another app that i wrote.It is bringing various articles from Guardian website using the API that is served.By the way,i highly recommend anyone to have a look at The Guardian Open Data Platform.

Guardian Newspaper Football Content

It is using express web application framework for server side operations.REST services are defined and connecting to and querying couchbase is implemented inside.Getting and querying data from a nosql db -including map-reduce functions- is showcased here.

<br /><%%KEEPWHITESPACE%%>  var db = new couchbase.Connection( config, function( err ) {<br /><%%KEEPWHITESPACE%%>    if(err) {<br /><%%KEEPWHITESPACE%%>      console.error("Failed to connect to cluster: " + err);<br /><%%KEEPWHITESPACE%%>      process.exit(1);<br /><%%KEEPWHITESPACE%%>    }<br /><br /><%%KEEPWHITESPACE%%>    console.log('Couchbase Connected');<br /><%%KEEPWHITESPACE%%>  });<br /><br /><%%KEEPWHITESPACE%%>  var params = { limit:1000, reduce:false};<br /><%%KEEPWHITESPACE%%>  params.key =category;<br /><%%KEEPWHITESPACE%%>    db.view( "dev_news", "news_by_section", params).query(function(err, results) {<br /><br /><%%KEEPWHITESPACE%%>      res.json({<br /><%%KEEPWHITESPACE%%>        posts: results<br /><%%KEEPWHITESPACE%%>      });<br /><%%KEEPWHITESPACE%%>    });<br /><br />

On client-side Angular.js is used as MVC javascript framework.For templating, Jade is in place.

All in all,two things to install and run as well as source code of course available in Github:

  1. Data populator using The Guardian Open Data platform:https://github.com/selmantayyar/GuardianRestClient
  2. Actual web application:https://github.com/selmantayyar/guardian-news-content

README files have clear instructions for installation.

Please let me know if you face any issues.

Next step is to do content base search using ElasticSearch.