Creating a Search Engine using Lucene - PART 2 : Sample search engine in ASP.NET MVC applic...

Datetime:2016-08-23 02:11:36          Topic: Lucene  ASP.NET MVC           Share

Article Content

  1. Creating a SEARCH ENGINE using Lucene.NET - PART 1 : Understanding Lucene.
  2. Creating a SEARCH ENGINE using Lucene - PART 2 :  Sample search engine in ASP.NET MVC application using Lucene.NET

Introduction

After we explained the main functionnality of Lucene and how the indexing and searching process are done.

Now we will view those functionnalies through an example done with c# Asp.net MVC.

Background

This article may be useful for intermediate developers who have a some basic in C#, ASPNET MVC, HTML, CSS3, Jquery, Ajax.

Using the code

A. Installation

First, To use Lucene project you must add Lucene.NET in your project references (you can use Manage NuGet Packages).

B. Indexing

The indexing part, is the step where the application stores index related to the data uploaded, to can easly fount in the searching process.

The example adopted in our application is : the user must enter a image file, writing some description related to the image, and click on indexing button.

Then the application will check if the index Directory exist, and create indexWriter with the specification of the nature of analyser.

after there will be the build of document that stores the following fields :

  • first field take the name of file,
  • second field store the description.

1. Implementation

  • Code JavaScript
//JQuery code that detect the click on searching button
   $("#formSearch").on("submit", function (e) {
       e.preventDefault();
       //url of service
       var textToSearch = $("#textToSearch").val();
       var url = "/Home/Searching";
      //send request to the mentioned service in url variable
       $.ajax({
           type: "POST",//choose your HTTP request type ('POST' or 'GET')
           url: url,
           data: { 'textToSearch': textToSearch },
           success: function (data) {
               //display data (HTML+CSS) into html bloc
               $("#idOutputSearch").html(data);
           },
           error: function (xhr, ajaxOptions, thrownError) {
               //display error message  into html bloc
               $("#idOutputSearch").html("Echec Loading");
           }
       });
 });
  • Code Html
  1  <form action="/Home/Indexing" method="post" enctype="multipart/form-data" role="form" class="form-horizontal">
  2     <div class="form-group">
  3           <label class="col-lg-3">Image File :</label>
  4            <div class="input-group col-lg-offset-3">
  5                 <input type="file" name="filePicture" placeholder="Add a description">
  6             </div>
  7       </div>
  8       <div class="form-group">
  9              <label class="col-lg-3">Description :</label>
 10              <div class="input-group col-lg-offset-3">
 11                   <input type="text" name="description" placeholder="Add a description">
 12               </div>
 13       </div>
 14       <div class="form-group">
 15              <div class="col-lg-offset-3">
 16                    <input type="submit" class="btn-default  btn-sm" value="Go Indexing" />
 17               </div>
 18        </div>
 19   </form>
  • Code C# Asp.net MVC
  1  public ActionResult Indexing()
  2         {
  3             string description = Request.Params["description"];
  4             HttpPostedFileBase filePicture = Request.Files["filePicture"];
  5             if (description != null && filePicture != null)
  6             {
  7                 //create file path
  8                 string fileName = filePicture.FileName;
  9                 string pathFile = Server.MapPath("~/Content/images/");
 10                 string pathImage = System.IO.Path.Combine(pathFile, fileName);
 11                 //saving image to server
 12                 filePicture.SaveAs(pathImage);
 13                 //Create Index :
 14                 string pathIndexFolder = Server.MapPath("~/Content/indexDirectory");
 15                 Directory directory = FSDirectory.Open(pathIndexFolder);
 16                 //Create standardAnalyser
 17                 //with StandardAnalyzer : Most sophisticated analyzer that knows about certain token types, lowercases, removes stop words, ...
 18                 //WhitespaceAnalyzer :Splits tokens on whitespace
 19  
 20                 Analyzer analyser = new Lucene.Net.Analysis.WhitespaceAnalyzer();
 21                 bool create = !IndexReader.IndexExists(directory);
 22                 //initialize indexWriter
 23                 IndexWriter indexWriter = new IndexWriter(directory, analyser, create, IndexWriter.MaxFieldLength.UNLIMITED);
 24                 //intialize document
 25                 Document doc = new Document();
 26                 //Create fiels
 27                 Field field1 = new Field("name", fileName, Field.Store.YES, Field.Index.ANALYZED);
 28                 Field field2 = new Field("description", description, Field.Store.YES, Field.Index.ANALYZED);
 29  
 30                 // remove older index entry
 31                 var searchQuery = new TermQuery(new Term("name", fileName));
 32                 indexWriter.DeleteDocuments(searchQuery);
 33                 //add new index entry
 34                 //add field to document
 35                 doc.Add(field1);
 36                 doc.Add(field2);
 37                 //add document to index Directory
 38                 indexWriter.AddDocument(doc);
 39                 //free indexWriter
 40                 indexWriter.Dispose();
 41             }
 42             return View("Index");
 43         }

2. Result

C. Searching Part

The searching part, is the step where the application start the searching process to find the suitable indexed data that match with the searched text from indexed document stored into specific Index Directory.

In our example, when user click on searching button after writing a text. the application create a Query statement based on the nature of analyser (must take the some analyzer as the indexing step) and with specification of field key. after the searching process will start to fetch between stored documents based on search text.

If the result are not null, the application build html data that take information from documents fields.

1. Implementation

  • Code Html
  1  <form enctype="multipart/form-data" id="formSearch" role="form" class="form-horizontal">
  2       <div class="form-group">
  3            <label class="col-lg-3">Write a text :</label>
  4            <div class="input-group col-lg-offset-3">
  5               <input type="text" name="textToSearch" id="textToSearch" placeholder="write a term"/>
  6              </div>
  7         </div>
  8         <div class="form-group">
  9               <div class="col-lg-offset-3">
 10                     <input type="submit" class="btn-default btn-sm" value="find" />
 11                </div>
 12          </div>
 13  </form>
 14   <div role="form" class="form-horizontal">
 15       <div class="form-group">
 16          <label class="col-lg-12">Search Result</label>               
 17          <div id="idOutputSearch">
 18              <!--Result of searching-->
 19          </div>
 20        </div> 
 21    </div>
  • Code C# Asp.net MVC
  1  public ActionResult Indexing()
  2         {
  3             string description = Request.Params["description"];
  4             HttpPostedFileBase filePicture = Request.Files["filePicture"];
  5             if (description != null && filePicture != null)
  6             {
  7                 //create file path
  8                 string fileName = filePicture.FileName;
  9                 string pathFile = Server.MapPath("~/Content/images/");
 10                 string pathImage = System.IO.Path.Combine(pathFile, fileName);
 11                 //saving image to server
 12                 filePicture.SaveAs(pathImage);
 13                 //Create Index :
 14                 string pathIndexFolder = Server.MapPath("~/Content/indexDirectory");
 15                 Directory directory = FSDirectory.Open(pathIndexFolder);
 16                 //Create standardAnalyser
 17                 //with StandardAnalyzer : Most sophisticated analyzer that knows about certain token types, lowercases, removes stop words, ...
 18                 //WhitespaceAnalyzer :Splits tokens on whitespace
 19  
 20                 Analyzer analyser = new Lucene.Net.Analysis.WhitespaceAnalyzer();
 21                 bool create = !IndexReader.IndexExists(directory);
 22                 //initialize indexWriter
 23                 IndexWriter indexWriter = new IndexWriter(directory, analyser, create, IndexWriter.MaxFieldLength.UNLIMITED);
 24                 //intialize document
 25                 Document doc = new Document();
 26                 //Create fiels
 27                 Field field1 = new Field("name", fileName, Field.Store.YES, Field.Index.ANALYZED);
 28                 Field field2 = new Field("description", description, Field.Store.YES, Field.Index.ANALYZED);
 29  
 30                 // remove older index entry
 31                 var searchQuery = new TermQuery(new Term("name", fileName));
 32                 indexWriter.DeleteDocuments(searchQuery);
 33                 //add new index entry
 34                 //add field to document
 35                 doc.Add(field1);
 36                 doc.Add(field2);
 37                 //add document to index Directory
 38                 indexWriter.AddDocument(doc);
 39                 //free indexWriter
 40                 indexWriter.Dispose();
 41             }
 42             return View("Index");
 43         }

2. Result

C). Reset Indexing

In this step, when clicking on Reset indexing button, the index directory will be empty of documents.

1. Implementation

  • Code JavaScript
  1  //detect the click  on Reset index button
  2   $("#idResetIndex").on("click", function () {
  3        //invoke the Delete Indexes service
  4        $.post("Home/DeleteIndexes");
  5   });
  • Code Html
  1  <input type="button" id="idResetIndex" class="btn-default btn-lg" value="Reset Index Directory" />
  • Code C# Asp.net MVC
  1  public ActionResult DeleteIndexes()
  2    {
  3        string pathIndexFolder = Server.MapPath("~/Content/indexDirectory");
  4        Directory directory = FSDirectory.Open(pathIndexFolder);
  5        var reader = IndexReader.Open(directory, false);
  6        var docs = new List<document>();
  7        //get all documents inside the index Directory
  8         var term = reader.TermDocs();
  9         //Fetch all available documents 
 10         while (term.Next())
 11          {
 12            //removing document from index Directory
 13            reader.DeleteDocument(term.Doc);
 14          }
 15          //free reader
 16          reader.Dispose();
 17          return View();
 18    }
 19  </document>

In closing

i hope that you appreciate this article. try to download the source code and i'm waiting for your feedbacks.





About List