Zend Lucene in Symfony Projects

In this article we will introduce you with an easy way to implement the Zend Lucene Full Text Search project in Symfony 1.4 Doctrine ORM website. We will present things a little differently then the original documentation of Symfony. We will try to explain the principle of implementation of a search engine and not providing you with code to use on set. There are things that are very specific and individual to each project and it is necessary to understand the principle according to the situation. Once you have learned how to insert this functionality in the application you can put the elements like autocompleter, advanced search and etc. They are not the subject of this article. Originally you need to download Zend Framework in the following directory:


When you are testing on the localhost, it is not a problem to leave all the files on Zend Framework, but when you are uploading to the server you can leave only the following files and folders from Zend:

• Exception.php
• Loader/
• Autoloader.php
• Search/

//exams/config/ProjectConfiguration.class.php
Now we have to add these lines of code at ProjectConfiguration.class.php

static protected $zendLoaded = false;
static public function registerZend() {
   if (self::$zendLoaded) {
      return;
   }
   set_include_path(sfConfig::get('sf_lib_dir').'/vendor'.PATH_SEPARATOR.get_include_  path());
   require_once sfConfig::get('sf_lib_dir').'/vendor/Zend/Loader/Autoloader.php';
 
   Zend_Loader_Autoloader::getInstance();
   self::$zendLoaded = true;
   Zend_Search_Lucene_Analysis_Analyzer::setDefault(new Zend_Search_Lucene_Analysis_Analyzer_Common_Utf8Num_CaseInsensitive());
 
   ini_set("iconv.internal_encoding", 'UTF-8');
   Zend_Search_Lucene_Search_QueryParser::setDefaultEncoding('utf-8');
}

If you read the lesson for Zend Lucene integration in the official documentation of Symfony Framework fron here, you will notice that this code has some differences than the present example there. The differences are that we add some additional lines of code and the search works with Cyrillic letters. The next step is to choose a table from the database, which will crawl. In the example, the table is a test. In the application it is represented by two classes. It is necessary to orientate yourself to some of the things we show you because your projects’ name of the classes will be different. But once you learn the technique, it will not be a problem for you.

In testTable.class.php we enter the following function:

static public function getLuceneIndex() {
   ProjectConfiguration::registerZend();
   if (file_exists($index = self::getLuceneIndexFile())) {
      return Zend_Search_Lucene::open($index);
   } else {
      return Zend_Search_Lucene::create($index);
   }
}
static public function getLuceneIndexFile() {
   return sfConfig::get('sf_data_dir').'/exams.'.sfConfig::get('sf_environment').'.index';
}

With this function we create folder, that looks like this:

In this folder we keep the indexes of your entered data that are called to check on demand. Naturally, in this last row:

return sfConfig::get('sf_data_dir').'/exams.'.sfConfig::get('sf_environment').'.index';

You can change the folder name to whatever you want. Now it is necessary to do two more things:
The first is to change the function save() in test.class.php

public function save(Doctrine_Connection $conn = null) {
 
   // ...
 
   $ret = parent::save($conn);
   $this->updateLuceneIndex();
   return $ret;
}

and then create the following two functions in the same file:

public function delete(Doctrine_Connection $conn = null) {
   $index = testsTable::getLuceneIndex();
 
   foreach ($index->find('pk:'.$this->getId()) as $hit) {
      $index->delete($hit->id);
   }
   return parent::delete($conn);
}
 
public function updateLuceneIndex() {
   $index = testsTable::getLuceneIndex();
 
   // Премахване на съществуващият запис
   foreach ($index->find('pk:'.$this->getId()) as $hit) {
      $index->delete($hit->id);
   }
   $doc = new Zend_Search_Lucene_Document();
 
   // Съхраняваме първичният ключ на елемента за проверка при търсене
   $doc->addField(Zend_Search_Lucene_Field::Keyword('pk', $this->getId()));
 
   // Индексираме поле
   $doc->addField(Zend_Search_Lucene_Field::UnStored('name', $this->getName(), 'utf-8'));
   //$doc->addField(Zend_Search_Lucene_Field::UnStored('description', $this->getDescription(), 'utf-8'));
 
   $index->addDocument($doc);
   $index->commit();
}

In the first function has there is nothing to change. In the second, the fields:

$doc->addField(Zend_Search_Lucene_Field::UnStored('name', $this->getName(), 'utf-8'));
//$doc->addField(Zend_Search_Lucene_Field::UnStored('description', $this->getDescription(), 'utf-8'));

appear to have the value index and search displayed.
This was our the first part of implementing to our application. Now in CMD write:

And now comes to creation of the route.

In routing.yml we create the following path:

search:
url: /search
param: { module: tests, action: search }

In this case, the module which will be our search engine is the tests module. Item is search.
The controller should look like this:

public function executeSearch(sfWebRequest $request) {
   //проверяваме дали имаме post и дали този post ни е изпратил параметър query
   if ($request->isMethod('post') || $request->getParameter('query')) {
      $query = $request->getParameter('query');
      //проверяваме дали заявката за търсене е по голяма или равна на 3 символа
 
      if (strlen($query)>=3) {
         //ако е >=3 символа изпращаме запитване и извличаме резултатите в променливата query
         $this->query = Doctrine_Core::getTable('tests')->getForLuceneQuery($query);
      }
   } else {
      //ако условието за post не е изпълнено тогава извеждаме текст : няма резултат
      $this->renderText('No results');
   }
}

In SearchSuccess.php we have to develop simple code:

if (isset($query) && count($query) != 0) {
   foreach($query as $tests):
      echo $tests--->getCategories()->getName();
   endforeach;
} еlse {
   еcho “no results”;
}

In the if expression we check whether there is the variable $query and whether it contains result. Then with the foreach expression we show the result. In the example the field is only one, but in normal circumstances the variable $test contains more the one variable (name, category etc. …). Naturally with html & css we shape the results in a way we want.
//your_project/lib/model/doctrine/testTable.class.php
Now, we go back to the class tesTable.class.php and add the following function:

public function getForLuceneQuery($query) {
   $hits = self::getLuceneIndex()->find($query);
   $pks = array();
   foreach ($hits as $hit) {
      $pks[] = $hit->pk;
   }
   if (empty($pks)) {
      return array();
   }
   $q = $this->createQuery('t')
             ->where('t.active = ?', 1)
             ->whereIn('t.id', $pks)
             ->limit(3);
 
   return $q->execute();
}

You can look at your controller and see that this function is called after we have entered a search term.
In part:

$q = $this->createQuery('t')
          ->where('t.active = ?', 1)
          ->whereIn('t.id', $pks)
          ->limit(3);
return $q->execute();

the simple query that we do, is checking whether the given element in the table is active and put limit 3. Thus in the match, the search will give only 3 results. Of course you can create custom queries that
meet the requirements and needs of your application.
And the last part is the form. To have permanently visible form, we will put the code in the layout.php file.

In the form controller we point that after the “submit”, request is sent to the search controller. There they pass through function getForLuceneQuery($query) and if the result of the in query is displayed through the template searchSuccess.php.
We hope this article is useful for developing your websites. Even if you are not successful the first time, do not be disappointed. Nothing is too easy and for everything takes hard work and experience. Our advice is to check the way of implementation of Zend Lucene in the official web site of Symfony Framework at the jobeet project and learn our article too. The most important thing is do not stop testing.