ZF-2779: chmod in Zend_Search_Lucene_Storage_Directory_Filesystem causes problems

Description

[reference issue ZF-1429] Only file owners can chmod files. Since the apache user is usually a restricted user, with no shell, it's difficult to fire off cron jobs to update indexes out of band. This means that typically another user should do this. But the other user needs to take over ownership of the index files, and in that case, apache can no longer read the indexes and search breaks.

I don't see a problem with ZSL checking to see if the files are writable because that level of permission can be assigned to a group (apache user + indexer user). In that case, search only needs to check if the files are readable, no?

I haven't seen any reason for ZSL to chmod files (am I missing something?), and doing so breaks good OS admin practices.

Comments

File privileges are changed only at a file creation time. That guarantees we have current user as file owner at chmod() call time.

Right privileges for new files can be specified using Zend_Search_Lucene_Storage_Directory_Filesystem::setDefaultFilePermissions($mode) call (http://framework.zend.com/manual/en/…).

This method should be used if someone wants to override default index files permissions (e.g. give read access only for owner). That may be necessary for some cases.

Probably I don't see where the problem is. Please describe it in more details.

[copied comment from ZF-1429]

Please reconsider your thoughts on forcing the use of chmod.

In restrictive environments it is completely valid that a non-owner of the lucene index files has proper write permissions to write but still is not owner. And, as you're ware, chmod can only be issued by the owner of the files. Group ownership is not considered.

I would strongly advice to remove this patch in its entirety. Zend_Search_Lucene should not deal with permission in anyway itself. This should be left to the environment it is running in, and especially under Unix systems this is easily done by making sure the proper umask is set.

If this is not an option, please at least consider the ability to disable the call to chmod().

Here is an example scenario of mine: The index is created from a non www-data user. When I do this and access the index from within the web, I get

Error: chmod(): Operation not permitted in includes/Zend/Search/Lucene/Storage/Directory/Filesystem.php on line 189

0 includes/Zend/Search/Lucene/LockManager.php(85): Zend_Search_Lucene_Storage_Directory_Filesystem->createFile('read.lock.file')

1 includes/Zend/Search/Lucene.php(410): Zend_Search_Lucene_LockManager::obtainReadLock(Object(Zend_Search_Lucene_Storage_Directory_Filesystem))

2 includes/App/Search.php(252): Zend_Search_Lucene->__construct('/var/www/netcar...')

3 http/suche/index2.php(59): App_Search->search('milbe', 0)

4 {main}

However, there's actually no technical reason why a chmod() call is needed here. The permissions are completely fine as they're and it should not concern the web application that it is not the owner of some of the files. Still the write permissions to create the file are there.

thanks

Aha! I see.

The problem is in the several files used for index access synchronization. These files may be recreated (also by index 'readers'). All other index files are not updated during their lifetime, so chmod() is applied only to just created files and shouldn't make a problem in the case you described.

Reasons not to use chmod() at all are clear. But I am not sure we have other ways to provide right (user defined) permissions for index files. umask is applied to all files created by application and non-index files permissions may differ from what we need for index files. In addition to this umask() usage may cause problems under multithreaded environment (http://www.php.net/manual/en/function.umask.php) :(

I see only one use case for applying different permissions to index files now. It's giving or restricting index read/write access for everyone... but, I think, we need these possibilities.

Thanks for your reply, good to see this issue hasn't fell of the radar ;)

What if setting up proper permissions is just left out completely of Z_S_L? I mean in the whole Java Lucene library there isn't a single bit of permission handling code. Why do we need it in PHP?

I didn't mean that Z_S_L should take care of the umask from within PHP but the environment under which Z_S_L is operating should do so. There's probably a real-life problem with my suggestion I don't see (and why this change from ZF-1429 has been introduced) but I can't see this limits.

thanks

Please categorize/fix as needed.

Got the same issue stated at response #2 (Markus Fischer - 05/Mar/08 05:07 PM). Another user created Lucene index but web search form query prints :

exception 'Zend_Search_Lucene_Exception' with message 'chmod() [function.chmod]: Operation not permitted' in /home/fabrice/library/Zend/Search/Lucene/Storage/Directory/Filesystem.php:189 Stack trace:

0 /home/fabrice/library/Zend/Search/Lucene/LockManager.php(85): Zend_Search_Lucene_Storage_Directory_Filesystem->createFile('read.lock.file')

1 /home/fabrice/library/Zend/Search/Lucene.php(410): Zend_Search_Lucene_LockManager::obtainReadLock(Object(Zend_Search_Lucene_Storage_Directory_Filesystem))

2 /home/fabrice/library/Zend/Search/Lucene.php(186): Zend_Search_Lucene->__construct('/home/fabrice/p...', false)

3 /home/fabrice/public_html/fabrice-tests/minisite/application/default/controllers/RechercheController.php(34): Zend_Search_Lucene::open('/home/fabrice/p...')

4 /home/fabrice/library/Zend/Controller/Action.php(502): RechercheController->indexAction()

5 /home/fabrice/library/Zend/Controller/Dispatcher/Standard.php(293): Zend_Controller_Action->dispatch('indexAction')

6 /home/fabrice/library/Zend/Controller/Front.php(914): Zend_Controller_Dispatcher_Standard->dispatch(Object(Zend_Controller_Request_Http), Object(Zend_Controller_Response_Http))

7 /home/fabrice/public_html/fabrice-tests/minisite/html/index.php(101): Zend_Controller_Front->dispatch()

8 {main}

Tristan Celder sent me this patch, thanks him !

Our project is glad to use Zend_Search_Lucene: http://code.google.com/p/qubit-toolkit/

In our case, it is an exception if we cannot read / write to one of the index files, but if we can read / write to the file, it is not an exception if we cannot chmod() it.

Since normally all the files in our project have the correct permissions, though they may be owned by a user other than Apache, I think it would solve our problem if, in the case that chmod() fails, Zend_Search_Lucene only threw an exception if the file permissions did not already match $_defaultFilePermissions

Fixed.

Thenks for all for really productive discussion around the problem!

Updating for the 1.6.0 release.

The current solution still gives us some headaches because in the development context of our application even PHP notices are converted into exceptions (with good reason) and if the app is used via CLI and web, we've got a problem.

So, why not add a check for the file owner? See the second patch for a version which works for us.