Posted by robertdouglass on February 6, 2007 at 4:59pm
The title says it all. This is what we need to do now. I have no code, but J & J both do. Please share your code, compare where you are, admire each other's brilliance, and propose a scope and roadmap for the Drupal project.
Comments
CVS account already
CVS account already requested, just waiting on approval!
Please alert AjK.
Done
you're approved. Take the lead =)
I like the idea...
I like the idea of a Drupal search api allowing for plugin backends, but am a little nervous about the possible third party library dependencies. How do other Drupal projects deal with this? (ie do they branch for both Drupal-X and also for third party versions?
Since my search was just for me I didn't mind refactoring everytime Zend changes, but for an official Drupal project I'm not sure how this would work.
I even see jquery causing riffles. In their last version they deprecated a couple api methods which could cause Drupal core problems if they used them.
What are your thoughts?
-John
-John
Known risk
I was recently bitten by my 3rd party dependency (Magpie RSS, ask evaal for the painful details :\ )
The best you can do is provide the version number with which you have tested your code. All of this is open source, and the old versions don't go away. Drupal could go a long time with the jQuery version we have.
That is my biggest fear of
That is my biggest fear of going in now and not waiting for a "Zend 1.0" ... but I do not think that waiting is the answer. I want to quickly identify an easy abstraction level and then make sure that the contributed module is more at home in Drupal than with Zend.
I've been doing that by putting only the Drupal "hooks" in the module file then the abstraction layer in some include files. If we do our homework correctly the module code need only change for Drupal versions and the includes will follow the ZF release cycle. In practice the module will be more procedural and the includes more object based...
A big issue will be the location of the ZF library... right now it is at 6.4MB. For managed hosts this is not a big deal and it can be installed and configured for the PHP include paths, but users in a shared hosting environment will want a Drupal/local path. Do we pull out the dependencies from the ZF lib and make a "lite" package just for the lucene search? I would not want to do this, but I don't know how the user community will react.
Don't depend on Zend
Make the core of your code Lucene-implementation agnostic. I want to use this with the Java Lucene, others might want the other ports. The process of building queries from the user interface should stay consistent no matter what is implemented on the Lucene side. Likewise, the results returned should always be the same, no? So in terms of the Drupal module, it should know how to make queries, how to get a wrapper for the Lucene instance, and how to handle the results. Then only your wrapper is dependent on Zend. I could care less about Zend, but I see lots of common ground in the query building and the result handling.
How do you not depend on a php implementation?
Are you thinking of making a remote call using some sort of http protocol,
and having a service run on another machine?
-John
-John
The wrapper would handle it differently
For the Zend wrapper, you'd most likely call the php functions directly. For the Java wrapper, I expect the PHP/Java bridge would be a decent solution. I haven't thought ahead to a networked solution, but the approach of the Nutch servlet seems reasonable in a case like that.
ahh.
I hadn't thought of the bridge, I didn't know that it was ever fully production ready.
-John
-John
This is technically
This is technically possible, but why do that? If you have direct file access to the index binaries just use the applications native implementation. If you already have a java server running then you would be better served using it. Isn't this the point of the Open Search Client?
I'm not sure I understand you fully
If I have direct file access to the index binaries.... what do you mean? The index files built by Lucene, right? You still need Lucene to query those indexes, whether Lucene is implemented in PHP or Java. This is where the wrapper comes in, in my mind. The Drupal module gathers the query texts from the user, formats them into valid query strings for Lucene, calls an abstraction of IndexSearcher, Queryparser, and gets an abstraction of Hits. The abstractions are all Drupal/PHP code, but they are tied to a wrapper that knows how to get those objects either from Zend, or Java, or the C++ binary, or the Python library.... whatever Lucene you want to use.
It will be compatible with
It will be compatible with Java Lucene since the index files are binary compatible. But it is the ZF developers who have the ultimate responsibility to make sure that that query building and search results are consistent with what is expected from a Lucene search. The Lucene abstraction occurs at the ZF layer not in the Drupal module. So in that sense I don't think that it can be "Lucene-implementation agnostic." This module will be structurally tied to the Zend search component But that is not as bad as it sounds...
Status?
By any chance, have you been able to make progress with this project?
// David Lesieur // Associé // Whisky Echo Bravo // Développement Web, experts Drupal // Montréal //
I used a LITE version of Zend
Since we can't commit a 3rd party lib into CVS I'd say we create instructions and say copy Zend 1.x ../here and if you
wish, you only need these files.
-John
-John
I agree with this
I agree with this approach... we will need settings to store the library location in the settings.