Posted by drunken monkey on March 26, 2008 at 4:47pm
I am planning to hand in a proposal on improving the Apache Solr Search Integration module.
The project would include:
- Porting the module to drupal 6 (if necessary)
- Integration in Views 2, enabling the use of Views 2 as a front-end to display the search results
- Writing simpletest unit tests for this module, especially for the new functionality
What's your opinion on that? I have already contacted Robert Douglas to ask for his.
Comments
Yay! I'd be happy to mentor such a project.
Could you elaborate on this? Would this be replicating views_fastsearch in a way?
Good monkey!
interesting!
I already thought about something related: using views together with an sparql query builder - this would be awesome for a semantic enabled drupal. Of course this is scifi, but a first step would be to make it possible to use views and it's style plugins with arbitrary search result sets. If I understand right, this is was drunkenmonkey would need. I don't think it's possible yet, but I think merlinofchaos mentioned something like that in barcelona. I don't know if he still plans that for views2.
views_fast_search directly access the search_index table I think? So this won't help much for displaying solr results.
You're right...
...that's basically what I'd need. I'm already in contact with merlinofchaos about it and hope, this functionality will be available when I really start with the project. Otherwise, of course, I would have to add it to the project as a prerequisite to achieve the rest.
Thanks a lot!
I'm not sure, but don't think so: views_fastsearch uses the existing drupal search index, whereas the solr index is of course not integrated in that. So while fastsearch shares views' backend, I would have to completely detach the query part of views and just use its frontend presentational functionality.
ApacheSolr as a views filter
It should be possible to execute the view to get a set of nodes and pass the whole set of nodes as a filter to Solr. I'm not sure if this will perform well in the case that there are thousands or hundreds of thousands of nodes, but it would at least work, an could be implemented as a views filter. Anyway, the goal is great, and the SoC proposal can honestly say "Research a solution and build a prototype for what seems the best candidate". Pure R&D is fine, imo.
I'd love to see the Apache
I'd love to see the Apache Solr Search Integration tied in with Views2 -- it would be a great combination.
Like others, I'd love to see some details.
And the fact that a drunken monkey would work on it? HUGE bonus :)
Cheers,
Bill
FunnyMonkey
Tools for Teachers
FunnyMonkey
I believe that using views
I believe that using views to display results would be a great thing. This would make it really simple for someone who knows how to theme a view to then theme the results that their search engine puts out. I actually just submitted a proposal for a search API. I think that using a core search api to handle this views integration would be a good route to go. That means that other engines could then utilize this without even writing any extra code. It also means that instead of solr representing its results to views, the core api would take results from solr and then build the view from the results and output to display. This would ensure that any other search engine module that wanted to interface with this, could easily do so by implementing the proper hooks/callbacks from the core search api.
I like your ideas and hope we can work together in creating some kick ass search improvements.
Also, I forgot to mention,
Also, I forgot to mention, the Faceted Search module does integration with views to display results with a helper module that comes with the package. It may be worth a look to see how David did his implementation.
http://drupal.org/project/faceted_search
How it's done in Faceted Search
At the moment Faceted Search creates a temporary table with the search results from the user's request, and then a Views handler uses nids from that table to filter the view. The temporary table actually existed even before Views integration was added, because all the information that's displayed in the Guided search derives from that temporary table.
// David Lesieur // Associé // Whisky Echo Bravo // Développement Web, experts Drupal // Montréal //
Thanks for the comments!
Thanks for all the encouraging comments! Especially to Blake, your project and the whole new search API sound really promising. I'll be sure to watch it for ways to help me with my project - or maybe even ways my project could help you.
Thanks also for the information about Faceted Search, that could also prove very useful for me.
As for some details, well, I admit that I am not really sure on an exact course of action. Just the different approaches mentioned on this page would keep me thinking/testing until long after the deadline. Therefore I intend to follow Robert's suggestion of making this primarily an R&D project - with the abundance of ideas here, in my head and those yet to come, I'm sure that there will be satisfying results.
But, as said, I don't really know what they might look like, there are just too much possibilities.
Don't tell anyone, but the essentials of my plan include a cloning machine and an infinite amount of typewriters. ;D
Views as exclusive frontend not a great idea
While I'd love to make a vfs clone (or addition) using Solr, I am worried about getting too node-centric about search. Many sites utilize panels, og, etc for building pages which are entities in their own right, and it is already a problem that they are not getting searched properly by drupal. I suspect this is one reason people move to the ugly googlesearch implementation.
I think we have to strike a balance between the advanced functionality / flexibility of nodes and still preserve a way to get a great search of all content. I like how the Mini does it, where the meta-tags are in the page. This is a bit clunky, but provides all pages the same presence in the index.
Best,
Jacob
not node centric any more.
Views is no longer node centric in D6. It can take any base table, including non-Drupal-native ones for queries now.
ApacheSolr is node centric at the moment
The ApacheSolr module is node centric. It would be easy enough to write other search implementations for it (the underlying Solr technology doesn't care), but the current implementation is a core-search inspired, node-centric search.
this is all I need..
wow..finally i found this board. This is what I'm looking to aid my context-based searching which is still in core search using the views fastsearch module. Certainly Apachesolr - Views integration is a big thing for apachesolr. I'll be waiting for this release than creating mine -- i considered myself beginner compared to you guys. Is there any release so far?
No release yet. Thomas is
No release yet. Thomas is working on it.
hmmm is the an Apachesolr -- Views integration project underway
We were thinking of writing out own.
More info please?
Thanks.
Thomas is working on
Thomas is working on ApacheSolr/Views integration as part of his Summer of Code project. He's currently in the R&D phase, I believe, and he is required to be finished by the end of the summer. You are welcome to write your own, of course. If it is business critical for you, that's what you should do. I think Thomas will produce good, working code by the end of summer, though, so if you can wait that long, you have that option, too.
left him an offer for help.
Just left him a note if we @ worthpoint.com can assist. Sophisticated module I must say Robert.
On the core module we've come up with a wish list:
- working with a Solr cluster
- generating the Solr scheme updates for cck fields
--> would there be any interest for rolling these into the apache_solr module?
I guess we're struggling with some of the issues he's looking into
Arman.
Yes, great feature list
I'd be happy to help anyone who is looking to implement these features in any way I can.
Robert we @ Worthpoint.com interested in implemeting these
Can you help me how we can proceed -- I'll get the clearance from
management that we can contribute the code back -- I'm certain
they will be thrilled.
Thanks,
Arman.
Lucene
Hi All,
I wan to know what is the status of the lucene-drupal integration for Drupal6
I have this scenario:
- A Java engine application that generate the lucene index and insert the Drupal nodes in the drupal mysql database.
- I want to Drupal6 to search in this index.
Thanks in advance and sorry if the message isn't clear or other issues 'cause is my first one :)
Matias Basilico
http://www.mbasilico.com.ar
Matias Basilico
Blog: http://www.mbasilico.com.ar
Twitter: http//twitter/mbasilico
How about Views 1?
It's just now I realized that the site I co-maintained is using Views 1 and this one in development uses Views 2. Any idea how to deal with this? Thanks :)
For reference, the issues on
For reference, the issues on drupal.org are:
http://drupal.org/node/293841 "Allow different back-ends for Views"
http://drupal.org/node/254565 "Integrate with Views front-end"
The patch in 254565 contains the patch found in 293841