We are hoping to use SOLR in a couple of non-standard implementations, and I just have a few questions.
-
If we want to index all of the documents in a specific file directory (e.g. TIFF images of scanned documents), can we do this directly with SOLR, or do we need Nutch? (I realize TIFFS are quite legacy but in this instance conversion to PDF is not an option.)
-
If there are specific documents on a remote site we want to index with SOLR (again, specific TIFF documents), what is the best way to accomplish this. (We have the specific URLs for these documents.)
-
If we have a domain which includes approximately 50 sub-domains each running independent web sites, would we index these with Nutch, or is there a way to do this directly within SOLR?
-
If we index data from a 3rd party database (e.g. a DB2 database) with SOLR, can these searches be made available in Drupal?
-
If we index external sites such as those mentioned in #3, can we provide links to their pages through Drupal?
My grandfather always said "It's better to let people think you're a fool than to open your mouth and remove all doubt." Today, I had to open my mouth. Thank you for your patience with these newbie questions and for any guidance you can provide!
Anthony
Comments
Hey agatlin, If we want to
Hey agatlin,
If we want to index all of the documents in a specific file directory (e.g. TIFF images of scanned documents), can we do this directly with SOLR, or do we need Nutch? (I realize TIFFS are quite legacy but in this instance conversion to PDF is not an option.)
You can do this directly with Solr I'm slightly confused as I thought TIFF's where an image format where is the other metadata that would go with this??
If there are specific documents on a remote site we want to index with SOLR (again, specific TIFF documents), what is the best way to accomplish this. (We have the specific URLs for these documents.)
You can tackle it 2 ways if you have a complete list of urls you can write a simple bash/python/php scipt to grab all of the files down and repeat question 1. If the files are being updated on a regular basis you could use nutch to grab the documents but are they discoverable i.e. through crawling could they be discovered or would you have to update a list as new files are added?
If we have a domain which includes approximately 50 sub-domains each running independent web sites, would we index these with Nutch, or is there a way to do this directly within SOLR?
Yep this is a job for Nutch
If we index data from a 3rd party database (e.g. a DB2 database) with SOLR, can these searches be made available in Drupal?
There are some new developments I am doing in this are that will make this possible for a client, its in alpha and we can talk through how we could make this available if of interest. Otherwise not currently
If we index external sites such as those mentioned in #3, can we provide links to their pages through Drupal?
yes using the Nutch module (which isnt production ready) this is possible. I am its current maintainer so we can make it work for your use case
My grandfather always said "It's better to let people think you're a fool than to open your mouth and remove all doubt." Today, I had to open my mouth. Thank you for your patience with these newbie questions and for any guidance you can provide!
I will counter with the only bad question is the one you didn't ask!
Hi
How is it possible to get apache solr to add special directories to the search results in drupal? Is there a way to get Tika to run through any files setup just as links in a node? Or would it be easier to write an alternative to Solr_attachments for special file locations?