How to search 30,000 SQL records
I'm going to do a simple object search on my site where a user can enter about 2-4 keywords that they will get when they search in two columns of a table in my MS SQL database. One column is varchar (50) called name and one column is varchar (2500) called description. At any given time about 20,000-30,000 records will be available for search.
The keywords will have to return the "best matches" - you know the kind you get on search pages like ebay that return the closest matches. The way I thought of doing this seems kind of naive - I thought I could read all 30,000 table records in an object and like this:
public class SearchableObject
{
string Title {get; set;}
string Description {get; set;}
int MatchedWords {get; set;}
}
Then create a list of that List object for example go through all 30,000 records, populate the list, figure out the ones that match most of the time and go back to the top 10 using something like
if Description.contains(keyword1);
But then figure out how many times this happens per line to populate the MatchedWords field.
My question is, is this the best way to do this? If not, what will happen?
source to share
You must use full text indexing. MS SQL Server 7 and later has a full text indexing engine ( decent review article here ). You might also consider using external products such as Lucene (available for Java and C # /. NET ).
source to share
take a look at lucene for .net, which will allow a complete index of your text.
http://incubator.apache.org/lucene.net/
Net developers on this site can tell you if there are any better alternatives
source to share
If you are working with Java or C # I would recommend Lucene or Lucene.NET respectively.
source to share