http://groups.drupal.org/soc-content-recommedation-engine Group consists of those interested in a content recommedation engine. Currently, one is being authored by Scott Reynolds for Google's Summer of Code en http://groups.drupal.org/node/3445 <p>Hey there,</p> <p>I'm really looking into using CRE for a new project and eventually sponsoring/building new plugin modules for it, but first I really need to understand what the actual modules do, and how the recommandation works now, in the stock module.</p> <p>My main question is this:</p> <p>When a user votes on a node, does the system remebers which user 'added' the voted on node and which taxonomy terms where used??</p> <p>If yes, does this mean that when a user votes on a node it also gives a vote to the node author and the taxonmy terms that are attached to the nodes?</p> <p>If no, it means that the actual relation created by this module is based specifically on 'who' voted on 'what' regardless of node author and taxonomy terms attached??</p> <p>My question is, would it be too much of a load for a site to implement modules that would create relations to the node author (when I vote on a node, I give the node author a +1 as well) and to the taxonomies involved?? (when I vote on a node, I give a +1 to the taxonomies involved as well...)</p> <p>I guess then the issue would be to be able to have a single output based on those three recommandation modules??</p> <p>1) regular 'who' voted 'what' output<br /> 2) output based on authors that you 'recommended'<br /> 3) output based on taxonomies that you voted on...</p> <p>Well, I think that the CRE module at this point does not create any relations regarding the node author and the taxonomy terms used, but i guess adding those two modules would create a really nice recommandation system and 'soft' personnalisation of a content site??</p> <p>Any ideas where to start, how to implement, and tips about what's already in the module and where to read the docs for the API??</p> <p>I'd also be willing to team up with other users to maybe sponsor the creation of a 'set' of modules that would create the recommandation system I'm looking for here... If anyone is interested in 1) coding this or 2) sponsoring this with me, please comment here...</p> <p>Thanks,<br /> Patchak</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/3445#comments SoC 2006: Content Recommedation Engine Sun, 01 Apr 2007 10:13:10 +0000 patchak@drupal.org 3445 at http://groups.drupal.org http://groups.drupal.org/node/3319 <p>Hi there,</p> <p>I'm building a new site which uses the leech.module to let people add feeds for their own blogs, etc...</p> <p>What I'd like to know is if it's possible to establish some relation between score given to the feed node (the source) and it's childs. For example if a user would give a 5 start rating to the node of the feed source, is there any way to actually recommend it,s feed items??</p> <p>I was thinking of using the node review of vote up down modules on the feed sources, but realized that this would allow to recommend sources, but not necessarily the related feed items...</p> <p>How would you approach this?</p> <p>Thanks</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/3319#comments SoC 2006: Content Recommedation Engine Sun, 25 Mar 2007 12:33:13 +0000 patchak@drupal.org 3319 at http://groups.drupal.org http://groups.drupal.org/node/3170 <p>I noticed that there are recommendation modules for comments, nodes, and users, but not taxonomy. Would it be possible to recommend taxonomy to people?</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/3170#comments SoC 2006: Content Recommedation Engine Wed, 14 Mar 2007 20:54:37 +0000 greggles 3170 at http://groups.drupal.org http://groups.drupal.org/node/2435 <p><del>5-0 development snapshot has been created (waiting for scripts to roll the tarball). </del></p> <p>5.x dev snapshot has some bad problems, including old code and missing .info file. I have released a official release 5.x-1.0 that is proper and ready for consumption.</p> <p>What happened was i expected the -dev snapshot to be rolled on branch DRUPAL-5-0, NOT DRUPAL-5. I will fix the -dev snapshot or remove it entirely somehow. But I thought it was important to get the word out</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/2435#comments Content Recommendation Engine summer of code SoC 2006: Content Recommedation Engine Thu, 18 Jan 2007 06:20:10 +0000 Scott Reynolds@drupal.org 2435 at http://groups.drupal.org http://groups.drupal.org/node/1526 <p>Netflix has released its dataset to the public in hopes of finding a new recommender system. <a href='http://www.nytimes.com/2006/10/02/technology/02netflix.html?_r=2&amp;oref=slogin&amp;oref=slogin'> Here is NYT </a> article (subs required: see bugmenot.com).</p> <p>Interesting news story at least. Perhaps I will spend a weekend and submit Slope One</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/1526#comments SoC 2006: Content Recommedation Engine Mon, 02 Oct 2006 16:46:57 +0000 Scott Reynolds@drupal.org 1526 at http://groups.drupal.org http://groups.drupal.org/node/1442 <p>Alright,</p> <p>There is a feature request in the queue for this. <a href="http://drupal.org/node/85360" title="http://drupal.org/node/85360">http://drupal.org/node/85360</a></p> <p>Here is a quote on my plan: </p> <blockquote><p> Sorry awefully cryptic, but essentially, User1 liked Node A. The average difference between Node A and Node D is .14. Average Difference between Node A and Node B is .25. Therefore, if User1 hasnt voted on Node D, he is more likely to like Node D then Node B. Thats the plan, and its not quiete the same algorithm as with percentage, though very close.</p> <p>The difference between perctange and points is that a user doesn't have really have an 'average vote' that could be used to personalize the recommendations. It has instead a bipolar system (like or dislike). So in a sense, it is actually considerably less computationally intense.</p></blockquote> <p>Now, I havent commited anything to code yet. I have to set up a development system with up/down and the like. But this is my idea. But I am very open to other opinions.</p> <p>I plan on making percentage and points a seprate query type (i.e. part of the same radio functions that determine fast/accurate query).</p> <p>Again, i would like to implement this for e-commerce as well so that whenever someone buys a product it gets a VotingAPI +1 point.</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/1442#comments SoC 2006: Content Recommedation Engine Fri, 22 Sep 2006 16:38:06 +0000 Scott Reynolds@drupal.org 1442 at http://groups.drupal.org http://groups.drupal.org/node/1368 <p>I have updated Node Recommmendation to fix some bugs in the page callback and to use teasers instead of a item list to list recommended nodes on the page. My hope of course is that node recommendation page becomes the front page of many drupal sites.</p> <p>Summer of Code is offically over but I do intend to keep this list semi-updated with work on the recommendation engine. There are several different things I will be expanding the project to do. I plan on setting it up for VotingAPI's point system. And then writing a module that will work with e-commerce to add a VotingAPI 'point' to each purchase. This of course would allow the engine to draw relationships between content based on who bought what. (Which I think is particalarly cool and interesting).</p> <p><em>done</em>Though my immediate first step is to change the 'date' functionality of node_recommendation module. Robert Douglass sugguested that I add the functionality to set it up so it recommends content that is less then a week old and etc which of course makes perfect sense. <em>done</em> 9/11</p> <p>And of course, I swear at some point I am goin to sit down and write out the views stuff.</p> <p>Please let me know if you are still using the engine, and if not what is holding you back from using it</p> <p>Scott</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/1368#comments Content Recommedation Engine summer of code SoC 2006: Content Recommedation Engine Sat, 09 Sep 2006 00:49:52 +0000 Scott Reynolds@drupal.org 1368 at http://groups.drupal.org http://groups.drupal.org/node/1231 <h2>Install instructions</h2> <p>Simply <a href='http://drupal.org/project/cre'>download</a> the project and place all modules within the /modules subdirectory.<br /> Please make sure that you have downloaded BOTH <a href='http://drupal.org/project/votingapi'>VotingAPI</a> and some voting module (for example: <a href='http://drupal.org/node/42905'>SimpleVote</a>)</p> <p>And then turn them on!</p> <h4>To use SimpleVote on comments</h4> <p>Add this function to SimpleVote.module<br /> <div class="codeblock"><code><span style="color: #000000"><span style="color: #0000BB">&lt;?php<br /></span><span style="color: #007700">function </span><span style="color: #0000BB">simplevote_comment</span><span style="color: #007700">(&amp;</span><span style="color: #0000BB">$comment</span><span style="color: #007700">, </span><span style="color: #0000BB">$op</span><span style="color: #007700">) {<br />switch (</span><span style="color: #0000BB">$op</span><span style="color: #007700">) {<br />case </span><span style="color: #DD0000">'view'</span><span style="color: #007700">:<br /></span><span style="color: #0000BB">$comment</span><span style="color: #007700">-&gt;</span><span style="color: #0000BB">comment </span><span style="color: #007700">= </span><span style="color: #0000BB">theme</span><span style="color: #007700">(</span><span style="color: #DD0000">'simplevote_widget'</span><span style="color: #007700">, </span><span style="color: #0000BB">$comment</span><span style="color: #007700">-&gt;</span><span style="color: #0000BB">cid</span><span style="color: #007700">, </span><span style="color: #DD0000">'comment'</span><span style="color: #007700">) . </span><span style="color: #0000BB">$comment</span><span style="color: #007700">-&gt;</span><span style="color: #0000BB">comment</span><span style="color: #007700">;<br />break;<br />}<br />}<br /></span><span style="color: #0000BB">?&gt;</span></span></code></div></p> <p>To install over existing VotingAPI data please make sure cron runs are enabled. As existing voting data is processed via cron.</p> <h2>user_recommendation.module</h2> <p>Not your standard recommendation module no! It recommends users<br /> <strong>So the general idea, to recommend a user because you might like what that user likes. </strong></p> <h4>Alogrithm</h4> <p>First we want the difference between the averages of the current user and all other users.</p> <pre><code>SELECT a.uid as 'current_user', a2.uid as 'sugguested_user', a.sum/a.count as 'current_average', a2.sum/a2.count as 'sugguested_average',ABS((a.sum/a.count)-(a2.sum/a2.count)) as 'avg_diff' FROM cre_average_vote a, cre_average_vote a2 WHERE a.uid &lt;&gt; a2.uid AND a.uid=2 ORDER BY avg_diff; </code></pre><p>Next, we want to use a count of votes that were cast for the same piece of content to weight the 'avg_diff' between the two users</p> <p><code>&nbsp;&nbsp;&nbsp; SELECT COUNT(*) FROM votingapi_vote r, votingapi_vote r2 WHERE r.uid = 2 AND r2.uid = 3 AND abs( r.value-r2.value) &lt;= 20; </code></p> <p>Here I used a difference of 20 percentage points. This difference is what determines what votes are revelant. Any vote that is on the same piece of content BUT the difference between user1's vote and user2's vote is MORE THAN 20, it doesnt get counted.</p> <p>So combining those two sql statements into one HUGE SQL statement</p> <p><code><br /> SELECT COUNT(<em>) as 'count', a.uid as 'current_user', a2.uid as 'sugguested_user', a.sum/a.count as 'current_average',a2.sum/a2.count<br /> as 'sugguested_average',ABS((a.sum/a.count)-(a2.sum/a2.count)) as 'avg_diff',ABS((a.sum/a.count)-( a2.sum/a2.count)) / count(</em>) as<br /> 'score' from votingapi_vote r, votingapi_vote r2,cre_average_vote a, cre_average_vote a2 where r.uid = a.uid AND r2.uid = a2.uid AND<br /> abs(r.value-r2.value) &lt;= 20 AND a.uid &lt;&gt; a2.uid AND a.uid=2 GROUP BY avg_diff ORDER BY score ASC; </code></p> <p>score is calculated by taking the avg_diff and dividing it by the number of revelant votes (votes cast on the same piece of content whos difference was less then or equal to 20). Obviosuly, all those columns are not needed in the final module, they are there because they helped me better understand what I was playing with and I hope they help you understand.</p> <p>_user_recommendation_user_to_user($uid) is a function that will return a db_result object for the big query ^ It is a great way to tie your module into recommended users</p> <p>Please Thoughts? Sugguestions? Criticisms?</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/1231#comments Content Recommendation Engine Google SoC2006 summer of code SoC 2006: Content Recommedation Engine Wed, 16 Aug 2006 00:26:42 +0000 Scott Reynolds@drupal.org 1231 at http://groups.drupal.org http://groups.drupal.org/node/1153 <p>Just released new version. Included in the this release is the new-fangled cre query object and comment_recommendation.module which recommends comments within the current node being viewed.</p> <p>The object allows for joining on any table. Please remember to use {} and a name to identify your table.</p> <p>For instance node_recommendation does this:<br /> <div class="codeblock"><code><span style="color: #000000"><span style="color: #0000BB">&lt;?php<br /></span><span style="color: #007700">function </span><span style="color: #0000BB">node_recommendation_cre_query </span><span style="color: #007700">(&amp;</span><span style="color: #0000BB">$query</span><span style="color: #007700">) {<br />&nbsp;&nbsp; </span><span style="color: #0000BB">$query</span><span style="color: #007700">-&gt;</span><span style="color: #0000BB">add_table</span><span style="color: #007700">(</span><span style="color: #DD0000">"{node} n"</span><span style="color: #007700">);<br />&nbsp;&nbsp; </span><span style="color: #0000BB">$query</span><span style="color: #007700">-&gt;</span><span style="color: #0000BB">add_column</span><span style="color: #007700">(</span><span style="color: #DD0000">"n.title"</span><span style="color: #007700">);<br />&nbsp;&nbsp; </span><span style="color: #0000BB">$query</span><span style="color: #007700">-&gt;</span><span style="color: #0000BB">add_where</span><span style="color: #007700">(</span><span style="color: #DD0000">"n.nid=d.content_id1"</span><span style="color: #007700">);<br />}<br /></span><span style="color: #0000BB">?&gt;</span></span></code></div></p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/1153#comments Content Recommedation Engine Google SoC SoC 2006 SoC 2006: Content Recommedation Engine Mon, 07 Aug 2006 05:56:30 +0000 Scott Reynolds@drupal.org 1153 at http://groups.drupal.org http://groups.drupal.org/node/1108 <p>Ok major speed fixes. Redid the query compeletly and added a average user vote table. This allows for the movelens database to run MUCH MUCH MUCH MUCH FASTER. I will begin testing it later this week and give back some scores that should be interesting. Try running the update script for me. I did not test it out and I am worried that I didn't write it properly (my first update() :-D ).</p> <p>Still working on creating a query object that will prevent the one issue that is in the queue right now. More to come soon...</p> <p><a href="http://drupal.org/project/cre" title="http://drupal.org/project/cre">http://drupal.org/project/cre</a></p> <p>Scott</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/1108#comments Content Recommedation Engine Google SoC SoC 2006 SoC 2006: Content Recommedation Engine Mon, 31 Jul 2006 18:49:34 +0000 Scott Reynolds@drupal.org 1108 at http://groups.drupal.org http://groups.drupal.org/node/1052 <p>OK,</p> <p>Finally finished it. Their happens to be a permission problem still though. Users not uid=1 will not be able to view the node that is recommended or access the recommendations page. Pretty sure you can fix that in the admin panel but havent done that yet. If any can shed some light on it please do. Here is where you can find the stuff: <a href="http://scottreynolds.us/Downloads/movielenssql.tar.gz" title="http://scottreynolds.us/Downloads/movielenssql.tar.gz">http://scottreynolds.us/Downloads/movielenssql.tar.gz</a></p> <p>100k movielens is not done, it is in a different file format thereby requiring a different program.<br /> Info from readme file: </p> <blockquote><p>These files contain 1,000,209 anonymous ratings of approximately 3,900 movies<br /> made by 6,040 MovieLens users who joined MovieLens in 2000.</p></blockquote> <p>Please remember then when using this to test the module it takes MANY MANY cron runs to fully install the module it. Please play with the settings in admin/settings/cre to adjust the number of votes and report any findings (PLEASE!).</p> <h3>Info on the java program</h3> <p>The program itself generates .sql files. There are 3 classes and each generate their repesective .sql files (users.sql, movies.sql, votes.sql). Testcases.txt is a selection of randomly selected votes that WERE NOT added to the database. These should be used to check the alogrithm. To adjust the number of test cases you must adjust the % chance of being selected. That decimal is located at line 56 column 74 in the votes.java file ( I know it should be a commandline option... i just forgot to implement). Please make sure you use the .dat files located in the root of my sandbox and not in the movielens1mill folder. I had to alter movies.dat because of conjunctions like "You're" would mess up the sql statement.</p> <p>Program was created using NetBeans IDE so there is an ANT script as well. In the dist folder there is a jar file.</p> <p>Of course please remember the votes.sql is HUGE! and it should take awhile to download. Any performance issuses and sugguestions are welcome.</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/1052#comments Content Recommedation Engine Drupal Google summer of code summerofcode Voting Systems SoC 2006: Content Recommedation Engine Sat, 22 Jul 2006 17:03:43 +0000 Scott Reynolds@drupal.org 1052 at http://groups.drupal.org http://groups.drupal.org/node/1013 <p>Ok did my first commit of the modules (with README and API documentation) into contrib/modules/cre. The links in the welcome message are updated of course.</p> <h1>NEW INSTALL METHOD</h1> <p>The install method now, runs through intitially as usual but only processes a limit number of votes. It then waits to process the rest of the votes once cron.php gets called again EDIT: cron only installs a certain number of votes each cron run. Thereby, lengthing the install process over several cron runs instead of one _install() call which crashes and never completes.</p> <p>This is done this way so that large EXISTING votingapi datasets can be utilized. PHP only has a limited about of 'user' space, and because the amount of data involved in this module is N*N where N is the number of voted content, it gets rather large, and becomes a whole lot of DB queries. I am open for better solutions, mostly cause I don't particularlly like this one.</p> <p>I have been meaning to post the 'drupalized' verisions of the movielens database.</p> <p>Views intergration is coming, though it will be more complicated then I expected.</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/1013#comments Content Recommedation Engine summer of code SoC 2006: Content Recommedation Engine Sun, 16 Jul 2006 07:51:23 +0000 Scott Reynolds@drupal.org 1013 at http://groups.drupal.org http://groups.drupal.org/node/831 <p>Wondering what datasets you have available for testing?</p> <p>There are two datasets for film here,</p> <p><a href="http://www.grouplens.org/" title="http://www.grouplens.org/">http://www.grouplens.org/</a></p> <p>richard</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/831#comments dataset movies xml SoC 2006: Content Recommedation Engine Tue, 20 Jun 2006 18:39:36 +0000 schavester 831 at http://groups.drupal.org http://groups.drupal.org/node/778 <p>New modules are up. Lots of changes to cre. Including new table name and new table format. It is setup now to be able to recommend any content(user, comments, developer defined, nodes).</p> <p>And I would also like to pose a question for discussion....</p> <p>When recommending some piece of content (whether a node, comment, user etc...) does the alogrithm only recommend based on content that is the same type? so when recommending a node, only use similarity matrix referencing content that is both the same type. Thats not making much sense typing it out but here is an example.</p> <p>Johny is a user. The site is recommending both nodes and users. The block for 'Johny's recommended users' should be populated based only similarities between user and user votes or similarities between BOTH user to user and user to node. Should this be up to user level module (node_recommendation)?</p> <p>So the new cre_similarity_matrix (new name btw. Like it tons better) would have two additional fields content_type1, and content_type2. Or if we do not want to do similarities between both user and user and user to node, then it would have one additional field, content_type, that would hold the content_type of both content_id1 and content_id2.</p> <p>Here is the new table structure to help clear up this cryptic message....<br /> db_query("CREATE TABLE {cre_similarity_matrix}(<br /> content_id1 int(10) NOT NULL DEFAULT 0,<br /> content_id2 int(10) NOT NULL DEFAULT 0,<br /> content_type1 varchar(20) default NULL,<br /> content_type2 varchar(20) default NULL,<br /> count int(10) NOT NULL DEFAULT 0,<br /> sum int(10) NOT NULL DEFAULT 0,<br /> PRIMARY KEY (content_id1, content_id2)<br /> ) /*!40100 DEFAULT CHARACTER SET utf8 */;");</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/778#comments Content Recommedation Engine cre Drupal SoC 2006: Content Recommedation Engine Thu, 15 Jun 2006 00:08:21 +0000 Scott Reynolds@drupal.org 778 at http://groups.drupal.org http://groups.drupal.org/node/765 <p>just installed the module, and it looks good.</p> <p>going to fill up the database more with some ratings in the next few days.</p> <p>richard</p> <div class="og_rss_groups"><a href="/soc-content-recommedation-engine">SoC 2006: Content Recommedation Engine</a></div> http://groups.drupal.org/node/765#comments SoC 2006: Content Recommedation Engine Wed, 14 Jun 2006 03:34:43 +0000 schavester 765 at http://groups.drupal.org