DSNF - Drupal Social Network Framework (James version)

We encourage users to post events happening in the community to the community events group on https://www.drupal.org.
JamesAn's picture

Overview:

Create a social networking (SN) framework in Drupal 6 (D6) that integrates existing contrib modules together and create modules to extend SN features currently implemented in contrib modules.

This proposal is based on justageek's community proposal.

Motivation:

There exist a fair number of contrib modules the enable users to socialize in multi-user Drupal sites. Together, they represent many of the features required or desired to build a social site. Unfortunately, working these modules together can be difficult if not impossible when they're not coded to integrate into each other. There also exists some basic SN features like user search that are missing in Drupal.

Creating a SN framework and SN modules in Drupal will pull together these modules into a cohesive package and make it easier for site admins to roll out sites social networking features.

Deliverables:

  • Create an install profile to streamline how SN modules are configured.
  • Build modules to fill gaps in social networking features where identified.
    • Extend Token to support Advanced Profile Kit (APK) and Content Profile (CP) fields.
      • Some work has already been done for core Profile in Drupal 5 (D5) and D6 through a Token issue by recidive (D6) and smk-ka.
      • Work has stalled since October 2008 and there were comments that the profile extension should not be included in [[http://drupal.org/project/token|Token]] as an .inc file.
      • Support can be added for APK and CP if either modules are enabled and used.
    • Create a APK, CP, and Profile (let's call all three profile modules as ACP) field-specific to replace the default user search.
      • Current user search is done through a single text field on the username.
      • In a SN site, users may want to search for site members with similar profile fields.
      • If CP is used for user profiles, a user search listing can be constructed with Views using exposed filters.
      • General support for field-specific search can be added such that it works consistently across ACP.
    • Create a location- and proximity-based user search.
      • In some geographically-dispersed SN sites, users may want to find members physically near them.
      • Using a field-specific search is very clunky as it's challenging to define nearness besides an exact match with some geographic name (ie. city or state/province name).
      • Using Location/GMap, geocoding locations can be added to ACP as a field.
      • The field-specific user search can be extended to allow users to search for nearby members (and specify the nearness distance) by comparing geocoded locations.
    • Collect the various privacy modules and create a single and simple way for users to set privacy modules with respect to their profile fields and content
      • A number of privacy modules exist (like CCK field privacy and Profile Privacy) that give users some control over their content on Drupal sites.
      • There lacks a consistent way to allow users to configure privacy settings across all their site content.\
      • Users should be able to control who can and cannot see their content at a per content type level.
      • Users should also be able to set profile privacy at a field-specific level.
      • In terms of granularity, user profiles can be broken down into the following:
        • entire profile
        • individual field groups in profile
        • individual fields in profile
      • and user content into the following:
        • online status
        • activity
        • other content types (except forum posts, maybe?)
          • blog entries
          • galleries or photos (which image/gallery module should this support?)
          • friend list (user relationships or flag friend or friendlist).
      • Visibility options can be divided into the following:
        • public (only if site allows anonymous users to see the specific piece)
        • members only
        • friends only (as defined by user relationships or flag friend or friendlist)
        • no one (except admins)
  • Create an administrative section to configure the various SN framework modules (time-permitting).
    • An notable obstacle in administering a Drupal site is the high learning curve needed to navigate the admin area.
    • Interesting admin sections/panels/widgets were showcased by D7UX here, here, here, here, and here.
    • An SN admin section to manage the various SN settings would be very helpful in pulling everything together.

Many thanks to Michelle for bouncing some of these ideas around with me. It really help ground my thoughts and think about how some of these could work.

Other deliverables:

  • Submit research and module evaluations as part of a Drupal.org (DO) handbook.
  • Various "bridging" modules, as needed, to synchronize or integrate contrib modules together.
  • Create a theme to demonstrate how social framework can be styled/themed (time-permitting).

Underlying Project Themes:

  • Enable non-technical (ie. no coding knowledge) administrators to roll out a social site using this framework.
  • Allow admins to choose, where appropriate, which module to implement a particular feature (ie. Markdown or WYSIWYG or FCKEditor).
  • Document how the contrib modules fit together to form the framework and how custom code bridges these modules together.
  • Benchmark code and how modules work together specifically focusing on the site's memory footprint.

Preliminary list of functionality to integrate:

  • Photo/image handling
  • Friends, forums, private messaging, chat, organic groups, activity/heartbeat/etc.
  • Date/events, spam control, input editors/filters, finer-grain (role-based?) access control
  • Profile with custom (CCK) fields

Roadmap:

  1. Scope project and research: (done mostly before GSoC starts)
    • Scope the extent of the SN framework.
    • Cover as much of identified SN features with existing contrib modules as possible.
    • Evaluate modules for their readiness for use, quality of documentation, code efficiency, and support.
    • Report on the findings through documentation similar to fago's "Comparing Social Networking Modules for the drupal.org Redesign.".
      • The associated discussion can be found here.
    • Identify gaps in contrib module feature coverage, integration between modules, and documentation.
  2. Fill gaps: (May 23 to late-June)
    • Create modules to add missing functionalities (See above in Deliverables).
    • Document custom modules and also how the modules fit together to form the framework.
  3. Install profile and config: (late-June to early-July)
    • Create an install profile to configure the framework initially.
    • Create an administrative section to coordinate the configuration of the different framework modules (time permitting).
  4. Test, test, test! (early-July to late-July)
    • Create coverage test against custom modules written for Deliverables (with SimpleTest framework).
    • Test framework against default Garland theme (as framework should be theme-agnostic).
  5. Create a theme to demonstrate how social framework can be styled/themed (time-permitting).
  6. Create an administrative section to configure the various SN framework modules (time-permitting).

I'd actually like to complete most of step 1 (scope project and research) before the GSOC begins. That way, I can hit the ground coding (or running, metaphorically) when GSOC begins. I've left some scheduling space at the end of the GSoC to allow for the couple of time-permitting deliverables.

Personal Bio:

I'm a computer engineering student at the University of Toronto, currently on a co-op placement in the research group of a hedge fund company in Toronto. I work as a quantitative researcher, modeling market prices, testing models on a Java/C++ platform, and improving the testing plaform in an agile development cycle.

Learning the technical aspects of web development has been quite straight-forward, as I am already fluent in a number of programming languages and am also aware of the foundations behind languages themselves. I'm an advocate for compliant XHTML 1.0 and CSS 2.0, as well as accessible (and thus search-engine-friendly) code and secure coding practises. My technical background in more complex logic in languages like C++ and Java have helped me rapidly become fluent in PHP. I've also worked with MySQL databases (as well as other SQL implementations). My brief work with Javascript and Ajax is mainly through jQuery and XML.

My freelance work so far has primarily involved D6, from building basic sites ground-up to migrating from HTML/CSS-based sites to Drupal. I'm comfortable working with Drupal's API and have submitted a number of core novice patches for Drupal 7 development ever since learning about the core novice issue queue.

Do check out my oDesk profile at http://www.odesk.com/users/~~a045fdc56020fe95. It includes some relevant Drupal work I have done, under the Portfolio section, as well as oDesk-administered aptitude tests in various web technologies. I have references if needed. You can download my resumé at http://jamesan.ca/resume.pdf, which highlights some of my technical skills and work, as well as my involvement in a broad context around the City of Toronto. I hope you come away from this with the impression that I passionate about my work, that I am goal-focused, business-minded, and deadline-driven, and that I thrive in a diverse range of work environments with a large variety of people.

Personal Motivation:

My partner and I launched a D5 social site last May (OpenMynds, aka OM) An online community we both belonged to fell apart and - last May - it showed no signs of return, so we cobbled together some D5 modules and launched a temporary makeshift social site for friends and former members. OM is now here to stay, as the former site is not returning.

We've been trying to design an upgrade path to D6, as D6 has awesome UI improvements. After reading the Pro Drupal Development, I was raving about install profiles but realized how little they're used and how silly it would be to create a D6 install profile just for OM. We've been evaluating various D6 contrib modules and eagerly waiting for their stable release. Increasingly, I think we'll end up designing OM in D6 from scratch and this social framework fits perfectly. We'll simply have to import the D5 data in with some MySQL fun!

It'd be very nice to be sponsored for work that I'm very committed to see completed, that I'm capable of doing, and that will help others facing steep time requirements to wade through module directories and issue queues to evaluate modules when building a social site. OM was built on the spare time of my partner and I; its expenses are personally covered by us as we'd like to keep the space ad-free. We've been researching various youth grants and financial awards as a way to fund the work we're doing with OM, but the international nature of OM makes it very difficult to qualify for region-specific funding. The GSOC money will be able to indefinitely fund OM's hosting costs in a small endowment fund of sorts.

Why do Markdown abbreviations not work?.. =(

Comments

Not fleshed out enough

dmitrig01's picture

This proposal needs a lot more detail. Most of step one should be figured out before a summer of code starts, so that we know what you actually meant by this proposal -- it's kind of hard to see what you meant if you don't even know what specifications you're using. However once you do number one there's not much left of a project -- this might take a week or two but definitely not the whole summer.
Additionally, we found that it's extremely hard to identify a feature set for a generic social network site -- there are many many kinds of social network sites and you'll end up creating just one.

I'll flesh it out some more tonight.

JamesAn's picture

I'll flesh it out some more tonight. There's a vision for this proposal in my mind that I'd like to sketch/write out.

Like I said, I'd like to complete a lot of the high-level scoping and research before the summer, but I still expect hidden gaps to pop up when I begin inspecting code. Some contrib modules don't document everything they do and many modules don't document how they do what they do. That "How do they do that?" question will take some time to answer.

I'd argue bridging together modules is non-trivial. A lot of it depends on the quality of the contrib module I'll be trying to bridge together and the quality of contrib modules does vary. For example, some don't pass strings through the t() function while others output hard-coded mark-up without passing it through a theme() function to give developers/themers a chance to override it.

As a testament to how non-trivial it can be, check out Buddypress. What started as a one-person mission to make Wordpress MU more interactive has become a sizable OS project with hundreds of contributers all working for the sole purpose of extending Wordpress with social networking features. It's been over a year and they're just coming up to a release candidate now. Not trivial at all.

Please share your work, if you (or other Drupal folks) have thought about abstracting a feature set for a generic social network site! I'd love to check out what's been done. (There's no point in re-inventing the wheel, right?) Yes, it's true that social sites can vary widely, but there still tends to be some common features. So no, I won't be creating a particular social site. I'll be creating a framework to building social sites more easily with Drupal from a pool of common features implemented by various contrib modules. Of course, users can extend it further; the focus here is to provide solutions for common needs like profile, fine access control (or privacy control) that users can set, or image sharing.

I've sift through previous and current GSoC projects and the vast majority of them focus on back-end functionality from Views improvements to WebDAV integration. Great work. But UI and usability seem relatively absent from the mix.

As a newcomer to the Drupal community, the "There's a module for that!™" slogan (which I'm guilty of using on many occasions too!) seems to neglect how steep the learning curve can be for (non-technical) people to figure out how to implement modules. Without PHP skills and good knowledge of Drupalisms (or lots of time and the ability to scavenge for and understand Drupal API documentation), I'd argue it's virtually impossible to make a social site with Drupal that provides good user experience in any sense of the word.

Drupal is supposed to make it easy(er) for people to create websites. It does, in some ways, and it doesn't, in other ways. This is one notable way to make it easier.

Anywho.. more actual details coming up tonight!

Maybe not tonight.. >_<

JamesAn's picture

Tomorrow then! ^^


Update: I rejigged some things.

Michelle messages in the other thread and I've been excitedly looking through her work at http://shellmultimedia.com/tutorials/contributed-modules. It felt like being a kid on Christmas morning. XD

I have my own running list of modules that I've been watching in D6; I plan to merge my thoughts with Michelle's list and try to better solidify the research piece of this proposal. Maybe with more concrete modules and features listed, there'd be specifics to talk about.

Some rough work

JamesAn's picture

This borrows quite extensively from Michelle's http://shellmultimedia.com/tutorials/contributed-modules. I've found other handbook pages and GDO/DO nodes that compare various relevant modules. More to come.

Comparison Pages

Module Type Link to Comparison Page
Lightbox http://drupal.org/node/266126
Friends http://groups.drupal.org/node/19748
Images/Gallery http://drupal.org/node/316884
Links Pages http://drupal.org/node/298397
Access Control http://drupal.org/node/270000

Categories

  • Utility
  • Registration
  • Friends
  • Blogs
  • Incentives
  • Profiles
  • Privacy/Access Control
  • Forums
  • Groups
  • Chat
  • Physical Location
  • Event/Date
  • Images/Gallery
  • SEO/Analytics
  • Other
  • Spam Control
  • Input Filters
  • Ajax
  • Search
  • Usability
  • Announcements

Additional Module Groups to Research

Category Placeholders
Announcements Simplenews Modules
Images/Gallery User Image Gallery
Search Improved User Search
Chat Dimdim
Chat Realchat
Chat Userplane
Images/Gallery Merge Avatar Image Gallery
Input Filters Video/Image filtering
Usability Jquery stuff
Privacy/Access Control Content Access Control

Module Matrix

Category Module Name D6 Status Last Update Notes/Integration
Utility [[http://drupal.org/project/admin_menu Admin menu]] stable
Utility [[http://drupal.org/project/devel Devel]] stable
Utility [[http://drupal.org/project/views Views]] stable |OO docs are lacking
Utility [[http://drupal.org/project/cck CCK]] stable
Utility [[http://drupal.org/project/panels Panels]] alpha |Panels 2 won't move beyond alpha; Panels 3 in alpha 3
Utility [[http://drupal.org/project/advanced_help Advanced Help]] stable
Utility [[http://drupal.org/project/pathauto Pathauto]] stable
Utility [[http://drupal.org/project/flag Flag]] stable
Utility [[http://drupal.org/project/token Token]] stable
Registration [[http://drupal.org/project/logintoboggan Login Toboggan]] stable
Registration [[http://drupal.org/project/legal Legal]] beta
Profiles [[http://drupal.org/project/auto_nodetitle Automatic Nodetitles]] stable
Profiles [[http://drupal.org/project/content_profile Content Profile]] beta
Friends [[http://drupal.org/project/user_relationships User Relationships]] beta
Friends [[http://drupal.org/project/flag_friend Flag Friend]] beta
Friends [[http://drupal.org/project/privatemsg Privatemsg]] rc2
Friends [[http://drupal.org/project/ignore_user Ignore User]] stable
Profiles [[http://drupal.org/project/author_pane Author Pane]] stable
Blogs [[http://drupal.org/project/usercomment User Comment]] dev
Blogs [[http://drupal.org/project/bloginfo Blog Information]] dev
Blogs [[http://drupal.org/project/blogtheme Blog Theme]] stable
Incentives [[http://drupal.org/project/userpoints User Points]] stable
Incentives [[http://drupal.org/project/userpoints_contrib User Points Contributed Modules]] dev Apr 1, 2009
Incentives [[http://drupal.org/project/userpoints_top_contributors User Points Top Contributors]] stable
Incentives [[http://drupal.org/project/user_titles User Titles]] beta Jan 6, 2009
Incentives [[http://drupal.org/project/user_badges User Badges]] stable
Incentives [[http://drupal.org/project/fivestar Fivestar]] stable
Incentives [[http://drupal.org/project/fivestarextra Fivestar Extra]] beta Nov 6, 2009
Incentives [[http://drupal.org/project/kudos Kudos]] stable
Profiles [[http://drupal.org/project/advanced_profile Advanced Profile Kit]] alpha Jan 17, 2009|Waiting for Panels 3 release
Profiles [[http://drupal.org/project/activity Activity]] stable
Profiles [[http://drupal.org/project/facebook_status Facebook-Style Statuses]] stable |User Relationships, Advanced Forum
Utility [[http://drupal.org/project/me Me Aliases]] stable
Profiles [[http://drupal.org/project/user_display User Display API]] no port Dec 16, 2009
Privacy/Access Control [[http://drupal.org/project/cck_field_privacy CCK Field Privacy]] stable |User Relationships
Forums [[http://drupal.org/project/advanced_forum Advanced Form]] stable
Usability [[http://drupal.org/project/live Live]] stable
Forums [[http://drupal.org/project/quote Quote]] beta Feb 24, 2009
Forums [[http://drupal.org/project/smileys Smileys]] alpha Jan 5, 2009
Forums [[http://drupal.org/project/signature_forum Signatures For Forums]] stable
Forums [[http://drupal.org/project/comment_upload Comment Upload]] alpha Jan 30, 2009|Looking for new maintainer
Forums [[http://drupal.org/project/modr8 Modr8]] stable
Forums [[http://drupal.org/project/comment_mover Comment Mover]] dev Mar 30, 2009|New maintainer with clear roadmap to release
Forums [[http://drupal.org/project/comment_subject Re: Comment Subjects]] stable
Forums [[http://drupal.org/project/forum_access Forum Access]] beta Mar 18, 2009|ACL beta4, OG Forum
Usability [[http://drupal.org/project/fasttoggle Fasttoggle]] stable
Forums [[http://drupal.org/project/flatcomments Flatcomment]] stable
Messaging [[http://drupal.org/project/notifications Notifications]] stable |OG, Mail to Web, Team Notifications
Messaging [[http://drupal.org/project/comment_subscribe Comment Subscribe]] stable
Messaging [[http://drupal.org/project/subscriptions Subscriptions]] beta Dec 27, 2008
Messaging [[http://drupal.org/project/messaging Messaging]] stable |Twitter, XMPP Framework, SMS Framework, Messaging Send To
Forums [[http://drupal.org/project/commentrss Comment RSS]] stable
Forums [[http://drupal.org/project/advpoll Advanced Poll]] dev Mar 21, 2009|jQuery Update, jQuery UI
Forums [[http://drupal.org/project/comment_page Comment Page]] stable
Groups [[http://drupal.org/project/og Og]] stable
Groups [[http://drupal.org/project/og_block_visibility Og Block Visibility]] stable
Groups [[http://drupal.org/project/og_calendar Og Calendar]] stable
Physical Location [[http://drupal.org/project/gmap Gmap]] rc1 Mar 3, 2009|Requires Gmap API key
Physical Location [[http://drupal.org/project/location Location]] rc1 Mar 3, 2009
Physical Location [[http://drupal.org/project/addresses Addresses]] stable
Event/Date [[http://drupal.org/project/date Date]] stable
Event/Date [[http://drupal.org/project/calendar Calendar]] stable
Event/Date [[http://drupal.org/project/event Event]] dev Mar 15, 2009|Includes signup/RSVP extensions
Images/Gallery [[http://drupal.org/project/imagefield ImageField]] rc1 Apr 1, 2009
Images/Gallery [[http://drupal.org/project/imagefield_crop ImageField Crop]] beta Mar 19, 2009
Images/Gallery [[http://drupal.org/project/imagecache ImageCache]] beta Feb 17, 2009
Images/Gallery [[http://drupal.org/project/transliteration Transliteration]] stable
Images/Gallery [[http://drupal.org/project/custom_pagers Custom Pagers]] beta
Images/Gallery [[http://drupal.org/project/lightbox2 Lightbox 2]] stable
SEO/Analytics [[http://drupal.org/project/nodewords Meta Tags]] stable |Waiting on stable/rc panels release
SEO/Analytics [[http://drupal.org/project/page_title Page Title]] stable
SEO/Analytics [[http://drupal.org/project/google_analytics Google Analytics]] stable |Requires Google Analytics account
SEO/Analytics [[http://drupal.org/project/path_redirect Path Redirect]] stable
SEO/Analytics [[http://drupal.org/project/globalredirect Global Redirect]] stable
Announcements [[http://drupal.org/project/simplenews Simplenews]] rc6 Mar 9, 2009|Mime Mail
Announcements [[http://drupal.org/project/mimemail Mime Mail]] dev Feb 22, 2009
Usability Custom Error
Search Google CSE
Utility Hirarchical Select
Spam Control Mollom
Images/Gallery Views Slideshow
Chat Chat Room
Friends Buddylist2
Registration Username Originality AJAX Check
Usability Front Page
Profiles Guestbook
Profiles Unique Avatar
Images/Gallery Avatar gallery
Usability Automated Logout
Usability WYSIWYG
Spam Control CAPTCHA
Spam Control reCAPTCHA
Profiles User Stats
SEO/Analytics XML Sitemap
Usability Ajax Submit
Usability Popups API (Ajax Dialogs)
Usability jQuery UI Tabs
Usability Markdown Filter
Alex UA's picture

@JamesAn-

I really want to see this get in, but it needs more work, and it's going to need to happen ASAP if this is going to get in. Some items that are missing:

  • Please clear the mentor section until you actually have mentors.
  • You need much more well defined set of deliverables- right now you are way too vague. Please let us know what code you expect to write, exactly what modules you intend to create, etc.
  • While it's good to look towards getting module maintainers to "play nice", it can be extremely difficult (or impossible) to get some of them to work together. Look beyond bridge modules to what you can deliver within the timeframe you'll have.

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

Alex Urevick-Ackelsberg
ZivTech: Illuminating Technology

Thanks for the advice

JamesAn's picture

While retaining the lofty (ie. vague) goals that I'd like to keep in mind, I've include some tangible deliverables: the install profile and two modules to fill missing features: user tokens for contrib profile modules (APK and Content Profile) and field-specific and proximity user search.

I suspect there are other notable gaps in features that may come up as I research and test the modules in greater detail.

I did something similar to

rpfilomeno's picture

I did something similar to this 2 years ago and i called it cribz (http://corruptedpartition.blogspot.com/2007/10/cribz-on-final-developmen...) Basically it a D6 with roughly 30 modules and a lot of views. It's good but I had to discontinue it for one reason, its has a very heavy footprint -- it needs 83MB memory!

Then after having around 1000+ beta users, it started choking my machine (non-dedicated). One of the conclusion I got is using existing modules isn't the best way to build an SNS site. You must build the module yourself and make it very lean (by passing hooks and views). Now doing this i was able to build one that can take around 11K users before needing a sort caching and around 100K I started using memcache -- 2 years later we sold it ($cathing$).

So I'm very interested on this project as well but doubt that using existing module's will yield a good performance or will be usable beyond being hobby site size (ROI: userbase to resource cost ratio).

repost from socghop

thanks for this information

justageek's picture

Is the memory load simply caused by loading so many modules for each page render / load?

Did you create a single module that did all of your community functionality, or did you still use a handful of contributed modules?

Congratulations James

karunadave's picture

I just noticed you are in the Google summer of code. Great. I look forward to making use of your contributions (can't wait).

How did this go?

yautja_cetanu's picture

How did this go?