Enterprise

Drupal / PHP Developer | Vivakos, Inc.

dpaolicelli — Wed, 13 Aug 2008 20:07:32 +0000

Vivakos, Inc., a leading integrator of open source and open standards based Web applications, is seeking talented and motivated PHP / Drupal developers with 3+ years experience.

As a Developer, you will design and implement Web applications for our customers, usually involving content management platforms such as Drupal. In addition, Developers may also serve in a product development capacity building marketable, integrated, packaged best-of-breed open source applications. Developers will be responsible for the planning, designing, coding, testing, and support of Web-based business applications. Candidates must be able to work autonomously and in team environments.

Technical Skills:

Deep experience coding web applications in PHP is a must.
Experience building modules for Drupal is a huge plus. (some coding experience with Drupal is required)
Some experience with Java/J2EE is helpful but not required.
Working knowledge of any object-oriented programming languages is a plus.
Willingness to apply current coding knowledge to new programming languages.
Must feel comfortable using various target Operating Systems such as Linux, Solaris, and Windows.
A demonstrated understanding of SQL.
Database experience with one or more of the following: MySQL, PostgreSQL, SQL Server, or Oracle.
Experience installing, configuring, and customizing Alfresco (i.e., developing against the API) would be very valuable, though it is not required.

Other Skills:

Strong analytical and problem-solving skills are required.
Fluency in English is critical (both oral and written).
Ability to work independently or as a member of a larger team.
Ability to document software specifications and write technical papers, reports, and client deliverables.
Ability to manage and deliver on multiple concurrent project tasks.
Willingness to learn and apply new software development tools, technologies, and methodologies.

Please note: This is a posting for a full-time hire, though temporary contracts directly with individuals will be considered. Corporate-to-corporate contracts, however, will not be considered.

Washington, DC Drupalers

Various Drupal positions | Arizona State University

jrbeeman — Wed, 23 Jul 2008 20:57:51 +0000

Arizona State University's Applied Learning Technologies Institute (ASU alt^I) is seeking Drupal developers for immediate full-time, part-time and contract-based work. We have a variety of needs including theme development and maintenance, module development and maintenance, site administration / build-out, and testing / quality assurance. For all positions, contributors to the Drupal community are preferred, but not required.

Please send Jeff Beeman (jrbeeman on Drupal.org or jeff.beeman@asu.edu) a private message or email with any questions you may have about the posting.

Posting at: http://snipr.com/33u8s
The description at the posting page is a general one for Drupal developers - please see the details on specific positions we are looking to fill below and apply based on the description(s) you see here.

Details:

Applicants for theme or module developer positions should have a solid foundation of PHP / MySQL skills. Module developers that have at least passing familiarity with the APIs of a variety of contributed modules, including CCK, Views and Organic Groups and have experience writing modules that integrate or layer on top of existing modules are preferred. Theme developers that have experience with the Zen theme and using templates to alter the display and structure of output content are preferred.

Applicants for site administation / build-out positions with experience managing large amounts of content and user data, and familiarity with using CCK and Views to build out site sections are preferred.

Applicants for testing / quality assurance positions with experience trying to break sites in any way imaginable - from finding security holes to exposing edge-case bugs, and applicants with some level of development experience and familiarity with the SimpleTest framework are preferred.

Interested parties that are out-of-state or world-wide are encouraged to apply, as working remotely is an option.

Theme development

3 Customers with more than 1000 users in production deployments needed for Gartner social software market assessment

Amazon — Wed, 16 Jul 2008 19:35:24 +0000

I am working on submitting the Drupal survey response for Gartner's latest research report. We are looking for three customers with 1000 users in a production deployment. Ideally these would be successful deployments and the customer would be willing to have a quick chat with myself to ensure we are providing the right customer reference. Good customer references are the single biggest factor in impacting report recommendations.

From the Gartner Social software market survey

We got interviews from:
Justin C. Klein Keane University of Pennsylvania School of Arts and Sciences which is standardizing their department on Drupal as the default CMS courtesy Alex Urevick-Ackelsberg.
BestBuy's BlueShirtNation intranet which has 25 000 users. Steve Bendt responded to my public post in the marketing group.
Tracy Frauzel from Greenpeace UK courtesy Robert Castello
Jupiter Media's Ivan Raszl for Adsoftheworld.com site, courtesy Khalid
SonyBMG Suzi Arnold courtesy Earl and recommended by Angie.
Morris Digital's Steve Yelvington regarding 20 sites and the largest with more than 175 000 users

"You must be able to name at least 3 customers with more than 1000 users (seats) in production deployments. You may optionally include two additional reference customers. If you meet the criteria for inclusion then some of your customer references will be contacted via email or phone. Otherwise we will let you know that we will not be contacting your references. Key points we will be looking for is large scale usage of the products and degree of success."

Newspapers on Drupal

Drupal For Enterprise

Robert Castelo — Fri, 04 Jul 2008 15:14:22 +0000

We are delighted to invite you to Drupal For Enterprise, an event focusing on the common benefits and challenges of using Drupal in a business organisation.

Whether you're evaluating Drupal or are already using it, this event is a great opportunity to learn from the experience of other organisations, put your questions to the experts, and connect with the Drupal community.

Presentations

We welcome suggestions for presentations - even better, do you have a talk you'd like to give?

These are the topics we plan to cover:

Feature Recipes (Intranet, KM, eCommerce)
Licensing and Costs
Scalability
Security
Interoperability (legacy systems)
Support

The presentation schedule will be finalised in the next two weeks, so keep watching this space....

Thank You!

Sun is an active Drupal user, long term Drupal contributor, and has very kindly provided us with an excellent venue for this event,
which includes great presentation facilities, and will also be
supplying drinks and nibbles. There is also a pub below the venue where we can move to after the event.

Sun Microsystems, Regis House, 45 King William Street, London, EC4R 9AN

Note

There are only 100 tickets, and event registration will close a day before the event, so don't wait too long to book your place. Registration for this event is though the Eventbrite system.

Please ignore the registration button below

Wales, UK

Sourceforge 2008 Community choice award finalists: Best project,for Enterprise, for Educators, Change the world, for Developers

Amazon — Thu, 03 Jul 2008 17:11:28 +0000

Drupal has been selected as a finalist for the SourceForge 2008 Community awards in 5 categories.

Special thanks to Nicholas Thompson, Michelle, Rahul and others in who were involved in the nomination and promotion of this award: http://drupal.org/node/267469

Marketing of Drupal

Enterprise 2.0 Expo/ Enterprise2Open - Free to attend

Amazon — Mon, 02 Jun 2008 16:46:24 +0000

http://www.enterprise2conf.com/
Demo pavilion pass are available for free here: http://www.enterprise2conf.com/exhibition/demo-pavilion.php
Tuesday, June 10
11:00 AM – 1:00 PM
4:30 PM – 6:30 PM

Wednesday, June 11
12:00 PM – 2:00 PM
4:30 PM – 6:30 PM

Enterprise2Open unconference: http://www.enterprise2conf.com/conference/e2open.php
If you are interested in presenting at Enterprise 2 Open you should be able to sign up at the event itself.

Enterprise

PHP/MySQL Developer | CEMPER GmbH

Christoph C. Cemper@drupal.org — Thu, 29 May 2008 14:49:10 +0000

CEMPER.COM is seeking a

Web Developer

with
strong software development skills

To join our growing team and help improve and expand the company's internal internet tools for search engine optimization, competitive seo analysis and link building tools.

We are seeking a seasoned professional who is capable of consistent
production of quality software in a fast-paced but relaxed environment - be
ready to work hard, but enjoy yourself in the process.

The ideal candidate will have 3+ years of full-time
development experience with PHP, and a working knowledge of (X)HTML, CSS and
Javascript. An understanding of the Web 2.0 development standards, database
architecture and clean, maintainable code will be crucial to success on this.
PHP Development experience must include usage of version 5.x and its object
oriented features. Serious experience on MySQL is required; candidates must
know how to both write, maintain and optimize queries with multi-table joins.

The following core skills are sought:

• PHP
• MySQL
• AJAX
• Web 2.0 concepts, dynamic user interfaces
• Version Control knowledge (CVS & Subversion)
• Great English skills
• Used to work with request trackers (i.e. Mantis, BugZilla)
• Used to work driven by english specification documents

We offer:

• Enjoy working in a growing team in a dynamic, fast-paced and exciting industry
• Work in a company like there’s no other in Austria – even among the EU and USA only very FEW do the things CEMPER.COM does (hence we have so many happy US and UK clients!)
• Super-Flextime
• Inspiring 17th floor water views of the wonderful nature of the Alte Donau and Gänsehäufel – it’s almost too nice to work here!

Your Salary:

Salary depends on experience but is highly competitive.

More Details at http://www.cemper.com/jobs/wd5.htm

Find more about our company CEMPER.COM Internet Marketing Services
and even more details of our online marketing efforts online marketing blog
which also has a german/austrian internet marketing weblog counterpart

Enterprise

Vote in Enterprise 2.0 Launchpad contest: Round 2 with a new Drupal ecosystem section

Amazon — Thu, 22 May 2008 19:53:49 +0000

Acquia would like to thank everyone who voted for our 1 minute video submission for the Enterprise 2.0 contest. We've now made it to the second round and we've added an important section to the video, the Drupal Ecosystem, with dozens of Drupal consulting company logo's.

How did we select these logo's to represent the Drupal ecosystem:
1) Looked for sponsors of Drupalcon Barcelona and Drupalcon Boston
2) Drupal Association Organization Members
3) Organizations that registered on acquia.com as building sites for third parties.

As you can see at 1:48 in the video, the slide is pretty full. With your vote, we can make it to the Launchpad finals in Boston June 9th-12th and this will allow us to further expand on the vendors in the Drupal ecosystem with a longer live presentation. Thanks to everyone who has giving us feedback on how Acquia can fairly promote Drupal and it's partners in these kinds of events, we are listening, learning, and adapting to your suggestions.

Kieran Lal
Drupal community adventure guide
Acquia, Inc.

Marketing of Drupal

Xapian Search for Drupal - Metrics and Benchmarks

techsoldaten — Wed, 14 May 2008 01:38:23 +0000

I posted some internal benchmarks from the Xapian module we are working on in the Trellon.com blog. This post describes the performance gains using Xapian can have and does some benchmarking on a really, really slow server.

Would love it if someone could verify those results. We are about to take the sample size of the data up to about 1mil nodes and repeat.

Vote for Drupal in Enterprise 2.0 Launchpad contest

Amazon — Fri, 09 May 2008 19:46:08 +0000

Acquia has submitted a 1 minute video about Drupal to the Enterprise 2.0 Launchpad contest. The video site, is actually a Drupal site. If you get a chance to watch this one minute video and vote, it will help give Drupal some exposure at Enterprise 2.0.

http://launchpad.enterprise2conf.com/node/14

Marketing of Drupal

Selenium and Drupal

Souvent22@drupal.org — Sat, 03 May 2008 16:03:04 +0000

First, for those that don't want to read the full post, here's the "30-second elevator speech":
This post to to discuss using Seleinum with Drupal. Specifically using Maven to run the selenium tests and writing the Selenium tests in Java.
The example code be downloaded from the Workhabit Inc. public repository here: https://svn.workhabit.com/svn/public/drupal/selenium/trunk
One must have the following installed to run:
1. Firefox
2. Java 1.5+
3. Maven
All 3 are easy to install and aquire. Once you download the code from the repository, you can just navigate to the directory and run:
mvn -P drupal integration-test
Maven and writing tests are beyond the scope of this blog post.
I'm writing this up for comments, thoughts, and hopefully for others to use and/or contribute.

Ok, now the long winded version:

I've been in the Java world for a bit and I'm now bringing my self back into the PHP/Drupal realm.
With that said on my last project we had to validate ALL code with unit tests, and then we did fully functional integration tests using Selenium.

The Build management tool we used was Maven ( way past the scope of discussion here ). However, what I have done is GREATLY simplified down what we used into a very short, simple, and small file which allows one to run Selenium tests out of the box. It uses Maven/Java to do the tests...but, that's neither here nor there.
It is available here: https://svn.workhabit.com/svn/public/drupal/selenium/trunk

I know that there is a Selenium module for Drupal...however when I thought about this, I also felt that this was an inherit problem. This dawned on me when I read the link that there is a patch to port it to 5.x. There in lies the problem. As we move versions/etc., we're updating the testing frame-work instep with the item we're testing. E.g. PHPUnit may be at 2.3, and we can still use PHPUnit 2.3 for Drupal 4.7, 5.1, 6.x, etc. However, by tightly integrating the testing frame work into the item we are testing, I feel one is creating a cyclic-redundancy in which the end result is chaos.

Goals

My only goals of this project is 2 fold:

Give a solid base so that anyone can download the drupal-seleinum project and write tests for their drupal site.
The community should have an entire suite of tests to test the Drupal installation as a base install ( I have currently started these in the com.workhabit.drupal.selenium.core package. Inside the package, it think they should be split into their subsystems, e.g. Authentication, Taxonomy, CCK, etc.).

Why Java/Maven?

Well, there are a number of reasons; I will not list/discuss all of them here as they are not really part of this post. But on the short list; Java allows one to set break points where Selenium is running to see why a test may be failing ( it should be noted you can also do this with PHP; if you setup your IDE correct and get xdebug installed/setup correctly also).
Also, by using Maven, one doesn't have to worry about dependencies with the unit testing suite (versions, etc.) as Maven takes care of all of that. Java/Maven will also allow us to do more in terms of continuous-build integration. Further, writing tests in Java is not that much harder/different that writing them in PHP. The Selenium IDE gives code in many languages (HTML, PHP, Java, Ruby, etc.) so anyone should be able to create tests easily. I'm hoping to write a post of how to write/add tests in Java. Most of the hard work has already been setup in the code, so all one has to do for a new test pretty much is create a new class and create an annotated java method, such as:

@Test
public void myDrupalTest() {
  BROWSER.open("/path/to/my/site");
  // ....other browser test code from Selenium IDE
  // ...Assertions
  return;
}

Also, in the coming days, I shall be adding to this suite a "MockSoapServer". This will allow you to hit a webservice in point (in this case the MockSoapServer), and it will return "canned" results. Maven will fire up both the Selenium server AND the MockSoapServer BEFORE running the test suites. It will then create a report and take down both after the tests are run.

Environment Setup

First things first, you will need to have Maven installed on your local machine in order to use the Drupal-Seleinum tests. Installing maven is very simple and straight forward. See the Apache Maven Site for more information on installing Maven.

Setup your Tests

So, lets be real; Usually when we are developing, we are changing some of the default workings of Drupal; So we will need to have a some-what custom test suite for each site we develop; or at least we will be extending the tests with more/new tests.

*EXPORT* the base Selenium maven build from the Workhabit repository.
svn export https://svn.workhabit.com/svn/public/sandbox/earnest/drupal/drupal-selenium
Update the POM (If you are unfamiliar with Maven, ignore this step)
You can update the POM if you want to better describe you project.
Modify the "setup.xml"

You will notice when you download the code that there is a file called "setup-example.xml" in the src/test/resources folder. This file is read by the test suite and contains some general settings regarding your site. The settings are pretty self explanatory. The main ones you will need to change out of the box are:
- siteRootPath
- siteDomain
- testUsername
- testPassword
I will hopefully be documenting the settings further, but for now, I apologize that the documentation is a bit sparse.
Test to see that you have everything setup correctly by running a maven integration test (Note: All tests are done with the drupal plugin)
mvn -P drupal integration-test
This should run the DrupalLoginTest which will bring up your Drupal site, and login a user.
Add the package to your own repository.
svn add drupal-selenium && svn commit -m "Adding the drupal-selenium package to my site"
You now have a customized maven/Selenium test suite for the project that you are working on.

Usability

Site architecture patterns, Install Profiles, & automated site configuration

ChrisB — Tue, 04 Mar 2008 20:15:47 +0000

EDIT: There will be a BOF discussion tomorrow, Wed morning at 9:00am here at Drupalcon. Please join the discussion and share your thoughts.

Setting up Drupal sites requires a lot of time clicking through forms and redoing common tasks. This can be automated and reused to save time. Install profiles were built to help solve this problem, but there are limitations. We'd like to discuss these limitations and figure out what can be done to address them and plan the next steps and evolution of automated site configuration? Also, what needs to be done or changed in Drupal core to facilitate this?

Defining site architecture and best practice patterns

It would be great to be able to define common and specific site architecture and best practice patterns in a format that is easy to read and work with and can easily be shared and collaborated on. Ideally the system would support different backends to allow site configuration documents in multiple languages.

Projects and implementations

Install Profiles (http://drupal.org/project/Installation+profiles)
Install Profile API & Profile Wizard, CRUD (http://drupal.org/project/install_profile_api)
Profile Generator (http://drupal.org/project/profile_generator)
Civic Space APIs (https://customprofiles.civicspaceondemand.org/api/file)

Related Projects and discussions

Cocktail - CCK Type Language: (http://raincitystudios.com/blog/cocktail-cck-type-language)
GHOP - Drupal automated staging toolkit project proposal: (http://code.google.com/soc/2007/drupal/appinfo.html?csaid=AD069F51C0E756...)
Distributions Group: (http://groups.drupal.org/distributions)
Effectiveness of install profiles?: (http://groups.drupal.org/node/6820)
Drush: (http://drupal.org/project/drush)
HostMaster2: (http://drupal.org/project/hostmaster)
AutoPilot: (http://drupal.org/project/autopilot)
Multisite Management Tools Review: (http://drupal.org/files/issues/review_0.pdf)

Limitations in Install Profiles

Here are some of the limitations with install profiles that would be great to address:
It's a one time initial setup and can't change anything on the site after the initial configuration.
You can't create separate, smaller groups of features and configurations that can be combined together and mixed and matched individually or as a group.
It can be hard to read and work with since it's in PHP code and so there is a disconnect between site planners/architects.
It's not possible to setup all the desired settings/configurations (ie., API and additional module support.)
It's hard to manage changes to the profile or setup as requirements and best practices change (which they constantly do.)
Can't easily have slightly differing configurations (would need a separate install profile for each.)

Proof of concept solution

We started work on a project called Patterns that attempts to address and solve some of these points and limitations. Our idea was to make a simple, flexible, powerful custom module/system to handle definitions of site configurations that was easy to change, update and manage. We decided to create a module that supports definitions of requirements (patterns of any type) and runs/enables them on an new or existing site. We also created a very simple install profile that's sole purpose is to set up the Patterns module.

It's in an early proof of concept stage. You can find the project placeholder on Drupal here:

http://drupal.org/project/patterns

Here are some design ideas and implementation details:

Overview

The patterns module will enable the setup of complex and detailed sites without actually doing any of the setup yourself. A pattern will be a collection of rules, definitions, and dependencies that will automatically setup and create simple or complex features for you. Patterns allow you to define features and functionality for a site in a standard text format (currently XML with planned support for other formats,) or directly on the site, and then apply them to an already running site. An administration page will let you enable (and possibly disable or remove) each pattern that has either been provided by default, contributed, or created specifically for the application at hand.

A typical example would be to easily and simply setup a fully functional forum with user registration, authorization, avatars, comments, captchas, views and more without touching a single setting. Of course the unique, specific settings for the site at hand can be adjusted like usual, however basic settings can also be tweaked when turning on the pattern.

For those that need more control over the setup of their desired features, patterns could possibly guide the user through all of the setting pages and settings that are available for a particular feature set. They could have stop points that prompt the user for a specific choice or input (similar to Photoshop actions.)

Patterns primary uses

Define reusable site architecture patterns and best practices that are easy to update and maintain.
Quickly setup and configure new (and existing!) sites based on site architecture documents/patterns. (Next evolution of Install Profiles)
Migrate changes, updates and new features from you local development environment to dev server to QA site to the final production environment. (Change Management)
Possibility of making a specific pattern to update/change a site(s) from some other known configuration (legacy best practice to current best practice for instance)

Architecture Overview

Base install profile (setup Patterns module)
Patterns Drupal module
Patterns component modules
Hooks/API
Pattern XML files
Stop points for allowing the user to set/change a value
Pattern groups/subgroups

Most patterns will be made up of a combination of any/all these components:

Module(s)
Module settings
CCK Content Type(s)
Fields
Field weights/order
Views
Type
Fields
Panels
Blocks
Permissions
Content/Nodes (not finished yet)
Files
Other patterns!

Features:

XML based. Keeps it open and flexible so you can possibly use other systems to process or work with the XML
Creation of content types and fields
Creation of views and settings
User and node creation
Setup of user roles and permissions
Automatic and assisted panel creation
Automatic and assisted menu setup
Basic conflict checking and resolution
A pattern can be a list of other patterns
Support for tokens from the Tokens module

Challenges

XML Based (other formats may be more readable or better to work with.)
Keeping component modules up to date
Resolving conflicts in patterns and pattern dependencies
Getting parts of this into Drupal core

Primary plans and ideas for the future

Nicely formatted view of the pattern live on the site (currently it just displays the raw XML)
Extensive pattern administration:
Pattern builder: choose configuration components from an existing site to make a pattern out of specific parts (or the entire site)
Import from another site/file
Export to another site/file
Record patterns when setting options or configuration
Stop points (to ask the user for an input or value)
Pattern versions and dependencies
Display and automatic downloading of patterns from a central repository
Run/control from the command line (Drush integration)
Add/Import by URL (with ability to choose from a list of available ones)
Publish/Export list of patterns to URLs, Master site (enable repository functionality) and individual patterns
Repository system (list of available sites to check for patterns)
Ability to easily add your site automatically to the repository list
Version checking

Possible other side benefits and solutions

Central or distributed pattern repositories with community managed best practice patterns
Unified API for configuration import/export
Change Management solution
Development shops can share common configurations and work together to improve them.

Sorry for the overload of information here. We'd love to hear your thoughts and feedback and work together to create a solution is properly planned and can hopeful integrated with core for Drupal 7. What do you think?

Enterprise

Potential RDF use cases for Drupal

arto@drupal.org — Tue, 19 Feb 2008 04:50:56 +0000

Following up on my previous post introducing the in-progress RDF API for Drupal 6.x, I'm going to be, bit by bit, posting some of the materials that I've earlier put together internally for the project this is being developed for. Development of this project is supported through M.C. Dean and MakaluMedia on behalf of their clients.

First, some requisite background. Our use cases for RDF evolved out of several years of building a communications and collaboration platform serving the needs of an international community of intergovernmental agencies and organizations. The project as well as the potential applications described below reflect Ivan's and Chris's ideas and vision.

This environment presented us with a variety of challenges, such as deploying the application across multiple discrete organizations that operate under little or no centralized authority, supporting a wide range of operational scenarios, and finally managing or brokering large volumes of context-sensitive information delivered via a variety of protocols from a multitude of sources, in formats ranging from web-native to rich media files to device specific data streams.

Development of many well-known Drupal modules has been funded through this project while tackling these challenges; examples include OpenID, OpenSearch, LDAP provisioning, Timeline and Boost. Even more modules are yet to be released as open source, including e.g. a WebDAV server module and embedded instant messaging and webmail for Drupal.

Our platform is currently based on Drupal 4.7.x, and is probably one of the most complex Drupal systems around: we're making use of over 130 contributed modules and have a total code base size of nearly 600 KLOC (not a point of pride). Moving onwards to Drupal 6.x, our goal is to radically simplify the system by standardizing on RDF to allow for more precise expression and efficient sharing of information, and to utilize Exhibit as the central technology driving the user interface.

I mention all this to make clear the context of our use of and interest in Drupal and to illustrate why the use cases that follow are necessarily of an "enterprisey" nature, dealing with reducing the complexity of our system design, increasing its interoperability with external data sources, and improving the usability of our user interface for dealing with large quantities of multi-faceted information.

Here to the right are two slides from a presentation at our recent developer meeting (click on the images to see the full-sized version on Flickr).

They depict (on a high level) a system architecture enabled by the RDF support for Drupal, tying several Drupal instances together into a federated system via RDFbus (publish/subscribe messaging for transmitting RDF payloads over various transports including XMPP and Stomp) and SPARQL.

Most significantly, this loosely-coupled architecture provides a measure of technology neutrality which allows individual system components (i.e. shards of the datagraph) to be replaced with other interoperable systems, ranging, for example, from high-productivity web frameworks such as Rails or Django to legacy J2EE systems. Because, frankly, Drupal isn't always the right answer regardless of the question.

I will need to write up more public details on RDFbus, later, but here follow some initial quick thoughts on potential specific RDF API-based use cases for Drupal 6.x - the low-hanging fruit, if you will. The list ranges from the obvious to the speculative, in ascending order.

(Note that if any of the preceding was all greek to you, there may be something more concrete in what follows; generally, though, I should warn that if you haven't previously heard of RDF, or know RDF only from the context of RSS, then reading at least this 10-minute introduction to RDF is pretty much a requirement for making any sense of this stuff... other introductory materials, including videos introducing these concepts, are linked to from the RDF API's developer documentation.)

Generic metadata storage

Many (most?) Drupal modules need to store metadata in some form or another. Until now, each Drupal module has needed to implement its own metadata storage in an independent, redundant and incompatible manner.

In practice, Drupal's configuration variables have also come under widespread abuse for purposes of storing metadata in situations where developers needed to describe information but felt that creating (and in the future supporting) a whole set of custom specific SQL table structures was too heavyweight a solution.

Drupal clearly would benefit from a generic solution here, and indeed implementing unified metadata storage in Drupal has been discussed on the development mailing list several times in the past. The RDF API provides a solution to this with sufficient generality to cover any specific use case.

Faceted browsing

Exhibit displays structured data in the form of rich visualizations that can be searched, filtered and sorted using faceted browsing. Exhibit is the perfect user interface complement to the flexibility that RDF brings to play on a systems level.

While the possibilities are really endless, some of the more obvious applications of Exhibit may include navigating the user profiles by their organization membership, expertise, and social relationships; filtering event data by subject, audience, and timeframe, and graphically representing the events on both a map and a time grid; combining full-text search with taxonomy, group, and user- based filtering to locate content; and others. All the preceding using a common, consistent set of data manipulation and visualization tools.

This is one of those cases where a demo speaks more than a thousand words. If you haven't seen Exhibit or Potluck in action, go do so now. The Exhibit module will bring these abilities to Drupal through the means of an Exhibit-SPARQL bridge.

Some sneak peeks of the module performing its magic are available on Flickr.

Utilizing open, linked data

Utilizing the RDF API, the openly available big datasets can be used to enhance mashups built on top of Drupal.

For instance, any factual information which Wikipedia has can be retrieved or copied from DBpedia. As a trivial use case: instead of manually importing, say, the list of countries into a Drupal instance from a CSV or XML file, just pull the equivalent triples from DBpedia or the CIA World Fact Book using a single SPARQL query. Along similar lines, the GeoNames dataset has information on 6.5 million geographical location worldwide, including facts on every city and town; there are sure to be some intriguing possibilities here for spatial mashups.

From modules to widgets

The RDF API for Drupal will include a jQuery API for querying and changing the metadata stored in the Drupal instance. This opens up a plethora of new possibilities.

Even today, many UI-oriented Drupal modules live almost wholly on the browser side, maintaining only a tenuous, feeble server-side implementation responsible for taking care of persistent storage needed by the client-side functionality provided by the module. With the introduction of unified metadata storage for Drupal, it is conceivable to cut this server-side cord altogether; this is in line with post-Web-2.0 trends which are seeing more and more UI logic being transitioned from the server into the browser.

A perfect case example is the Image Notes module. The sole purpose of this module is to dynamically attach an annotation widget to photos displayed on a Drupal site, allowing users to provide descriptive text for bounded regions of the image (e.g. marking up the names of the people depicted in the photo). Using the jQuery API for storing annotation metadata, this component needs no further customized server-side code, meaning that it can become either an extremely lightweight Drupal module, or perhaps even be directly incorporated into the Drupal site's theming layer without any need for an actual Drupal module per se.

In general, widgets like this could conceivably be faster and quicker to develop than Drupal modules, be more amenable to quickly changing requirements, and not require any Drupal-specific expertise on the part of the developer - and many widgets could indeed work with any backend application as long as a compatible jQuery metadata API was provided.

RDF export functionality

With the introduction of the RDF API, every Drupal resource now has an associated machine-readable RDF description available via auto-discovery. If desired by the administrator, all the data stored in a Drupal instance can be exposed this way. This supplants previous solutions such as the Import / Export API.

(Implementing RESTful POST, PUT and DELETE semantics - perhaps interoperable with AtomPub - for uploading changes in RDF format is beyond the current scope, though; RDFbus covers much the same ground, but using an asynchronous messaging paradigm.)

Publish/subscribe aggregation

RDFbus obsoletes the existing XML-RPC-based Publish/Subscribe solution for Drupal, goes one further by not being tied into specific content types or data, and potentially enables notification and synchronization with any client application or device which can talk XMPP and understands triples.

Drupal vocabularies, categories, user accounts, groups, pages, blog posts, events, files; RDFbus is content-agnostic and will transport anything which can be described in RDF.

Distributed identity and trust

On the one hand, RDF allows describing user/group equivalences between two Drupal instances; on the other, RDFbus allows the same physical user account to be synchronized between these instances for purposes of Drupal modules that aren't RDF-aware. In addition, FOAF relationships between user accounts, groups and sites (user-to-user, user-to-group, group-to-site, and so on) provide the basis for a distributed network of trust.

This might provide similar functionality as LDAP, but in a manner that is decentralized, happens over the cloud, and is FOAF-compatible. (Note that OpenID is orthogonal to this setup; each user merely needs a URI, and that can equally well be provided by OpenID or something else.)

Distributed search

RDF allows mashing up any information about anything. SPARQL has built-in support for federated queries over any number of RDF data sources. OpenID + FOAF provide a global URI-based identity with trust metrics for access control and personalization purposes.

Something might perhaps be built on top of SPARQL that would not only obsolete OpenSearch, but make it seem crude in comparison.

Aggregated views/visualizations

Think of the benefits that the Views module provided for individual Drupal instances, and imagine something that takes this concept and runs with it a little further: aggregated for any number of Drupal instances or other RDF-aware data sources, and based on SPARQL instead of SQL.

Imagine what you'd want to see, in a tabular format or otherwise, and if the source data for it exists on the local RDF cloud, a SPARQL query can be written to traverse the graph, transform the results, and pull up the data for visualization purposes.

Given the aforementioned paradigm shift to widgets, this yet-to-be-named component (Sparkling Views?) could support richer, higher-level output than what the Views module does. Timelines, sparklines, piecharts; you name it, and if a widget exists, it could be plugged in.

Global hub/dashboard

Given all the facilities detailed above, a next step would be building a dashboard-like central hub interface for monitoring real-time activity in all instances (over the RDFbus) and performing aggregate queries (using SPARQL) over the full datagraph for e.g. obtaining metrics and analytics while respecting local privacy concerns. This would be useful both for management and system administration purposes in an organization relying on a distributed Drupal platform.

Semantic Web

Some review and proposal about Drupal 7.x database stack

hswong3i — Fri, 15 Feb 2008 06:40:02 +0000

I start my Drupal + Oracle research since the end of Drupal 4.7 life cycle (around Oct 2006), based on my client's request. The project is still running, but I am not satisfy about its progress. After keep trace in Drupal database stack implementation for more than years, it is time for focusing on Drupal 7.x development.

So what are the changes since 4.7.x? And what will be happened for 7.x? I would like to share my research progress with you, and so let's brain storming for what's next :-)

Drupal 4.7.x/5.x

Fig 1.1 Drupal 5.x database stack.

The case of Drupal 4.7.x and 5.x is very similar: we support both MySQL and PostgreSQL backend, with specific database access API file (e.g. database.mysqli.inc), enclosed with database.inc, and that's all. We still have some extra abstraction build on top of this simple model, e.g. tablesort.inc and pager.inc, for some useful application layer abstraction.

During this period, core and contribute developers need to understand the differences between MySQL and PostgreSQL, since database schema must handle manually. I would like to call this as the dark age of our history: we are asking every contribute developers as both MySQL and PostgreSQL expert (but most of us coming with MySQL-only skill set), or else code was not portable. This result in a complicated PostgreSQL support, and so most contribute modules are MySQL only.

Drupal 6.x

Fig 2.1 Drupal 6.x database stack.

For our newest stable Drupal release - 6.x, Schema API is being introduced. We spend a lot of time for its development, during 6.x 1-year development life cycle. It is target to solve the schema problem as mentioned above. Contribute developers are no longer required as bi-expert nor multi-expert: we just need to know how to define our schema in a correct Schema API syntax, Schema API will handle the rest for us. IMHO, it is very powerful, and it is an important improvement for other database development.

drupal_write_record() is being introduce, too. This handy function will grep out existing cached schema, and build INSERT/UPDATE queries for us automatically. It try to handle all column type binding for normal developers, which is usually very bulky in implementation.

BTW, the work is not too elegant: all stuff are mixed into database.*.inc, and so files are even split as 2 section (groups database and schemaapi) internally. This may increase the difficulty for our core maintenance.

Most likely, this design is very similar as Moodle 1.7+ XMLDB implementation.

So what's coming next for Drupal 7.x?

Fig 3.1 Well... So what's coming next for Drupal 7.x?

The truth is: I don't know the overall schedule up to this moment. Data Architecture Design Sprint group is now hard working for this, and going to present their research progress in Boston DrupalCon 2008. Since we are going to use PHP PDO (instead of legacy PHP database connection drivers), people are also focusing on implement Data API or Active record model for a higher level of abstraction. Before they have their final decision, I would like to provide some successful case study as reference, and let's brain storming for the optimal solution :-)

Fig 3.2 Moodle 1.7+ XMLDB stack. From doc.moodle.org.

When we are talking about improve our database abstraction as more powerful and universal, Moodle 1.7+ XMLDB stack should act as a perfect example (it since Nov 2006). As Moodle core uses ADOdb internally, this possibility has been present since the beginning. Plus their XMLDB abstraction, Moodle 1.7+ is able to handle both MySQL, PostgreSQL, Oracle and MSSQL while maintaining everything working properly. I will not detail their implementation within this article, and you may refer to their introducetion, roadmap and problems document.

The most important idea of Moodle's XMLDB stack is split-brain handling. They left every layer keep focus on their duty ONLY, and so the final approach is perfect and elegant. This is also similar as the well-known OSI model: A layer is a collection of related functions that provides services to the layer above it and receives service from the layer below it.

Fig 3.3 Research progress of Siren + personal suggestion.

Well, Siren is my personal research project besides Drupal 6.x, which target to explore the possibility of multiple database support as like as Moodle, Gallery2, and eGroupWare. Moreover, prepare research progress whenever Drupal 7.x is open for public development (and that's now). Within this project, we are able to support both 5 database + totally 9 PHP database access drivers, including mysql, mysqli, pgsql, oci8, pdo_mysql, pdo_pgsql, pdo_oci, pdo_sqlite and pdo_ibm.

It sounds interesting, isn't it? So what changes I have been made in order to support this approach? Just some step forward improvement: split the handle of function calls into number of individual files, and each layer just keep focus on what they are needed for. E.g. split Schema API related functions into schema..inc, common DML into common..inc, and so on. Surly I plug some extra update to our core queries syntax; but most likely, the idea is just very strict forward. The research progress should belongs to OSI session layer when compare with OSI model.

IMHO, Data API is a must for our future, if Drupal is going to be a web service framework. Acting as a high level abstraction should be the best position for it. It should be similar as the idea of OSI presentation layer, which is one level below our core application APIs. It may implement as OOP or whatever; BTW, according to its duty and responsibility, it should be separated from PDO database access API (where PDO is much similar as OSI transport layer implementation).

One statement as conclusion: split-brain handling.

Any else suggestion?

Database driver is something only need to be implement once and forever. Once the design of logic is completed, the rest is all about the style of implementation, e.g. using OOP as ADOdb or traditional function callback as Drupal.

The design of abstraction layer is also similar, too. Once you get a complete study of the variation between each databases, and able to foresee ALL of the needs of their specific limitation, it is all done. There is no such idea of "Contribute Supported Database" within the above 2 layers, and it is what I call as "No gray area, but only black and white". Whatever you support it, or else you don't.

On the other hand, Data API is much different. It try to serve the need of our application, and so its requirement will keep on changing depend on time. Therefore a split-brain handle should be the most suitable implementation. Please correct me if you have some better idea :-)

But the truth case is: we are lack of PostgreSQL/Oracle/DB2/SQLite/etc developers, and not enough task force nor union for a complete and perfect support for them. Up to this point, I would like to proposal the idea of "Contribute Supported Database" in this way:

Implement ALL possible database drivers into core, and build an abstraction that cover ALL the needs of them. This is a once forever task, and the base of the rest of discussion.
ALL core libraries and essential modules for installation MUST implemented as universal, so basic installation MUST function for ALL databases. E.g. node, comment, users, block, etc. This is all the duty of our core developers which need to be ensure before stable release. On the other hand, if MySQL (and maybe SQLite, too) is our main focus, just keep its support as perfect as usual.
P.S. an additional pre-requirement for this point: try to implement ALL core module as SQL-99 compatible by default, and so save a lot of time for further more follow up handling.
Label the optional modules as "Not supported" or "Not perfectly supported" within modules selection page, if that is the case of specific database backend; give a different level of support status; or maybe even, disable its activation checkbox by default. Don't let our second/third priority database hold our stable release!
left the issue open for contribute developers if they really hope to let that module function for database A/B/C/etc, and release subversions besides stable release for this purpose. E.g. if now search and forum module are not compatible with Oracle, just let it be as Oracle developers duty!

This is my ideal proposal. So how about yours? Here is the raw file of above images, please feel free to utilize it if you hope to visualize your idea as above :-)

Postgresql

DB support as contribute: is it a good idea?

hswong3i — Wed, 13 Feb 2008 08:45:44 +0000

Abstract

Drupal 6.0 is revamped with Schema API, so what's next for Drupal 7.x? PDO for sure! With this powerful data-access abstraction layer, workload will much reduced for DB abstraction layer designers and developers, and finally benefit our contribute developers and end users.

By the way, together with the decision of Drupal 7.x + PDO, there is also some voice about moving PostgreSQL (and so other potential databases support, e.g. Oracle, DB2, MSSQL, etc) support away from core, but contribute; on the other hand, add official SQLite support into Drupal core, together with MySQL.

Is this really a good idea? Or even if it is possible? As an existing Drupal + PostgreSQL users, what will this affect your daily work? As a potential customer of Drupal + Oracle/DB2/MSSQL/etc, is this a good new for you, or just an evil? I would like to provide some brief idea for you within this article.

Background

After more than 6 years of PostgreSQL supporting in Drupal core, PostgreSQL users are going to have a revamped experience with Drupal 6.0. Drupal 6.0 will ship with Schema API, which simplify most of the variation between databases handling - DDL. This useful improvement will not only benefit our existing core and contribute developers, but also simplify most workload for other database driver development, besides MySQL and PostgreSQL. With Drupal 6.0, we can simply foresee the expand of existing market sharing, together with the explore of hidden customers.

As Drupal 6.0 is ready to launch, it is also a good timing to plan for Drupal 7.x. As we will only support PHP 5.2.x or above in Drupal 7.x, it is for sure that we will revamp our existing DB abstraction layer with the help of PDO. It does simplify and standardize a lot of function call implementation, together with some useful new features. If you are a DB abstraction layer designer and developer, moving your work to PDO will save you a lot of time.

On the other hand, some people also think it is a good timing for adding Oracle, SQLite, DB2 and MSSQL support into Drupal core. Most of this request are halted for 2~3 years.

Some of the developers are hard working with the future plan for how Drupal 7.x DB handling should looks like. BTW, the idea is most likely for expand Drupal usability and functionality, but not about maintain nor expand different databases supporting: some developers even think we should only official support MySQL and SQLite with Drupal 7.x + PDO, but move away PostgreSQL (nor other potential databases) supporting as contribute level.

What PDO Does and Doesn't?

For finding out why this brain storming happen, we need to have some idea about what PDO does and doesn't:

PDO is a data-access abstraction layer, but not provide a database abstraction. E.g., this means we can simply forget the function API differences between mysql_connect(), mysqli_connect() and also pg_connect(): we just need to construct a new-standardized PDO object with different DSN (Data Source Name). Moreover, we can forget those complicated programming logic differences between pg_escape_bytea(), OCI-Lob->load() and db2_lob_read(), but simply using standard Stream API as PDO LOB (Large OBject) handling.
BTW, these are most likely abstracted by Drupal 6.x DB API, e.g. db_connect(), db_query(), db_decode_blob(), etc.
PDO provide prepared statements and variable binding, even it is not supported by database, e.g. MySQL. With PDO variable binding, we may use ? or named-variable (leading with :, e.g. :value) within prepare statement. This seems to be a good idea for replace our existing user-space printf-syntax (e.g. %d, %f, %s, %b and %%) as PDO's C/C++ implementation, and simply forget the relationship between data, data type and database column type. This also give a great hope to Drupal 7.x Data API research and development.
So isn't it perfect? No, it is not... First of all, most databases require special LOB handle during variable binding (specific PDO::PARAM_LOB manually), e.g. PostgreSQL, Oracle, DB2, etc. Only MySQL and SQLite are able to handle variable binding without type specified. Also, there is no spotlight performance different, even we swap queries variable binding from user-space printf-syntax to PDO, which is proved by benchmarking result.
PDO isn't a database abstraction layer, it doesn't abstract and standardize ANY database variation. E.g., PDO doesn't solve the problem of reserved words conflict, it don't abstract any SQL function name variation, it also have not duty with resolve the maximum string size limitation across most databases.
Let's take Oracle LOB handling as an example: PDO give none of help to Oracle LOB INSERT/UPDATE handling, we still need to use RETURNING + Schema API. According to my research progress and case of Oracle, using standardized function call is the only benefit of using PDO, but didn't help or simplify any programming logic or implementation.

IMHO, PDO is very useful for DB abstraction layer designer and developer: it do simplify most of the DB function call variation; BTW, when we are talking about "PDO can improve database abstraction and so software with PDO can support more databases", it is not really truth. PDO don't have duty with this ;-(

How about Drupal + PostgreSQL (or other databases) support as contribute?

It is for sure that MySQL own around 90-95% of our existing market sharing; it is not question that SQLite is a manifest typing database engine; based on the above PDO stuff information, MySQL and SQLite are both very suitable with PDO lossy-form variable binding technology (able to use ? and named-variable binding without type specified). But when we are talking about "How many percentage of PDO technologies are utilized by MySQL and SQLite?" and "How many percentage of technologies that PDO NOT covered are used by MySQL and SQLite?", I guess the answer may just below 70%...

So what will happen if we seem MySQL and SQLite as our first priority core supported database, and ONLY focus with them? For sure that we will miss out the rest of 30%, which means the basic elements needed by other databases, within Drupal database abstraction layer. This also means asking other database drivers implement as contribute is just a nonsense, if we only support a subset of requirement within core.

On the other hand, writing MySQL-specific queries also generate a lot of incompatible issues for PostgreSQL during Drupal 6.x development life cycle. People also claim that PostgreSQL issues slow down our Drupal 6.x release. BTW, there is still a lot of incompatible issues even PostgreSQL is now seems as our "Official Supported Database" in Drupal 6.x, we can simply foresee about the case if we just seems it as contribute supported. It is better say that: we are NOT going to support PostgreSQL anymore! And so the similar case for other databases driver development: we will have NO WAY to implement other database drivers as contribute support!

PDO + PostgreSQL + LOB handling should be a good example of this assumption. In case of pdo_pgsql, we need to specify PDO::PARAM_LOB manually during INSERT/UPDATE variable binding, and must use Stream API for BLOB decode. Both of these handling are NOT required by MySQL and SQLite. If our Drupal 7.x Data API design are target for MySQL and SQLite ONLY, it is for sure that NO WAY for PostgreSQL driver implement as contribute: we will have no hook for it!

As a simple conclusion, multiple database support MUST build inside core, with a complete research of what are they needed for. There is no shortcut, and no gray area. The support of database is just simply a TRUE/FALSE question. Please don't be faked by the beautiful wording of "Contribute Supported Database": it is just a sweet poison candy, that is not the truth case, and I have NEVER seen a successful story about this :-p

Oh my god! So what should I do for it?

If you are an existing Drupal + PostgreSQL user, it is strongly suggest to keep you eye focus on the Data Architecture Design Sprint group, comment your needs and wish to them, or else you may be fired away without notice. You may also have a look about issues related to PostgreSQL, give a hand to it if possible, and comment them as a Wiki collection. IMHO, it is a critical timing for Drupal + PostgreSQL besides the past 6 years support period. It is time for YOU to take YOUR action, or else will be too late.

If you are a potential customer or user of Drupal + Oracle/SQLite/DB2/MSSQL/etc, beside the above suggestion for PostgreSQL, I would also like to invite you to give a hand on some existing issues, review and comment them, and finally record your work in another Wiki tasklist. According to some research founding, we would able to enrich our database supporting without a critical pain of revamp, based on our existing Drupal 6.x DB API implementation. Most research are now completed, so issues review and comment your point of view should be your highest priority. Please remember that: comment is power, and YOUR review is very important for speeding up the core support of your target databases.

My humble conclusion

Every developers have their own background, their own wish, their own needs, and also their own rose garden. I would not like to comment this "PostgreSQL as Contribute" idea as silly; but at least, it is not my cup of tea.

On the other hand, supporting more database won't be a conflict with the research and develop of Drupal's Schema API or Data API: they are target for a better developer and user experience, serving database abstraction should be one of their core duties. Asking for better abstraction and functionality but reduce the number of supported database, or even close the door for other database should be count as loss focus.

From my point of view, more choice is always better than just giving me a single solution. The use and choose of database should depend on the needed of each individual case of clients and users, but not forced by developers. If we are already close to our target, there is no point for rollback our 6-years progress on tomorrow.

A beautiful-sweet-poisoned candy? No thanks!

Postgresql

Boston Drupalcon showcase and case study contest

ChrisB — Fri, 08 Feb 2008 22:08:28 +0000

Cross posting from here: http://drupal.org/node/219476

There will be a Drupal site showcase and case study contest at the upcoming Boston Drupalcon.

Submitted sites will be voted online and judged by a panel of community nominated judges and some great prizes will be awarded to the winners! This is a chance to have all your hard work creating amazing Drupal sites recognized.

Go on over to the Drupalcon Boston site to submit your favorite sites and view and vote for the best ones.

We'll also be doing a session to discuss popular, high traffic and enterprise Drupal sites and why Drupal is often the platform of choice so please join us.

Please post comments here to nominate judges you feel would be best suited to judging these high caliber sites.

Thanks and see you in Boston!

Marketing of Drupal

Issues Need Review for Enhance Cross Database Compatibility

hswong3i — Tue, 05 Feb 2008 08:48:16 +0000

Enhance cross database compatibility is important for our Drupal Core, especially when we are going to ship D7 with PHP5.2.x PDO implementation. BTW, this will require number of individual patches, so your help and review are required - add it to the top list with a brief description and an estimate of the time to review. Once reviewed, move the issues to the bottom "reviewed" list, perhaps with your name attached.

Need Review

http://drupal.org/node/371 - Resolve ANSI SQL-92/99/2003 reserved words conflict. (just a few minute, but need a lot of follow up)
http://drupal.org/node/199217 - Split Schema API and shared functions from DB drivers. (just a few minute)
http://drupal.org/node/167335 - Handle database-specific SQL functions/operators by simple abstraction. (just a few minute)
http://drupal.org/node/199101 - Core queries coding style cleanup for PDO development (should be RTBC, but need a lot of follow up)
http://drupal.org/node/218128 - Enhance _db_query_callback() for PDO.
http://drupal.org/node/183148 - INSERT/UPDATE/DELETE API, drupal_drop_record() and enhance drupal_write_record().
http://drupal.org/node/147947 - Replace some TEXT:BIG with BLOB, based on cross database compatibility concern.

Reviewed

Committed without Thorough Review

Postgresql

My personal battle target for Druapl 7.x

hswong3i — Fri, 01 Feb 2008 03:45:51 +0000

My primary battle plan should be enhance Drupal cross database compatibility. I have involved in this topic for around a year, and I would like to keep it on going. To accomplish this target, I would like to complete the following tasks before D7 code freeze:

Improve Database API for better abstraction
Something should be done before any other targets, e.g. resolve reserved words conflict, split drivers into individual files for better management, indeed but simple abstraction for different database syntax, add INSERT/UPDATE/DELETE abstraction and enhance drupal_write_record() abstraction, etc. With a better common base for multiple database development, we will need much less time to study the differences between databases than that of before.
Add PHP PDO supporting
Shipping D7 with PHP PDO should be a spotlight topic, since it is now official package for PHP5.2.x, and legacy drivers will soon be removed in PHP6.x. We may need some cleanup in our core query syntax, in order to let PDO get works.
Introduce more database backend
Up to this point, I would like to introduce some other database backend for D7, e.g. Oracle, SQLite, IBM DB2 and also MSSQL. If we are able to support SQLite in D7, which also means we are going to standardize our core queries into SQL92 standard, the problem for maintain multiple database backend should no longer a critical problem; if we are able to support Oracle, the most complicated database I guess, this should no longer a problem for us to implement drivers for other databases.

I have summarize most of my personal battle targets in here. Most logic are proved as functioning, and they are all get set for the open of D7 public development. On the other hand, I would like to explore if this may integrate with other interest musings with Data API, so we will able get all stuff better in D7 ;-)

Database Schema API

Time to Give Back: Become a Speaker at Drupalcon Boston 2008

jwhatcott — Wed, 23 Jan 2008 12:49:57 +0000

Hello,

We are looking for speakers for the upcoming DrupalCon in Boston. Since you are subscribed to a business and marketing related group, you might be especially interested in the "Business and Marketing" track (see http://boston2008.drupalcon.org/business-and-marketing-track-description... ).

This track covers the following topics:

The Drupal ecosystem
Businesses learning to use Drupal
Case studies
Showcases: NPO, Education, News, Media, Government, Healthcare

You can find a list of existing proposals at http://boston2008.drupalcon.org/sessions, and submit your own proposal on http://boston2008.drupalcon.org/node/add/session. Of course, it is at any time possible to submit custom session proposals which we did not yet cover on the list of suggestions.

We are looking forward to create an great business and marketing track with lots of benefits for all participants.

Thanks,

The Business and Marketing track co-chairs
Jeff Whatcott, Jim Edgett, Liza Kindred
http://drupalcon.org

Search Engine Optimization

PHP 5.2.5 breaks SQL Server support; Yet MSFT SQL Driver Works

Souvent22@drupal.org — Thu, 20 Dec 2007 18:32:32 +0000

So, I've had some free time latley, and I've started back on the database API layers (DB2 and SQL Server currently).
However, I have found 2 disturbing issues.

Overview
First, I'm currently exploring the following 3 ways to connect to sql server:
1. FreeTDS
2. php_mssql driver
3. MSFT SQL Driver ( See pcorbetts posting for a link to the driver )

PHP 5.2.5 breaks support for SQL Server
I'm running PHP 5.2.5. I've had a perfectly good working SQL Server installtion, so I thought this was just a PHP upgrade and then I was going to continue to do some module dev. and bug fixing. Well, it seems that with PHP 5.2.5 neither the php_mssql extension OR the FreeTDS driver work with PHP any longer. I have submitted a bug report to PHP ( 43645 ). This has been verified by another developer also.
MSFT SQL Driver works in 5.2.5...but still alpha
The MSFT SQL Driver DOES work in 5.2.5. However, the driver is still in alpha quality, and will prob. be Spring before I would say it's a stable beta ( reference the IIS FastCGI module for a MSFT timeline :) ).

It should also be noted that the ntwlib client ( this is the library that allows FreeTDS and the php_mssql libs to connect nativley to SQL Server using TDS [Tabular Data Stream]) is no longer going to be distributed with SQL Server. It's been deprecated. This kind of makes sense, given MSFT is prob. going to start branching it's product away from it's Sybase origins, so who knows what funky stuff they'll do with the TDS format. Anyway, I'm still trying to get my head around what direction they're going to take at this juncture. They must know there's a need for PHP+MSSQL connectivity, thus the development of the SQL Driver. I know that it is beta ( and as I've said before...definatly a TRUE beta, not a "google" beta :) ), but I'm confident this driver will get finished and supported...but I just can't say when.

With that said, I'm going to try and continue development by downgrading my PHP, and keeping an eye on the but I opened at php.net.
That's all for now, but just thought I'd give a heads up about PHP+MSSQL and esp. the 5.2.5 issue.

Enterprise

Obstacles For Real SQL Server Support

pcorbett@drupal.org — Wed, 12 Dec 2007 21:12:24 +0000

There are some major issues, in my opinion, that are preventing MSSQL from being fully supported in PHP and thus Drupal. Most, if not all, of these revolve around the terribly outdated DBLIB library - forcing some to use alternative libraries (FreeTDS or the recently released SQL Server Driver for PHP). I want to get this out in the open so that we may address them appropriately. I have run into these myself and have had to do some creative work-arounds to get Drupal to behave the way I need it to while running on SQL Server. As far as I can tell, this applies to all versions of SQL Server and the latest PHP release.

Once we can reach agreement that these issues exist, we can then work to address them. I looked at the new SQL Server Driver for PHP, but it has an entirely new set of functions that would require an entire re-write of database.mssql.inc, however, should we start developing for this driver since Microsoft has discontinued (to my knowledge) ntwdblib.dll??

At the very least, we should develop a to-do list for all of this, perhaps a wiki page so that we can coordinate our efforts effectively. I also plan on creating a separate list for incompatibilities between MySQL / SQL Server 2000 / SQL Server 2005 / SQL Server 2008 so that module developers have a place to go to help guide them in supporting 'mssql' in addition to 'pgsql', 'mysql', and 'mysqli'.

VARCHAR Data Truncated at 255 Characters

There is an issue where you can not pull more than 255 characters from a VARCHAR field. The fix is either to change this field to type TEXT (as I usually do) or, there is a suggestion to try: "SELECT CAST(F AS TEXT) AS F FROM ...", which I have not tried yet.

For Reference:
http://bugs.php.net/bug.php?id=25544&edit=1
Fetch Column Name Truncated to 30 Characters

This issue is apparent, but not limited to, CCK fields whose names in the database exceed 30 characters. This results in CCK field data not appearing in, for example, a view. Currently I have a really ugly patch that truncates the name of the field CCK is expecting to 30 chars so that it matches the truncated field name fetched from the DB. From php.net:

"Note: In Windows, the DBLIB from Microsoft is used. Functions that return a column name are based on the dbcolname() function in DBLIB. DBLIB was developed for SQL Server 6.x where the max identifier length is 30. For this reason, the maximum column length is 30 characters. On platforms where FreeTDS is used (Linux), this is not a problem."

For Reference:
http://bugs.php.net/bug.php?id=33060&edit=1
Extra Space Added When Pulling Empty Data

When pulling "" (empty) data into a form field, for example, the field has an empty space entered into it. The suggestion is to use a trim() function to strip these. This only happens when the data stored in the DB is empty (no spaces). I am currently experiencing this with nodes that use CCK fields, for example, and the ones that are not required and that I don't fill out have a single space in them when I go to edit the node.

For Reference:
http://bugs.php.net/bug.php?id=36968&edit=1

Enterprise

RFC: Best practices for managing indexes (particularly on cck tables)

greggles — Mon, 10 Dec 2007 21:21:43 +0000

As a site goes live and we do performance tweaking and benchmarking one of the common results is adding database indexes. I post this into the CCK group since one of the common sources of slow queries is for queries against the CCK tables. For example, a site that has a block that queries a field table to look for information about that field for every page load can quickly make its way into the slow query log. Adding an index to the field is likely to be the best solution.

I'm not criticizing the cck table structure, which I think is quite good, but instead asking for advice on how to manage indexes on database tables in general and on cck tables specifically. Assuming that the query is in custom code:

Add an index via a hook_update_n in the .install file for that module with a comment explaining why we added it
Add a comment near the query that says "If you change this query, check the index from hook_update_X to see if it still makes sense"

If it's not in your code but in another module (e.g. views) then step 1 probably stays the same, but I'm not sure where to document item 2.

I want the index to be easy to migrate from the development environment to test/production and also to be documented in a way that future admins can easily know where to look to see if it is still necessary.

Any feedback on this idea? What do other people do to manage indexes - aside from "just install them and forget them" ;)

Thanks!

PS yes, I know block cache is a decent solution to my original problem, but sometimes indexes are good too.

High performance

Suggested Method For Choosing Between SQL Server 2000 and 2005/2008

pcorbett@drupal.org — Tue, 04 Dec 2007 18:27:22 +0000

I have been working to get SQL Server 2000 to work with Drupal 5.3. Everything seems to be working except for a small pagination issue. As part of this, I have been prepping it for use with 2005/2008 as well, but wanted to share with the group to see if we should agree upon a common way to distinguish between different versions (since syntax and functions vary so much between them). I am assuming that there will be differences between 2005 and 2008 that we may have to account for, so this doesn't have to be 2000 vs. 2005-specific.

Here is how I modified the db_version() function in database.mssql.inc:

/**
 * Returns the version of the database server currently in use.
 *
 * @return Database server version
 */
function db_version() {
  $version = db_result(db_query('SELECT @@version;'));
  list($version) = explode('-', $version);
 (strpos($version, '2000')) ? $version = 2000 : $version = 2005;   
  return $version;
}

In practice, it looks something like this:

switch ($GLOBALS['db_type']) {
  case 'mssql':
    if (db_version() == '2000') {
      ...
    }
    ...
    break;
  }
}

Let me know what you think. I all for helping out with this as much as I can, but definitely don't want to have to back-pedal if the group decides to do this a different way.

Enterprise

New Windows SQL Server Driver for PHP

pcorbett@drupal.org — Mon, 03 Dec 2007 22:01:11 +0000

A new SQL Server 2000/2005 driver (preview version) for PHP improves compatibility between the two: http://www.microsoft.com/sql/technologies/php/default.mspx

Now there's native support for PHP that will save us from having to deal with freeTDS and other things. For example, the current limitation between PHP and SQL Server is that it truncates text pulled from a varchar to 255 characters. I currently have to change anything > varchar(255) to a TEXT datatype in SQL Server to avoid this issue.

Has anyone else used this in their SQL Server setup?

Enterprise

New FastCGI extension for IIS

pcorbett@drupal.org — Mon, 03 Dec 2007 21:52:52 +0000

A new IIS 6/7 FastCGI extension was recently released: http://www.iis.net/downloads/default.aspx?tabid=34&g=6&i=1521 It runs on top of ISAPI and generally makes PHP within IIS more stable. Has anyone tried installing this yet? Any noticeable performance boost? I'll be setting it up soon and will reply with my own opinion soon...

Enterprise

Popular, high profile, high traffic Drupal sites for BADCAMP and Drupal.org

ChrisB — Mon, 29 Oct 2007 09:54:25 +0000

I'm cross posting this from here in case anyone else has any information to share: http://drupal.org/node/187464

Hi Fellow Drupalers,
I'm working on a session for BADCAMP (http://badcamp07.org) where we will walk through some cool, popular and high performance Drupal sites. The main focus here is high-profile, big companies, high traffic. What is the biggest and highest traffic Drupal site out there?

I'm gathering additional research and so if anyone is interested please fill out the below template with as much information as possible for a couple of the top sites you know about or have developed, ideally 3 or 4 sites? Possibly some nice rare ones people don't know about? Please fill out as much information as possible. If you can't fill out any info, at least the site URL would help and I can do more research from there.

This once this research is complete, it will be put on Drupal.org as well and this will be great marketing material for the Drupal community.

Website URL:

Info:
Company:
Development firm/team:
Est. date launched:
If part of a multisite setup, how many total sites?:

Stats:
Current number of nodes:
Current number of users:
Est. database size:

Traffic:
Avg visits/day:
Avg page views/day:
Avg visitors per/day:
Avg visits/month:
Avg page views/month:
Avg visitors per/month:
Est % authenticated users:
Est GB transferred/month:

Other:
Key Modules Used:
Any special challenges faced:
Additional Notes:

The event is next weekend (the 3rd and 4th) so if at all possible, please reply by Tuesday or Wednesday.
Many thanks for your time!
Chris

PS: Here is a list of some known large companies that use Drupal. Please add to it if you can, even if you don't know of any details about the site. If you know details about the site and can provide some, please do.

IBM (internal)
Yahoo
Adobe
Sun Microsystems
Sony
MTV
Fox
Warner Brothers
Best Buy
New York Observer
Phizer
Lifetime Television

Other:
The Onion
Morris Publishing Group (Savannah Now)
Sugar Publishing (Pop Sugar, Team Sugar)
Kernel Trap
Linux Journal
Wikimedia (Fundraising)
Mozilla Foundation (Spread Firefox)
Ubuntu
Kentucky Derby
PLUS MANY MORE...

Social Networking Sites

Enterprise

Drupal / PHP Developer | Vivakos, Inc.

Various Drupal positions | Arizona State University

3 Customers with more than 1000 users in production deployments needed for Gartner social software market assessment

Drupal For Enterprise

Presentations

Thank You!

Note

Sourceforge 2008 Community choice award finalists: Best project,for Enterprise, for Educators, Change the world, for Developers

Enterprise 2.0 Expo/ Enterprise2Open - Free to attend

PHP/MySQL Developer | CEMPER GmbH

Web Developer

Vote in Enterprise 2.0 Launchpad contest: Round 2 with a new Drupal ecosystem section

Xapian Search for Drupal - Metrics and Benchmarks

Vote for Drupal in Enterprise 2.0 Launchpad contest

Selenium and Drupal

Goals

Why Java/Maven?

Environment Setup

Setup your Tests

Modify the "setup.xml"

Site architecture patterns, Install Profiles, & automated site configuration

Defining site architecture and best practice patterns

Projects and implementations

Limitations in Install Profiles

Proof of concept solution

Here are some design ideas and implementation details:

Patterns primary uses

Architecture Overview

Challenges

Primary plans and ideas for the future

Possible other side benefits and solutions

Potential RDF use cases for Drupal

Generic metadata storage

Faceted browsing

Utilizing open, linked data

From modules to widgets

RDF export functionality

Publish/subscribe aggregation

Distributed identity and trust

Distributed search

Aggregated views/visualizations

Global hub/dashboard

Some review and proposal about Drupal 7.x database stack

Drupal 4.7.x/5.x

Drupal 6.x

So what's coming next for Drupal 7.x?

Any else suggestion?

DB support as contribute: is it a good idea?

Abstract

Background

What PDO Does and Doesn't?

How about Drupal + PostgreSQL (or other databases) support as contribute?

Oh my god! So what should I do for it?

My humble conclusion

Boston Drupalcon showcase and case study contest

Issues Need Review for Enhance Cross Database Compatibility

Need Review

Reviewed

Committed without Thorough Review

My personal battle target for Druapl 7.x

Time to Give Back: Become a Speaker at Drupalcon Boston 2008

PHP 5.2.5 breaks SQL Server support; Yet MSFT SQL Driver Works

Obstacles For Real SQL Server Support

RFC: Best practices for managing indexes (particularly on cck tables)

Suggested Method For Choosing Between SQL Server 2000 and 2005/2008

New Windows SQL Server Driver for PHP

New FastCGI extension for IIS

Popular, high profile, high traffic Drupal sites for BADCAMP and Drupal.org