Postgresql

Postgresql http://groups.drupal.org/postgresql Organizing ideas, maintaining code standards around Drupal and PostgreSQL en Is Drupal + PostgreSQL a reliable combination? http://groups.drupal.org/node/15612 <p>I have a simple Drupal 6.4 installation running on PostgreSQL 8.1.6 (because that is what ships with Centos 5).<br /> So far so good. However, I am evaluating Drupal for use with a large local government website which will probably grow quite complex.</p> <p>How likely is it that we will run into problems using PostgreSQL with Drupal? For us, PostgreSQL is the non-negotiable part. We can always use another CMS but we have ruled MySQL out in favour of what we believe to be the more rigorous, capable and standards-compliant database.</p> <p>Are we asking for trouble using Drupal with PostgreSQL?</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/15612#comments postgresql postgres Postgresql Mon, 06 Oct 2008 13:07:16 +0000 mickh 15612 at http://groups.drupal.org SimpleTests that are known to fail on PostgreSQL http://groups.drupal.org/node/14137 <p>Here are the results of running our whole test suite on PostgreSQL 8.3.</p> <ul> <li><code>Actions configuration 30 passes, 8 fails, 0 exceptions</code><br /> See <a href="http://drupal.org/node/261859" title="http://drupal.org/node/261859">http://drupal.org/node/261859</a></li> <li><code>Block functionality 57 passes, 1 fail, 0 exceptions</code><br /> No known issue (yet!)</li> <li><code>Blog API functionality 45 passes, 2 fails, 1 exception</code><br /> No known issue (yet!)</li> <li><code>Search engine ranking 21 passes, 5 fails, 20 exceptions</code><br /> See <a href="http://drupal.org/node/296624" title="http://drupal.org/node/296624">http://drupal.org/node/296624</a></li> <li><code>Module list functionality 53 passes, 4 fails, 0 exceptions</code><br /> No known issue (yet!)</li> </ul> <p>Some of them may be solved or easier to solve once the <a href="http://drupal.org/node/225450">PDO patch</a> will land. Anyway, please help us fix Drupal on PostgreSQL!</p> <div class="og_rss_groups"><a href="/testing-qa">Testing and Quality Assurance</a></div> http://groups.drupal.org/node/14137#comments Postgresql Testing and Quality Assurance Wed, 20 Aug 2008 21:28:47 +0000 Damien Tournoud 14137 at http://groups.drupal.org Table "Node" Field "language" http://groups.drupal.org/node/12590 <p>Drupal 6.x. Table "node".</p> <p>What should the value of the "language" field be if the displayed text is Chinese.</p> <p>If a value is required, are there any other things that need to be done so that Chinese symbols are correctly displayed.</p> <p>Any help appreciated.</p> <p>Steve<br /> <a href="http://prime357.org" title="http://prime357.org">http://prime357.org</a></p> <p>ps. I've got a <a href="http://prime357.org/node/109">thread</a> running at my site which I think this may be a part issue to it. As you can imagine it's becoming very confusing as all I see is either unicode or Chinese symbols (can't read Chinese).</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/12590#comments chinese encode encodings language Australia Database MySQL Postgresql Fri, 20 Jun 2008 04:45:10 +0000 superjacent 12590 at http://groups.drupal.org Looking for Contributors - C++ (and other stuff in time) http://groups.drupal.org/node/12566 <p>Hope I'm not out of line with this request.</p> <p>I'm now on the lookout for C++ contributors (and other like stuff in time). I started the site <a href="http://prime357.org" title="http://prime357.org">http://prime357.org</a>, which is a separation from my personal blogging (<a href="http://superjacent.net" title="http://superjacent.net">http://superjacent.net</a> - running) site, in early April this year. Too much geeky stuff was creeping into the running site.</p> <p>The purpose of Prime357 was to track my development in C++ and a place to continue development of the Wordpress to Drupal converter, a <a href="http://www.realsoftware.com/">Realbasic</a> application. I've got some good how-to type tutorials (my opinion) and short demonstration programs which are presented via the Drupal book method. I first wanted to get some content into the site before seeking contributors so at at least anybody interested could get a feel for my intent (not that it's set in concrete).</p> <p>Unfortunately, as regards C++ my time has been diverted to other IT stuff, the Wordpress converter and very soon <a href="http://www.purebasic.com/">Purebasic</a> development which will in time will replace Realbasic as the development tool for the Wordpress (and phpBB) converter. I'm getting a heap of hits relating to the C++ stuff and I'd hate to see it dry up.</p> <p>Contributors do not have to be fully fledged C++ coding experts but proficient enough to know what they are talking about which translates to explaining why things are/were done in a certain way. Ideally in normal English, non sms speak.</p> <p>What can I offer contributors? That's a good question, nothing but a sense of satisfaction.</p> <p>I'm Australian based (Melbourne) and the server is in the US. So any local contributors would be very welcome but that's not to say international ones wouldn't be.</p> <p>Okay, that's my pitch.</p> <p>Steve<br /> <a href="http://prime357.org" title="http://prime357.org">http://prime357.org</a> and <a href="http://prime357.org/node/108">Looking for Contributors</a>.</p> <p>ps. Where I've mentioned C++, if you wanted, you could substitute PHP and/or Drupal itself.</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/12566#comments C++ Australia Database MySQL Postgresql Thu, 19 Jun 2008 11:37:27 +0000 superjacent 12566 at http://groups.drupal.org Base 36 (Vancode) - leading digit/character http://groups.drupal.org/node/12038 <p>I hope this is the right area for this question which refers to the 'thread' field of the table 'comments'. I provide a non php solution to convert <a href="http://prime357.org/node/63">Wordpress data to Drupal</a>. I recently became aware that the thread field is not a string of decimal numbers but is in fact a string of base 36 (Vancode) numbers. I've created the necessary functions to convert to and from base 36 format but I'm a little confused re - the leading digit/character.</p> <p>Here is an excerpt from <a href="http://api.drupal.org/api/function/int2vancode/5" title="http://api.drupal.org/api/function/int2vancode/5">http://api.drupal.org/api/function/int2vancode/5</a></p> <blockquote><p> Generate vancode.</p> <p>Consists of a leading character indicating length, followed by N digits with a numerical value in base 36. Vancodes can be sorted as strings without messing up numerical order.</p> <p>It goes: 00, 01, 02, ..., 0y, 0z, 110, 111, ... , 1zy, 1zz, 2100, 2101, ..., 2zzy, 2zzz, 31000, 31001, ...</p></blockquote> <p>So, <code>decimal 35 = z</code> and so it is written as <code>0z</code>. The leading zero in this case, how does that indicate length? Therefore <code>decimal 36 = 10</code> and so is written as <code>110</code> as per the excerpt above. Again, how does the leading character <code>1</code> indicate length when the length of <code>10</code> is <code>2</code>.</p> <p>Obviously I can see a pattern forming whereby the leading character for the first 36 is 0, the next 36 is 1, the next 2 and so on up until z. Is there an assumption that the leading character will never be greater than "z".</p> <p>Any help appreciated.</p> <p>Steve<br /> <a href="http://prime357.org" title="http://prime357.org">http://prime357.org</a></p> <p><b>Update</b>: <i>Since no response from the initial group (Database Schema API) I'm posting to other groups I think might be relevant. Secondary question is, for specific backend database type questions which group or forum is most relevant. I've got other backend database type questions waiting in the wings.</i></p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/12038#comments base 36 vancode Australia Database DruBB Drupal Dojo MySQL Postgresql Fri, 06 Jun 2008 00:13:04 +0000 superjacent 12038 at http://groups.drupal.org Speakers needed: European PostgreSQL Day, Prato, Tuscany, Italy on October 17 and 18 2008 http://groups.drupal.org/node/11099 <p>Drupal speakers are needed for PostgreSQL Day. If you are available or interested please contact me.</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/11099#comments Italy Postgresql Tue, 29 Apr 2008 19:44:03 +0000 Amazon 11099 at http://groups.drupal.org Installing Drupal with PostgreSQL 7.4 http://groups.drupal.org/node/10122 <p>[RESOLVED: see thread below]</p> <p>I started in the regular support forums but they seem to be full of messages on the order of "I don't know anything about PostgreSQL, can anyone verify that it works?" That seemed rather odd to me given that Drupal supposedly now supports use of the PostgreSQL server.</p> <p>Anyway, I am trying to get a Drupal 6.1 installation up and running with PostgreSQL 7.4 running on Debian Linux 4.0 (etch). I've been over the installation instructions several times, but can't get the install.php procedure to successfully connect to the PostgreSQL server, which is running on the same host as Drupal.</p> <p>After supplying the database name, user name, and password, I get an error screen stating that the connection failed. It tells me to check the PostgreSQL log for the reason.</p> <p>The only error indicated there, once each time I try to run the Drupal installation, is:</p> <p>2008-03-24 10:02:02 LOG: incomplete startup packet</p> <p>PostgreSQL is working, and does accept queries and database modifications entered from the command line. The user and database created for Drupal to use are available and functional in that mode. This certainly appears to be a flaw in the Drupal setup code, but I'm not fluent enough in PHP to find it. I gather that many of you here have Drupal running on a PostgreSQL database. Any idea what I'm missing?</p> <p>--Gary Phillips, Librarian, Harvard Diggins Library, Harvard, Illinois</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/10122#comments postgresql installation Postgresql Tue, 25 Mar 2008 18:02:49 +0000 Altivo 10122 at http://groups.drupal.org Differences between PostgreSQL versions http://groups.drupal.org/node/9330 <p>There is a nice feature matrix on the PostgreSQL page in case you are wondering what features are available in different version of PostgreSQL.</p> <p><a href="http://www.postgresql.org/about/featurematrix" title="http://www.postgresql.org/about/featurematrix">http://www.postgresql.org/about/featurematrix</a></p> <p>Looks like 8.3 is really a killer release!</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/9330#comments Postgresql Mon, 03 Mar 2008 07:19:00 +0000 jaydub 9330 at http://groups.drupal.org Design suggestion of multiple databases vs multiple schemas within the database http://groups.drupal.org/node/9319 <p>I am building a web app with Postgres, that also uses Drupal with Postgres. I am new to all these frameworks.</p> <p>There is some data that I'll need to cross-reference between the two databases.</p> <p>Can I do a cross-schema/catalog join? Or is a cross-database join better?<br /> Are there any gotchas for the cross-schema/cross-database joins that I need to be aware of? Such as when using Shared hosting?</p> <p>Any suggestions/comments appreciated.</p> <p>Thanks</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/9319#comments Postgresql Sun, 02 Mar 2008 19:54:43 +0000 swami 9319 at http://groups.drupal.org Request Virtual Machine with Drupal working with PostgreSQL http://groups.drupal.org/node/9266 <p>Can someone please post a VMware VM appliance with Drupal working with Postgres, that us newbies can play with?<br /> Additionally, all popular modules working with Postgres may be enabled in this VM. This can help increase a beeline for the unsupported modules to add support.</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/9266#comments Postgresql Thu, 28 Feb 2008 21:56:16 +0000 swami 9266 at http://groups.drupal.org How to write PostgreSQL-friendly SQL http://groups.drupal.org/node/9243 <p>With the release of Drupal 6 support for PostgreSQL has come a long way in that core and contrib modules no longer have to know PostgreSQL syntax for CREATE TABLE and other table/index creation or alteration actions.</p> <p>However when it comes to writing actual SQL in core and contrib modules there will be many situations where SQL that works fine in MySQL will not work in PostgreSQL. Most of the time this is due to the use of MySQL-specific SQL functions or to PostgreSQL's stricter adherence to SQL 92 or SQL 99 standards.</p> <p>Now that D6 is out it's likely that many module authors will begin to port their modules to D6. With the schema API addition to D6, most authors will assume that PostgreSQL support is included without thinking about the SQL they write in their module functions. For those of us that are PostgreSQL supporters and have the knowledge to help module authors with their SQL this is a great time to make a push for more solid support of PostgreSQL in Drupal.</p> <p>This Wiki can help to identify different types of SQL that will have problems in PostgreSQL and include solutions if any to these problems. Please add or update anything you think is helpful.</p> <h3>Multiple value INSERTs</h3> </p> <ul> <li>In MySQL (and in PostgreSQL 8.2+) it's possible to insert multiple rows in a single INSERT statment. This is not available in PostgreSQL < 8.2</li> </ul> <p><div class="codeblock"><code>INSERT INTO foo (bar, baz) VALUES (1,2), (3,4), (5,6)<br /><br />replace with<br /><br />INSERT INTO foo (bar, baz) VALUES (1,2)<br />INSERT INTO foo (bar, baz) VALUES (3,4)<br />INSERT INTO foo (bar, baz) VALUES (5,6)</code></div></p> <h3>INSERT IGNORE</h3> <ul> <li>MySQL allows you to use INSERT IGNORE to insert rows into a table and instead of causing an error when attempt to insert a row with a duplicate of a primary key or unique column the insert for that row is ignored. <ul> <li><a href="http://dev.mysql.com/doc/refman/5.0/en/insert.html" title="http://dev.mysql.com/doc/refman/5.0/en/insert.html">http://dev.mysql.com/doc/refman/5.0/en/insert.html</a></li> </ul> </li> </ul> <blockquote><p> If you use the IGNORE keyword, errors that occur while executing the INSERT statement are treated as warnings instead. For example, without IGNORE, a row that duplicates an existing UNIQUE index or PRIMARY KEY value in the table causes a duplicate-key error and the statement is aborted. With IGNORE, the row still is not inserted, but no error is issued </p></blockquote> <ul> <li>PostgreSQL has no such functionality. The easiest workaround is to replace the INSERT IGNORE with a DELETE and INSERT<br /> <div class="codeblock"><code>DELETE FROM foo WHERE bar = 1 AND baz = 2<br />INSERT INTO foo (bar, baz) VALUES (1, 3)</code></div></li> </ul> <h3>REPLACE INTO</h3> <ul> <li>MySQL allows you to use REPLACE [INTO] in place of an INSERT. <a href="http://dev.mysql.com/doc/refman/5.0/en/replace.html" title="http://dev.mysql.com/doc/refman/5.0/en/replace.html">http://dev.mysql.com/doc/refman/5.0/en/replace.html</a></li> </ul> <blockquote><p> REPLACE works exactly like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is deleted before the new row is inserted. </p></blockquote> <ul> <li>No such function exists for PostgreSQL. The likely workaround would be the same as INSERT IGNORE, to replace with a DELETE and INSERT or a SELECT to find if row exists and UPDATE or INSERT if row does not exist. </li> </ul> <h3>UNIX_TIMESTAMP</h3> <ul> <li>MySQL has a date/time function UNIX_TIMESTAMP to return the current time in UNIX timestamp format. This is essentially the same as the PHP time() function. </li> <li>PostgreSQL has a method of extracting the UNIX timestamp but it's a bit more involved than the MySQL UNIX_TIMESTAMP <ul> <li>SELECT EXTRACT('epoch' FROM CURRENT_TIMESTAMP)</li> </ul> </li> <li>Suggested workaround is to pass in the UNIX timestamp from PHP (which is the better practice anyway, it keeps you from running into time offset issues between your web server and your database server)<br /> <div class="codeblock"><code>$result = db_query('SELECT foo, bar FROM baz WHERE created > UNIX_TIMESTAMP - 86400 * 7');<br /><br />replace with <br /><br />$time = time();<br />$result = db_query('SELECT foo, bar FROM baz WHERE created > %d - 86400 * 7', $time);</code></div></li> </ul> <h3>Date math</h3> <ul> <li>Both MySQL and PostgreSQL have a number of ways to do date arithmetic. Unfortunately they are different enough that whenever needed, it's best to try and accomplish any date math in PHP where possible. <ul> <li><a href="http://www.postgresql.org/docs/8.1/static/functions-datetime.html" title="http://www.postgresql.org/docs/8.1/static/functions-datetime.html">http://www.postgresql.org/docs/8.1/static/functions-datetime.html</a></li> <li><a href="http://dev.mysql.com/doc/refman/5.0/en/date-and-time-functions.html" title="http://dev.mysql.com/doc/refman/5.0/en/date-and-time-functions.html">http://dev.mysql.com/doc/refman/5.0/en/date-and-time-functions.html</a></li> </ul> </li> </ul> <h3>GROUP BY</h3> <ul> <li>MySQL allows you to SELECT columns that are then not explicitly stated in the GROUP BY clause. PostgreSQL does not <ul> <li><a href="http://www.postgresql.org/docs/8.1/static/sql-select.html" title="http://www.postgresql.org/docs/8.1/static/sql-select.html">http://www.postgresql.org/docs/8.1/static/sql-select.html</a></li> </ul> </li> </ul> <blockquote><p> When GROUP BY is present, it is not valid for the SELECT list expressions to refer to ungrouped columns except within aggregate functions, since there would be more than one possible value to return for an ungrouped column. </p></blockquote> <ul> <li>Workaround is to simply add the additional columns to the GROUP BY clause.</li> </ul> <div class="og_rss_groups"><a href="/mysql">MySQL</a></div> Postgresql MySQL Thu, 28 Feb 2008 04:42:26 +0000 jaydub 9243 at http://groups.drupal.org Drupal support for Postgres http://groups.drupal.org/node/9236 <p>Hello,</p> <p>I am new to Drupal, and am using Mono/Postgres with my personal project. I'd like to integrate a CMS with it, and zeroed on to Drupal over MojoPortal, for its much higher range of features.</p> <p>However, from my search, I find that Postgres support seems kind of step-motherly. As a newbie, I wanted your comments on whether I should still bite the bullet & go with Drupal 6, or should I wait for 7 till the support gets better.</p> <p>Here are the Drupal features I need:<br /> 1. Books<br /> 2. Wiki<br /> 3. Blog<br /> 4. Forum<br /> 5. Rating system<br /> 6. Statistics</p> <p>Thanks</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/9236#comments Postgresql Wed, 27 Feb 2008 21:19:00 +0000 swami 9236 at http://groups.drupal.org Postgres Support Task Force http://groups.drupal.org/node/9103 <p>I have been using Drupal with Postgres for some time now. I actually choose Drupal because of it supported Postgres (I really like the database). Throughout this time I’ve helped fix several issues related to module’s support to Postgres. This is a kind of chicken and egg problem. Since not all modules support Postgres, a lot of people are forced to use MySQL, and thus never improve Postgres support. Drupal 6 Schema API promises to improve this very much.</p> <p>However, after reading a lot of flames on dropping postgres supports from core, and how to fix it, I have come to the conclusion that we need a more coordinated effort to support Postgres. Fixing issues is not really hard, but you are normally focusing on your problems. I believe we need a team of user to help test and fix issues with postgres database support. </p> <p>I’d suggest some actions:</p> <ul> <li>Having a tag on the issues that indicates it’s a postgres problem (like the GOP and newbie tags); <li>Create and promote a Postgres taskforce of people that can help with patching and testing, maybe separate groups; </ul> <p>I really thing this could help, but am not sure how to do this in the “Drupal.org” way.</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/9103#comments Postgresql Fri, 22 Feb 2008 15:50:25 +0000 mailleux 9103 at http://groups.drupal.org Some review and proposal about Drupal 7.x database stack http://groups.drupal.org/node/8907 <p>I start my Drupal + Oracle research since the end of Drupal 4.7 life cycle (around Oct 2006), based on my client's request. The project is still running, but I am not satisfy about its progress. After keep trace in Drupal database stack implementation for more than years, it is time for focusing on Drupal 7.x development.</p> <p>So what are the changes since 4.7.x? And what will be happened for 7.x? I would like to share my research progress with you, and so let's brain storming for what's next :-)</p> <p><img src="http://edin.no-ip.com/html/files/DrupalDBStack_title.png"></p> <h2 id="drupal-4.7.x-5.x">Drupal 4.7.x/5.x</h2> <p><a href="http://edin.no-ip.com/html/files/DrupalDBStack_D5.png"><img src="http://edin.no-ip.com/html/files/DrupalDBStack_D5.png"></a><br /> Fig 1.1 Drupal 5.x database stack.</p> <p>The case of Drupal 4.7.x and 5.x is very similar: we support both MySQL and PostgreSQL backend, with specific database access API file (e.g. database.mysqli.inc), enclosed with database.inc, and that's all. We still have some extra abstraction build on top of this simple model, e.g. tablesort.inc and pager.inc, for some useful application layer abstraction.</p> <p>During this period, core and contribute developers need to understand the differences between MySQL and PostgreSQL, since database schema must handle manually. I would like to call this as the dark age of our history: we are asking every contribute developers as both MySQL and PostgreSQL expert (but most of us coming with MySQL-only skill set), or else code was not portable. This result in a complicated PostgreSQL support, and so most contribute modules are MySQL only.</p> <h2 id="drupal-6.x">Drupal 6.x</h2> <p><a href="http://edin.no-ip.com/html/files/DrupalDBStack_D6.png"><img src="http://edin.no-ip.com/html/files/DrupalDBStack_D6.png"></a><br /> Fig 2.1 Drupal 6.x database stack.</p> <p>For our newest stable Drupal release - 6.x, Schema API is being introduced. We spend a lot of time for its development, during 6.x 1-year development life cycle. It is target to solve the schema problem as mentioned above. Contribute developers are no longer required as bi-expert nor multi-expert: we just need to know how to define our schema in a correct Schema API syntax, Schema API will handle the rest for us. IMHO, it is very powerful, and it is an important improvement for other database development.</p> <p>drupal_write_record() is being introduce, too. This handy function will grep out existing cached schema, and build INSERT/UPDATE queries for us automatically. It try to handle all column type binding for normal developers, which is usually very bulky in implementation.</p> <p>BTW, the work is not too elegant: all stuff are mixed into database.*.inc, and so files are even split as 2 section (groups <code>database</code> and <code>schemaapi</code>) internally. This may increase the difficulty for our core maintenance.</p> <p>Most likely, this design is very similar as <a href="http://docs.moodle.org/en/XML_database_schema">Moodle 1.7+ XMLDB</a> implementation.</p> <h2 id="drupal-7.x">So what's coming next for Drupal 7.x?</h2> <p><a href="http://edin.no-ip.com/html/files/DrupalDBStack_D7_hook.png"><img src="http://edin.no-ip.com/html/files/DrupalDBStack_D7_hook.png"></a><br /> Fig 3.1 Well... So what's coming next for Drupal 7.x?</p> <p>The truth is: I don't know the overall schedule up to this moment. <a href="http://groups.drupal.org/data-architecture-design-sprint">Data Architecture Design Sprint group</a> is now hard working for this, and going to present their research progress in Boston DrupalCon 2008. Since we are going to use PHP PDO (instead of legacy PHP database connection drivers), people are also focusing on implement Data API or Active record model for a higher level of abstraction. Before they have their final decision, I would like to provide some successful case study as reference, and let's brain storming for the optimal solution :-)</p> <p><a href="http://edin.no-ip.com/html/files/MoodleDBStack.png"><img src="http://edin.no-ip.com/html/files/MoodleDBStack.png"></a><br /> Fig 3.2 Moodle 1.7+ XMLDB stack. From <a href="http://docs.moodle.org/en/Development:XMLDB_introduction#The_Stack">doc.moodle.org</a>.</p> <p>When we are talking about improve our database abstraction as more powerful and universal, Moodle 1.7+ XMLDB stack should act as a perfect example (it since <a href="http://docs.moodle.org/en/Release#Moodle_1.7">Nov 2006</a>). As Moodle core uses ADOdb internally, this possibility has been present since the beginning. Plus their XMLDB abstraction, Moodle 1.7+ is able to handle both MySQL, PostgreSQL, Oracle and MSSQL while maintaining everything working properly. I will not detail their implementation within this article, and you may refer to their <a href="http://docs.moodle.org/en/Development:XMLDB_introduction">introducetion</a>, <a href="http://docs.moodle.org/en/Development:XMLDB_roadmap">roadmap</a> and <a href="http://docs.moodle.org/en/Development:XMLDB_problems">problems</a> document.</p> <p>The most important idea of Moodle's XMLDB stack is split-brain handling. They left every layer keep focus on their duty ONLY, and so the final approach is perfect and elegant. This is also similar as the well-known <a href="http://en.wikipedia.org/wiki/OSI_model">OSI model</a>: A layer is a collection of related functions that provides services to the layer above it and receives service from the layer below it.</p> <p><a href="http://edin.no-ip.com/html/files/DrupalDBStack_D7_draft.png"><img src="http://edin.no-ip.com/html/files/DrupalDBStack_D7_draft.png"></a><br /> Fig 3.3 Research progress of <a href="http://edin.no-ip.com/project/siren">Siren</a> + personal suggestion.</p> <p>Well, <a href="http://edin.no-ip.com/project/siren">Siren</a> is my personal research project besides Drupal 6.x, which target to explore the possibility of multiple database support as like as Moodle, Gallery2, and eGroupWare. Moreover, prepare research progress whenever Drupal 7.x is open for public development (and that's now). Within this project, we are able to support both 5 database + totally 9 PHP database access drivers, including mysql, mysqli, pgsql, oci8, pdo_mysql, pdo_pgsql, pdo_oci, pdo_sqlite and pdo_ibm.</p> <p>It sounds interesting, isn't it? So what changes I have been made in order to support this approach? Just some step forward improvement: split the handle of function calls into number of individual files, and each layer just keep focus on what they are needed for. E.g. split Schema API related functions into schema.<em>.inc, common DML into common.</em>.inc, and so on. Surly I plug some extra update to our core queries syntax; but most likely, the idea is just very strict forward. The research progress should belongs to <a href="http://en.wikipedia.org/wiki/Session_layer">OSI session layer</a> when compare with OSI model.</p> <p>IMHO, Data API is a must for our future, if Drupal is going to be a web service framework. Acting as a high level abstraction should be the best position for it. It should be similar as the idea of <a href="http://en.wikipedia.org/wiki/Presentation_layer">OSI presentation layer</a>, which is one level below our core application APIs. It may implement as OOP or whatever; BTW, according to its duty and responsibility, it should be separated from PDO database access API (where PDO is much similar as <a href="http://en.wikipedia.org/wiki/Transport_layer">OSI transport layer</a> implementation).</p> <p>One statement as conclusion: split-brain handling.</p> <h2 id="other_suggestion">Any else suggestion?</h2> <p>Database driver is something only need to be implement once and forever. Once the design of logic is completed, the rest is all about the style of implementation, e.g. using OOP as ADOdb or traditional function callback as Drupal.</p> <p>The design of abstraction layer is also similar, too. Once you get a complete study of the variation between each databases, and able to foresee ALL of the needs of their specific limitation, it is all done. There is no such idea of "Contribute Supported Database" within the above 2 layers, and it is what I call as "No gray area, but only black and white". Whatever you support it, or else you don't.</p> <p>On the other hand, Data API is much different. It try to serve the need of our application, and so its requirement will keep on changing depend on time. Therefore a split-brain handle should be the most suitable implementation. Please correct me if you have some better idea :-)</p> <p>But the truth case is: we are lack of PostgreSQL/Oracle/DB2/SQLite/etc developers, and not enough task force nor union for a complete and perfect support for them. Up to this point, I would like to proposal the idea of "Contribute Supported Database" in this way:</p> <ol> <li>Implement ALL possible database drivers into core, and build an abstraction that cover ALL the needs of them. This is a once forever task, and the base of the rest of discussion.</li> <li>ALL core libraries and essential modules for installation MUST implemented as universal, so basic installation MUST function for ALL databases. E.g. node, comment, users, block, etc. This is all the duty of our core developers which need to be ensure before stable release. On the other hand, if MySQL (and maybe SQLite, too) is our main focus, just keep its support as perfect as usual. <p><i>P.S. an additional pre-requirement for this point: try to implement ALL core module as SQL-99 compatible by default, and so save a lot of time for further more follow up handling.</i></li> <li>Label the optional modules as "Not supported" or "Not perfectly supported" within modules selection page, if that is the case of specific database backend; give a different level of support status; or maybe even, disable its activation checkbox by default. Don't let our second/third priority database hold our stable release!</li> <li>left the issue open for contribute developers if they really hope to let that module function for database A/B/C/etc, and release subversions besides stable release for this purpose. E.g. if now search and forum module are not compatible with Oracle, just let it be as Oracle developers duty!</li> </ol> <p>This is my ideal proposal. So how about yours? <a href="http://edin.no-ip.com/html/files/DrupalDBStack.odg">Here</a> is the raw file of above images, please feel free to utilize it if you hope to visualize your idea as above :-)</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/8907#comments Database Enterprise Postgresql Fri, 15 Feb 2008 06:40:02 +0000 hswong3i 8907 at http://groups.drupal.org DB support as contribute: is it a good idea? http://groups.drupal.org/node/8855 <h2 id="abstract">Abstract</h2> <p>Drupal 6.0 is revamped with <a href="http://api.drupal.org/api/group/schemaapi/6">Schema API</a>, so what's next for Drupal 7.x? <a href="http://php.net/pdo">PDO</a> for sure! With this powerful <b>data-access</b> abstraction layer, workload will much reduced for DB abstraction layer designers and developers, and finally benefit our contribute developers and end users.</p> <p>By the way, together with the decision of Drupal 7.x + PDO, there is also some voice about <b>moving PostgreSQL</b> (and so other potential databases support, e.g. Oracle, DB2, MSSQL, etc) <b>support away from core, but contribute</b>; on the other hand, <b>add official SQLite support</b> into Drupal core, together with MySQL.</p> <p>Is this really a good idea? Or even if it is possible? As an existing Drupal + PostgreSQL users, what will this affect your daily work? As a potential customer of Drupal + Oracle/DB2/MSSQL/etc, is this a good new for you, or just an evil? I would like to provide some brief idea for you within this article. </p> <h2 id="background">Background</h2> <p>After <a href="http://cvs.drupal.org/viewvc.py/drupal/drupal/database/database.pgsql?hideattic=0&view=log">more than 6 years of PostgreSQL supporting in Drupal core</a>, PostgreSQL users are going to have a revamped experience with Drupal 6.0. Drupal 6.0 will ship with <a href="http://api.drupal.org/api/group/schemaapi/6">Schema API</a>, which simplify most of the variation between databases handling - <a href="http://en.wikipedia.org/wiki/Data_Definition_Language">DDL</a>. This useful improvement will not only benefit our existing core and contribute developers, but also simplify most workload for other database driver development, besides MySQL and PostgreSQL. With Drupal 6.0, we can simply foresee the expand of existing market sharing, together with the explore of hidden customers.</p> <p>As Drupal 6.0 is ready to launch, it is also a good timing to plan for Drupal 7.x. As we will only support PHP 5.2.x or above in Drupal 7.x, it is for sure that we will revamp our existing DB abstraction layer with the help of <a href="http://php.net/pdo">PDO</a>. It does simplify and standardize a lot of function call implementation, together with some useful new features. If you are a DB abstraction layer designer and developer, moving your work to PDO will save you a lot of time.</p> <p>On the other hand, some people also think it is a good timing for adding <a href="http://drupal.org/node/39260">Oracle</a>, <a href="http://drupal.org/node/67349">SQLite</a>, <a href="http://drupal.org/node/165788">DB2</a> and <a href="http://drupal.org/node/74308">MSSQL</a> support into Drupal core. Most of this request are halted for 2~3 years.</p> <p><a href="http://groups.drupal.org/data-architecture-design-sprint">Some of the developers</a> are hard working with the future plan for how Drupal 7.x DB handling should looks like. BTW, the idea is most likely for expand Drupal usability and functionality, but not about maintain nor expand different databases supporting: some developers even think <a href="http://lists.drupal.org/pipermail/development/2008-January/028343.html">we should</a> only official support MySQL and SQLite with Drupal 7.x + PDO, but move away PostgreSQL (nor other potential databases) supporting as contribute level.</p> <h2 id="pdo_does_doesnt">What PDO Does and Doesn't?</h2> <p>For finding out why this brain storming happen, we need to have some idea about what PDO does and doesn't:</p> <ul> <li><b>PDO is a data-access abstraction layer, but not provide a database abstraction.</b> E.g., this means we can simply forget the function API differences between <a href="http://php.net/mysql-connect">mysql_connect()</a>, <a href="http://php.net/mysqli-connect">mysqli_connect()</a> and also <a href="http://php.net/pg_connect">pg_connect()</a>: we just need to <a href="http://www.php.net/manual/en/function.PDO-construct.php">construct a new-standardized PDO object</a> with different DSN (Data Source Name). Moreover, we can forget those complicated programming logic differences between <a href="http://www.php.net/manual/en/function.pg-escape-bytea.php">pg_escape_bytea()</a>, <a href="http://www.php.net/manual/en/function.oci-lob-load.php">OCI-Lob->load()</a> and <a href="http://www.php.net/db2_lob_read">db2_lob_read()</a>, but simply using standard <a href="http://www.php.net/manual/en/ref.stream.php">Stream API</a> as <a href="http://php.net/pdo#pdo.lobs">PDO LOB (Large OBject) handling</a>. <p>BTW, these are most likely abstracted by Drupal 6.x DB API, e.g. <a href="http://api.drupal.org/api/function/db_connect/6">db_connect()</a>, <a href="http://api.drupal.org/api/function/db_query/6">db_query()</a>, <a href="http://api.drupal.org/api/function/db_decode_blob/6">db_decode_blob()</a>, etc.</li> <li><b>PDO provide <a href="http://php.net/pdo#pdo.prepared-statements">prepared statements and variable binding</a>, even it is not supported by database, e.g. MySQL.</b> With PDO variable binding, we may use <code>?</code> or named-variable (leading with <code>:</code>, e.g. <code>:value</code>) within prepare statement. This seems to be a good idea for replace our existing user-space printf-syntax (e.g. %d, %f, %s, %b and %%) as PDO's C/C++ implementation, and simply forget the relationship between data, data type and database column type. This also give a great hope to <a href="http://groups.drupal.org/node/8001">Drupal 7.x Data API</a> research and development. <p>So isn't it perfect? No, it is not... First of all, <b>most databases require special LOB handle during variable binding</b> (specific PDO::PARAM_LOB manually), e.g. <a href="http://edin.no-ip.com/viewvc/siren/includes/database.pdo_pgsql.inc?view=markup">PostgreSQL</a>, <a href="http://edin.no-ip.com/viewvc/siren/includes/database.pdo_oci.inc?view=markup">Oracle</a>, <a href="http://edin.no-ip.com/viewvc/siren/includes/database.pdo_ibm.inc?view=markup">DB2</a>, etc. Only MySQL and SQLite are able to handle variable binding without type specified. Also, <b>there is no spotlight performance different, even we swap queries variable binding from user-space printf-syntax to PDO</b>, which is proved by <a href="http://edin.no-ip.com/project/siren/siren-1.0-rc4#benchmarking_result">benchmarking result</a>.</li> <li><b>PDO isn't a database abstraction layer, it doesn't abstract and standardize ANY database variation.</b> E.g., PDO doesn't solve the problem of reserved words conflict</a>, it don't <a href="http://drupal.org/node/167335">abstract any SQL function name variation</a>, it also have not duty with <a href="http://drupal.org/node/147947">resolve the maximum string size limitation across most databases</a>. <p>Let's take Oracle LOB handling as an example: PDO give none of help to <a href="http://php.net/pdo#pdo.lobs">Oracle LOB INSERT/UPDATE handling</a>, we still need to use RETURNING + Schema API. According to <a href="http://edin.no-ip.com/project/siren">my research progress</a> and case of Oracle, using standardized function call is the only benefit of using PDO, but didn't help or simplify any programming logic or implementation.</li> </ul> <p>IMHO, PDO is very useful for DB abstraction layer designer and developer: it do simplify most of the DB function call variation; BTW, when we are talking about "PDO can improve database abstraction and so software with PDO can support more databases", it is not really truth. PDO don't have duty with this ;-(</p> <h2 id="contribute_support">How about Drupal + PostgreSQL (or other databases) support as contribute?</h2> <p>It is for sure that MySQL own around 90-95% of our existing market sharing; it is not question that SQLite is a <a href="http://www.cliki.net/manifest%20type%20system">manifest typing</a> database engine; based on the above PDO stuff information, MySQL and SQLite are both very suitable with PDO lossy-form variable binding technology (able to use <code>?</code> and named-variable binding without type specified). But when we are talking about <b>"How many percentage of PDO technologies are utilized by MySQL and SQLite?"</b> and <b>"How many percentage of technologies that PDO NOT covered are used by MySQL and SQLite?"</b>, I guess the answer may just below <b>70%</b>...</p> <p>So what will happen if we seem MySQL and SQLite as our first priority core supported database, and ONLY focus with them? <b>For sure that we will miss out the rest of 30%, which means the basic elements needed by other databases, within Drupal database abstraction layer.</b> This also means asking other database drivers implement as contribute is just a nonsense, if we only support a subset of requirement within core.</p> <p>On the other hand, writing MySQL-specific queries also generate a lot of incompatible issues for PostgreSQL during Drupal 6.x development life cycle. People also claim that <a href="http://lists.drupal.org/pipermail/development/2008-January/028295.html">PostgreSQL issues slow down our Drupal 6.x release</a>. BTW, there is still a lot of incompatible issues even PostgreSQL is now seems as our "Official Supported Database" in Drupal 6.x, we can simply foresee about the case if we just seems it as contribute supported. It is better say that: <b>we are NOT going to support PostgreSQL anymore!</b> And so the similar case for other databases driver development: <b>we will have NO WAY to implement other database drivers as contribute support!</b></p> <p>PDO + PostgreSQL + LOB handling should be a good example of this assumption. In case of pdo_pgsql, we need to specify PDO::PARAM_LOB manually during INSERT/UPDATE variable binding, and <a href="http://bugs.php.net/bug.php?id=37124">must use Stream API for BLOB decode</a>. Both of these handling are NOT required by MySQL and SQLite. If our Drupal 7.x Data API design are target for MySQL and SQLite ONLY, it is for sure that NO WAY for PostgreSQL driver implement as contribute: we will have no hook for it!</p> <p>As a simple conclusion, <b>multiple database support MUST build inside core, with a complete research of what are they needed for.</b> There is no shortcut, and no gray area. The support of database is just simply a TRUE/FALSE question. Please don't be faked by the beautiful wording of "Contribute Supported Database": it is just a sweet poison candy, that is not the truth case, and I have NEVER seen a successful story about this :-p</p> <h2 id="should_do">Oh my god! So what should I do for it?</h2> <p><b>If you are an existing Drupal + PostgreSQL user</b>, it is strongly suggest to keep you eye focus on the <a href="http://groups.drupal.org/data-architecture-design-sprint">Data Architecture Design Sprint group</a>, comment your needs and wish to them, or else you may be fired away without notice. You may also have a look about <a href="http://drupal.org/project/issues?text=postgresql&states=1,16,8,13,14,15,2,4&projects=&priorities=&categories=&users=">issues related to PostgreSQL</a>, give a hand to it if possible, and <a href="http://groups.drupal.org/node/6980">comment them as a Wiki collection</a>. IMHO, it is a critical timing for Drupal + PostgreSQL besides the past 6 years support period. It is time for <b>YOU</b> to take <b>YOUR</b> action, or else will be too late.</p> <p><b>If you are a potential customer or user of Drupal + Oracle/SQLite/DB2/MSSQL/etc</b>, beside the above suggestion for PostgreSQL, I would also like to invite you to give a hand on some <a href="http://drupal.org/project/issues?projects=3060&text=Siren&states=1,16,8,13,14,15,2,4&priorities=&categories=&users=">existing issues</a>, review and comment them, and finally record your work in another <a href="http://groups.drupal.org/node/8663">Wiki tasklist</a>. According to <a href="http://edin.no-ip.com/project/siren">some research founding</a>, we would able to enrich our database supporting without a critical pain of revamp, based on our existing Drupal 6.x DB API implementation. Most research are now completed, so issues review and comment your point of view should be your highest priority. Please remember that: <b>comment is power</b>, and <b>YOUR</b> review is very important for speeding up the core support of your target databases.</p> <h2 id="conclusion">My humble conclusion</h2> <p>Every developers have their own background, their own wish, their own needs, and also their own rose garden. I would not like to comment this <b>"PostgreSQL as Contribute"</b> idea as silly; but at least, it is not my cup of tea.</p> <p>On the other hand, supporting more database won't be a conflict with the research and develop of Drupal's Schema API or Data API: they are target for a better developer and user experience, serving database abstraction should be one of their core duties. Asking for better abstraction and functionality but reduce the number of supported database, or even close the door for other database should be count as loss focus.</p> <p>From my point of view, more choice is always better than just giving me a single solution. The use and choose of database should depend on the needed of each individual case of clients and users, but not forced by developers. If we are already close to our target, there is no point for rollback our 6-years progress on tomorrow.</p> <p><b>A beautiful-sweet-poisoned candy? No thanks!</b></p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/8855#comments db2 mssql oracle PDO PostgreSQL SQLite Database Enterprise Postgresql Wed, 13 Feb 2008 08:45:44 +0000 hswong3i 8855 at http://groups.drupal.org Issues Need Review for Enhance Cross Database Compatibility http://groups.drupal.org/node/8663 <p>Enhance cross database compatibility is important for our Drupal Core, especially when we are going to ship D7 with PHP5.2.x PDO implementation. BTW, this will require number of individual patches, so your help and review are required - add it to the top list with a brief description and an estimate of the time to review. Once reviewed, move the issues to the bottom "reviewed" list, perhaps with your name attached. </p> <h3>Need Review</h3> <ul> <li><a href="http://drupal.org/node/371" title="http://drupal.org/node/371">http://drupal.org/node/371</a> - Resolve ANSI SQL-92/99/2003 reserved words conflict. (just a few minute, but need a lot of follow up)</li> <li><del>http://drupal.org/node/199217 - Split Schema API and shared functions from DB drivers. (just a few minute)</del> (Already come with TNG DB)</li> <li><a href="http://drupal.org/node/167335" title="http://drupal.org/node/167335">http://drupal.org/node/167335</a> - Handle database-specific SQL functions/operators by simple abstraction. (just a few minute)</li> <li><del>http://drupal.org/node/199101 - Core queries coding style cleanup for PDO development (should be RTBC, but need a lot of follow up)</del> (Already come with TNG DB)</li> <li><del>http://drupal.org/node/218128 - Enhance _db_query_callback() for PDO.</del></li> <li><del>http://drupal.org/node/183148 - INSERT/UPDATE/DELETE API, drupal_drop_record() and enhance drupal_write_record().</del> (Already come with TNG DB)</li> <li><a href="http://drupal.org/node/147947" title="http://drupal.org/node/147947">http://drupal.org/node/147947</a> - Replace some TEXT:BIG with BLOB, based on cross database compatibility concern.</li> </ul> <h3>Reviewed</h3> <h3>Committed without Thorough Review</h3> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> Database Enterprise Postgresql Tue, 05 Feb 2008 08:48:16 +0000 hswong3i 8663 at http://groups.drupal.org Are performance optimisations still going into D6? http://groups.drupal.org/node/8492 <p>I'm assuming it's too late, but if not here's an easy performance win for D6:</p> <p><a href="http://drupal.org/node/215080" title="http://drupal.org/node/215080">http://drupal.org/node/215080</a></p> <blockquote><p>The type column of the {system} table is currently a varchar(255) field. Yet in 99% of all Drupal installations it stores either the text 'module' or the text 'theme' and nothing else. This is already a strong case for making it an int column and defining constants for module and theme. Then, on every page, there is this query:<br /> SELECT filename FROM system WHERE name = 'user' AND type = 'module';</p> <p>This query always takes over 1 ms to run making it not a criminally slow query, but relatively slow, nonetheless. This is because the list of indexes available doesn't help the query much</p></blockquote> <p>By shortening the {system}.type column to 32 chars and adding an index we reduce the query by around 60%... and this query runs on every page load.</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/8492#comments D6 high performance optimization Benchmarking Drupal Database High performance MySQL Postgresql Mon, 28 Jan 2008 15:07:59 +0000 robertDouglass 8492 at http://groups.drupal.org Core / Contrib Issues That Require a Postgres Review http://groups.drupal.org/node/6980 <p>If you know of an issue that is being postponed while waiting for a Postgres review or that adds PostgreSQL compliance to a module and therefore needs testing on PostgreSQL - add it to the top list with a brief description and an estimate of the time to review. Once reviewed, move the issues to the bottom "reviewed" list, perhaps with your name attached.</p> <h3>Need Reviews</h3> <ul> <li><a href="http://drupal.org/node/225450" title="http://drupal.org/node/225450">http://drupal.org/node/225450</a> - New database layer</li> <li><a href="http://drupal.org/node/285074" title="http://drupal.org/node/285074">http://drupal.org/node/285074</a> - Simpletest broken on postgres</li> <li><a href="http://drupal.org/node/107051" title="http://drupal.org/node/107051">http://drupal.org/node/107051</a> - Same user listed multiple times in who's online block</li> <li><a href="http://drupal.org/node/223820" title="http://drupal.org/node/223820">http://drupal.org/node/223820</a> - Book module tests for postgres error</li> <li><a href="http://drupal.org/node/181689" title="http://drupal.org/node/181689">http://drupal.org/node/181689</a> - Modify db_distinct_field() so it doesn't fail with count() queries. 15 minutes. needs a review on the regex and perhaps an install to verify the fix. install OG, enable access control for OG, then go to path: /q=og. should be error-free with the patch. (mmilano)</li> <li><a href="http://drupal.org/node/189810" title="http://drupal.org/node/189810">http://drupal.org/node/189810</a> - Recipe Modules fails to insert ingredients.</li> <li><a href="http://drupal.org/node/212264" title="http://drupal.org/node/212264">http://drupal.org/node/212264</a> - SQL Query failed on PostgreSQL 7.4 (syntax errors on admin pages with a fresh install).</li> <li><a href="http://drupal.org/node/128846" title="http://drupal.org/node/128846">http://drupal.org/node/128846</a> - Rewritten PostgreSQL queries fail / Nodes are sorted in the wrong order on pgsql when using a node access module.</li> <li><a href="http://drupal.org/node/229038" title="http://drupal.org/node/229038">http://drupal.org/node/229038</a> - "Query failed" error in Chart API.</li> <li><a href="http://drupal.org/node/222389" title="http://drupal.org/node/222389">http://drupal.org/node/222389</a> - Add PostgreSQL support to "Job queue" module.</li> <li>[Comment notify query for people to send mails to missing column aliases and uses = '' instead of is null in where] (<a href="http://drupal.org/node/308585" title="http://drupal.org/node/308585">http://drupal.org/node/308585</a>)</li> <li>Add your issue here</li> </ul> <h3>Reviewed</h3> <ul> <li><a href="http://drupal.org/node/208938">Critical Core Comment Index</a> - To test: install drupal5 with comment module, upgrade to Drupal6.</li> <li><a href="http://drupal.org/node/247526" title="http://drupal.org/node/247526">http://drupal.org/node/247526</a> - PostgreSQL support in FeedAPI update functions.</li> <li><a href="http://drupal.org/node/212327">Improve bulk update speed using a cast</a> - Especially needs postgres testers - reviewed by aufumy.</li> </ul> <h3>Committed without thorough review</h3> <ul> <li><a href="http://drupal.org/node/242048" title="http://drupal.org/node/242048">http://drupal.org/node/242048</a> - Replace 'Blue Smurf' in search results message - and it apparently fails on postgresql: <a href="http://drupal.org/node/264836" title="http://drupal.org/node/264836">http://drupal.org/node/264836</a></li> <li><a href="http://drupal.org/node/190128" title="http://drupal.org/node/190128">http://drupal.org/node/190128</a> - _block_rehash fails on pgsql.</li> <li><a href="http://drupal.org/node/146466" title="http://drupal.org/node/146466">http://drupal.org/node/146466</a> - Search improvements.</li> <li><a href="http://drupal.org/node/189837" title="http://drupal.org/node/189837">http://drupal.org/node/189837</a> - Remove temporary table from comment.install | Drupal6 Core | 5 minutes (setup new installation of Core, make sure it doesn't blow up.</li> <li><a href="http://drupal.org/node/195044">Pathauto new install fails on Postgresql</a> - Code needs review. Just have a clean drupal5 installation on postgres, install Pathauto for the first time (after patching pathauto). Make sure the variable table's column width gets resized.</li> <li><a href="http://drupal.org/node/232564" title="http://drupal.org/node/232564">http://drupal.org/node/232564</a> - Comment notify refactored data storage and postgresql compliance</li> </ul> <div class="og_rss_groups"><a href="/reviewers">Reviewers</a></div> Postgresql Reviewers Wed, 07 Nov 2007 13:08:02 +0000 greggles 6980 at http://groups.drupal.org Developer | Catalyst IT http://groups.drupal.org/node/5493 <p>Want to live and work in New Zealand?</p> <p>We are expanding our drupal work, and are looking for more developers who love drupal.</p> <p>Other Opportunities within catalyst to learn and work with other opensource technologies as well as drupal including php, perl, java, ruby, erlang, more...</p> <p>send your CV to <a href="mailto:brenda@catalyst.net.nz">brenda@catalyst.net.nz</a>. Please include links to modules you've contributed or patches/testing/benchmarking you've done on drupal.org, or another open source project.</p> <p><a href="http://catalyst.net.nz">http://catalyst.net.nz</a></p> <p><a href="http://catalyst.net.nz"><img src="http://www.catalyst.net.nz/sites/all/themes/Catalyst/images/logo.gif" alt="Catalyst logo" /></a></p> <p> Here, have some blurb:</p> <p>Catalyst is a fully New Zealand owned company, directed and staffed by bright and intelligent Information Technology experts who deliver critical open source business systems to some of NZ's largest organisations.</p> <p>For nearly five years Catalyst has been working with Telecom New Zealand to deliver a truly impressive range of mobile services for the country's leading telecommunications provider. The key to the relationship is not so much the technology, it's Catalyst's no-fuss ability to deliver.</p> <p>Catalyst is spearheading the New Zealand Open Source Awards to highlight the broad range of open source activity in New Zealand and by New Zealanders. The Awards coincide with Catalyst's ten year in business as New Zealand's leading independent open source solutions provider.</p> <p>Catalyst has enabled the Electoral Enrolment Centre to fully embrace open source from desktops to the high-end applications that manage the electoral roll, and secure communications between the Centre and its field officers.</p> <p>Catalyst has now had several years involvement with Moodle - an open source Learning Management System, helping educators create effective online learning communities.</p> <p>In that time we have become Moodle Developers - a status given to only a few - and we have built up a Moodle hosting infrastructure that offers robustness, performance, scaleability and full redundancy via a remote disaster recovery site.</p> <p>In fact we host and support the Open Polytechnic's Moodle sites, which is one of the largest on-line learning services in the world.</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> elections newspapers opensource perl php PostgreSQL telecommunications High performance New Zealand Newspapers on Drupal Postgresql Tue, 07 Aug 2007 22:42:52 +0000 Shiny 5493 at http://groups.drupal.org php/drupal programmer | Database Designs Associates http://groups.drupal.org/node/4998 <p>Database Designs Associates is looking for a programmer with PHP/ MySQL background. Projects will include both general web development projects and Drupal customization. Other potentially useful skills: Access database development, postgreSQL, perl, HTML, XML and related web site developer skills. Ability to communicate well is also valuable.</p> <p>We’re looking for someone to</p> <pre><code>*learn our systems working with senior consultants; *assist on development tasks related to forms, reporting, data management; *assist on testing, quality control, installation, and support. </code></pre><p>The opening would be part time to start, and could lead to full time. We are based in Boston, though we would also consider someone based in the Amherst area.</p> <p>We are a 15 year nonprofit oriented consulting and software development practice, committed to a collaborative work environment, progressive values, and client-centered practices. Please visit our website for more about us. If you know someone who might be interested, please respond to <a href="mailto:jobs@dbdes.com">jobs@dbdes.com</a>.</p> <div class="og_rss_groups"><a href="/western-massachusetts">Western Massachusetts</a></div> jobs Boston Postgresql Western Massachusetts Thu, 05 Jul 2007 18:52:49 +0000 jobs 4998 at http://groups.drupal.org Converting from MySQL to PostgreSQL http://groups.drupal.org/node/4680 <ol> <p><strong>NOTE:</strong> I suppose you could use a converter on your schema, but I was using MySQL 4.1 and wanted the constraints that came with Postgresql. I was able to just convert it with no problems. I started with MySQL 4.1 and moved to PostgreSQL 7.4</p> <li> Figure out which modules might not support Postgresql. I put this part first since it might take you a while to track down upgrades, patches, or fixes you apply yourself. Find out all the modules that you are using. <p><div class="codeblock"><code># modules=`echo "SELECT name FROM system WHERE status=1 AND type='module'" | mysql --skip-column-names -u <em>prototype -p prototype`</code></div> </p> <p>Find all the install files that contain the word db_type:</p> <p><div class="codeblock"><code># cd /path/to/modules<br /># installs=`for module in $modules; do find . -name $module.install -exec grep  db_type {} /dev/null \; ; done | awk -F: '{ print $1 }' |sort | uniq`</code></div></p> <p>Find all the install files that do not contain the word pgsql:</p> <p><div class="codeblock"><code># for install in $installs; do grep -q 'pgsql' $install /dev/null || echo "$install does not contain psql schema code."; done<br />./devel/devel.install does not contain psql schema code.</code></div></p> <p>With this list you should see if you can find a patch like this one:</p> <p><a href="http://drupal.org/files/issues/devel_18.patch" title="http://drupal.org/files/issues/devel_18.patch">http://drupal.org/files/issues/devel_18.patch</a></p> <p>Or perhaps you need to upgrade the version of the module you are using. Otherwise you have to leave it off the list when we update the fresh database we are about to create.</p> </li> <li>Create a fresh database on you development server. This way we can get a fresh schema for importing into a second database. This works better than try to import data into an existing fresh installation. <p> <div class="codeblock"><code>$ createdb -U <em>drupal5 -W drupal5fresh -E UNICODE<br />CREATE DATABASE</code></div><br /> <Strong>NOTE:</strong> You should already have a drupal5 user setup that can add databases</strong> </li> <li>Checkout a fresh installation of your drupal code <strong>-or-</strong> use your existing development setup</li> <li>Update sites/defaults/settings to point to a new database</li> <li>Perform a normal installation (e.i. goto <a href="http://127.0.0.1/install.php" title="http://127.0.0.1/install.php">http://127.0.0.1/install.php</a>)</li> <li>Goto the Admin page and bring up the list of modules. By hand add all the modules based on the list you produce with the code below. At this point leave out any modules that don't support PostgreSQL. <p><div class="codeblock"><code>#  for module in $modules; do find . -name $module.info -exec grep ^name {} \; ; done | perl -pi -e 's/name = //' | sort<br />Actions<br />Aggregator<br />"Basic event"<br />Block<br />Captcha<br />Content<br />Devel<br />Devel Node Access<br />Drake<br />Event<br />. . .</code></div></p> <p><strong>NOTE:</strong> If you get database errors or warning, then the code isn't supporting PostgreSQL very well. So you need to find a solution to fix it. I had problems with Panels, Textimage and Webform. Make sure you submit patches back to the project, or update tickets that note you success or failure. For example there was a small bug in the panels install I had to change dbtype to db_type. Then I uninstalled it (with errors), then reinstalled it and it worked. </li> <li>Now we dump out the schema but now the data. <p><div class="codeblock"><code># pg_dump -h 127.0.0.1 -U  drupal5 -W -s drupal5fresh  > /var/tmp/drupal5fresh_schema.sql</code></div></p> </li> <li>Create a second database to import the schema with no data. <p><div class="codeblock"><code># createdb -O  drupal5  drupal5pgsql -E UNICODE<br />CREATE DATABASE</code></div> </li> <li>Import the schema into the second database <p><div class="codeblock"><code># cat /var/tmp/drupal5fresh_schema.sql | psql -h 127.0.0.1 -U drupal5 -W drupal5pgsql</code></div></p> </li> <li>Dump you existing MySQL database <p><div class="codeblock"><code># mysqldump -t  --skip-opt --compact   prototype > prototype_data.sql</code></div> </li> <li>Convert it. There are two tools, the first one has been worked on recently. But I had success with the second one, during a complete conversion, it failed when I removed the CREATE TABLE syntax. I did have delete the devel_queries, devel_times, cache_menu, cache_views rows. They were just to big or a waste of time. <p><strong>NOTE:</strong> I had to modify the mysql2pgsql.perl code to remove the E'' notation. I am not using PostgreSQL 8.1.</p> <p><div class="codeblock"><code>#s/'(.*?)'([,)])/E'$1'$2/g;<br /> # for the E'' see <a href="http://www.postgresql.org/docs/8.2/interactive/release-8-1" title="http://www.postgresql.org/docs/8.2/interactive/release-8-1">http://www.postgresql.org/docs/8.2/interactive/release-8-1</a><br />.html</code></div></p> <p><div class="codeblock"><code># ./mysql2pgsql.perl --debug prototype_data.sql drupal5pgsql_data.sql</code></div></p> <p><a href="http://gborg.postgresql.org/project/mysql2psql/projdisplay.php" title="http://gborg.postgresql.org/project/mysql2psql/projdisplay.php">http://gborg.postgresql.org/project/mysql2psql/projdisplay.php</a><br /> <a href="http://www.omnistarinc.com/~fonin/projects/my2pg/my2pg.pl" title="http://www.omnistarinc.com/~fonin/projects/my2pg/my2pg.pl">http://www.omnistarinc.com/~fonin/projects/my2pg/my2pg.pl</a> </li> <li>Import the data, with a little luck you will have a full functional conversion. <p><div class="codeblock"><code>#  cat /var/tmp/drupal5pgsql_data.sql | psql -h 127.0.0.1 -U drupal5 -W drupal5pgsql</code></div></p> <p><strong>NOTE:</strong> If you are working with a lot of data, I would run script first. This way you can go back over the output for any errors. I had a couple of CCK tables that didn't get moved over. I am just going to do those by hand.</p> </li> <li>Take care of the sequences <p><div class="codeblock"><code># echo "SELECT * FROM sequences" | mysql --skip-column-names prototype | awk '{ print "SELECT pg_catalog.setval(\x27"$1"_seq\x27,"$2",false);" }' | psql -h 127.0.0.1 -U drupal5 -W drupal5pgsql</code></div></p> <p><strong>NOTE:</strong> There are any missing sequences there is a problem with the module. It isn't correctly creating the sequence during installation.</p> </ol> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> Postgresql Wed, 20 Jun 2007 20:21:29 +0000 pearcec@drupal.org 4680 at http://groups.drupal.org Changes needed for classified ads http://groups.drupal.org/node/3773 <p>Can anyone suggest what code is needed to enable classiified ads <a href="http://drupal.org/project/ed_classified" title="http://drupal.org/project/ed_classified">http://drupal.org/project/ed_classified</a> to work on pgsql?</p> <p>Here is part of the install program which needs updating</p> <p><div class="codeblock"><code>  switch ($GLOBALS['db_type']) {<br />    case 'mysql':<br />    case 'mysqli':<br />      drupal_set_message(t('Creating classified ads table'));<br />      // the {tablename} syntax is so multisite installs can add a<br />      // prefix to the table name as set in the settings.php file<br />       $success = db_query("CREATE TABLE  {edi_classified_nodes} (<br />          nid int(10) unsigned NOT NULL auto_increment,<br />          vid int(10) unsigned NOT NULL default '0',<br />          expires_on int(10) unsigned NOT NULL default '0' COMMENT 'Unix timestamp of expiration',<br />          expiration_notify_last_sent int(10) unsigned NOT NULL default '0',<br />          PRIMARY KEY  (nid,vid));"<br />          );<br /><br />      break;<br /><br />    case 'pgsql':<br />     watchdog('classified', 'install does not support postgres (pgsql)', WATCHDOG_ERROR);<br />     drupal_set_message(t('This module does not support postgres at this time.','error'));<br />      $success = FALSE;<br />     break;<br />  }</code></div></p> <p>It doesn't look like a lot of code to add - but that's only a guess - I can't code pgsql.</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/3773#comments Postgresql Sun, 22 Apr 2007 12:23:54 +0000 jpolt 3773 at http://groups.drupal.org Trust authentication http://groups.drupal.org/node/3771 <p>Hi folks</p> <p>i use trust mode for authentication within my DMZ. Drupal "outta da box" doesn't support this, and so i've got a patch for it here:<br /> <a href="http://drupal.org/node/125105" title="http://drupal.org/node/125105">http://drupal.org/node/125105</a></p> <p>Can postgresql folks stop by for a comment?</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/3771#comments PostgreSQL Postgresql Sun, 22 Apr 2007 05:05:10 +0000 Shiny 3771 at http://groups.drupal.org PostGIS in Drupal http://groups.drupal.org/node/3231 <p>Hello,</p> <p>I've may have some time available in the next few months to devote to upgrading Drupal's GIS abilities. One area where I'd like to concentrate is exposing <a href="http://postgis.refractions.net/">PostGIS</a> data and functionality. In a nutshell, PostGIS allows storing arbitrary geometries (points, lines, polygons, multipolygons and combinations of those types) in database columns.</p> <p>I'm in the process of writing a quick project proposal, and I wanted to see if anyone knew of existing work to pair Drupal with PostGIS. I would probably start by writing a CCK module to act as the Drupal <-> PostGIS bridge. Hopefully, since PostGIS is OGC based, I could keep it generic enough to later support MySQL spatial extensions or other OGC compliant database stores.</p> <p>In addition to the db layer stuff, I'd like to work on views integration (for more powerful spatial queries) and a UI for entering data that is better than just plopping in WKT. Suggestions or prior work on this front would be most appreciated.</p> <p>Please feel free to request features or specific behaviors. We'll see what I have time to complete.</p> <p>Cheers,<br /> -Mark</p> <div class="og_rss_groups"><a href="/content-construction-kit-cck">Content Construction Kit (CCK)</a></div> http://groups.drupal.org/node/3231#comments postgis PostgreSQL Views Developers Postgresql Location and Mapping Content Construction Kit (CCK) Sun, 18 Mar 2007 19:44:30 +0000 mfredrickson 3231 at http://groups.drupal.org Chx nested set implementation, now as wiki http://groups.drupal.org/node/3008 <p>Sorry for spamming the postgresql group -- but this is database related, so let the elephants lead the charge!</p> <p>Attached Files:</p> <p>nested-set-1 => is a represenation of traditional nested set (operating on DAG) vs. chx implementation<br /> chx2 => mfredrickson's visualization of what chx was describing.<br /> chx2_updated => now with labels<br /> chx_edge_case => a more degenerate DAG. Can chx's algorithm find the right numbers?</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> Postgresql Fri, 02 Mar 2007 21:53:08 +0000 mfredrickson 3008 at http://groups.drupal.org MySQL to Postgresql conversion tool http://groups.drupal.org/node/2956 <p>As a MySQL user I'm finding this a useful resource to help support Postgresql installations. It does mysql>psql conversions (and reverse, but that's usually not the problem!).<br /> <a href="http://sql.phpmix.com/index.php" title="http://sql.phpmix.com/index.php">http://sql.phpmix.com/index.php</a></p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/2956#comments Postgresql Mon, 26 Feb 2007 22:42:52 +0000 sime 2956 at http://groups.drupal.org Hashing in pgsql http://groups.drupal.org/node/2564 <p>Hi,</p> <p>Could someone please look at <a href="http://drupal.org/node/29706" title="http://drupal.org/node/29706">http://drupal.org/node/29706</a> and see if it's pgsql compliant? How do the CONCAT, MD5 and SUBSTRING functions behave on PostgreSQL?</p> <p>Thanks,<br /> Johan Forngren</p> <div class="og_rss_groups"><a href="/reviewers">Reviewers</a></div> http://groups.drupal.org/node/2564#comments PostgreSQL Postgresql Reviewers Sun, 28 Jan 2007 14:54:30 +0000 forngren@drupal.org 2564 at http://groups.drupal.org Request for help - Pathauto Module http://groups.drupal.org/node/2200 <p>Hello,</p> <p>I think this is a great group.</p> <p>I'm not sure if it's within the charter, but I'd like to request some assistance in testing/reviewing the Pathauto module to make sure that it is Postgresql compliant. I'm a maintainer of Pathauto and would love to find a user of both Postgresql and Pathauto to make sure that it works (and that changes I make don't break the compliance!)</p> <p>Anyone care to help? It's a pretty popular module so I imagine I'd hear about any problems, but it would be nice to know "for sure".</p> <p>Thanks,<br /> Greg</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/2200#comments pathauto paths PostgreSQL Postgresql Thu, 28 Dec 2006 14:30:02 +0000 greggles 2200 at http://groups.drupal.org First steps to a Recommendation Proposal http://groups.drupal.org/node/1651 <p>Almost every developer know that there is an other Open Source Database Server besides MySQL; her name is PostgreSQL. She is a good option if you are planning a scalable website with high traffic or specific transactions (e.g. e-commerce, large community sites). There are more advantages but my aim here is not a list of "top ten list of ... ". </p> <p>I'd like to make a tiny proposal that could end with a Common Recommendation to Drupal coders.</p> <p>Why Drupal coders?</p> <p>Drupal core (4.7.x and CVS) is fully PostgreSQL compliant. Amazing! But prepare the pain contributed modules can cause:</p> <ul> <li>no PostgreSQL scheme attached</li> <li>module installers without PostgreSQL option</li> <li>PostgreSQL un-compliant SQL queries</li> </ul> <p>I think Drupal coders (contributors etc.) need a Recommendation of how to handle the minimum standards of PostgreSQL compliance while Drupal end users need more patience :)</p> <p>Comments are welcome.</p> <div class="og_rss_groups"><a href="/postgresql">Postgresql</a></div> http://groups.drupal.org/node/1651#comments Drupal PostgreSQL proposal Postgresql Wed, 18 Oct 2006 07:12:59 +0000 yaanno 1651 at http://groups.drupal.org