My Idea

Events happening in the community are now at Drupal community events on www.drupal.org.
Vidarls's picture

The handling and publishing of files is one of Drupals weakest points. My site was largely based on me publishing mp3 files and code. I wanted to upload the files with FTP and publish them from an upload directory on the server. At that point I made my own "filepublishing module". And from that point of I started to ponder on what could be done with the way Drupal handles files. And now I have a rough idea of how I would like to do it.
I probably won't have the time to do any time soon, so I'm sharing it here to make shure the idea is not lost when I start to think about other things.

What I want to do:
-It should be possible to add files from different locations:
* Link to remote file, http / ftp
* Copy remote file http / ftp to local server
* Upload files to an upload dir on the local server with ftp, and then import them
* Possibly a mass eb upload java applet (like the one gallery has)
* Normal upload with a form

-Where and how the files are stored should be flexible:
* Store files locally on the server
* Store files on a remote http / ftp server
* Store files as blobs in the database

-A file should be a file whether it is an Image, a media file, a document or whatever this enables:
* Statistics for downloads
* Keeping track of how many files you have, who owns them, what they are attached to etc

So how am i planning on doing this:
I feel that each file should be a node, to keep compatibility with the rest of Drupals functions, taxonomy, access control modules etc. But how this node handles when you access it should depend on what kind of file it is.

the new_generation_filehandler.module:
- Handles a file like a file, nothing fancy
- provide easy to use widgets to put in forms to provide file uploads
- Provide easy to use widgets to put in forms to provide attaching of excisting files to a node
- provide other useful widgets and blocks related to files
- provide import functions to create file nodes from ftp / remote files
- provide a some sort of explorer application to manage all your files
* This explorer application would provide filtering on taxonomy, file type (more on that later) owner, most popluar etc
- provide the basic functions for creating a simple file node

To ensure custom handling of files according to type i suggest file api modules. For example you'd want to look at an image, watch a movie, listen to music and so on.

A file type module would have functions that will be called during node save, load update and delete to store file type spesific information. i.e. extended image info derived from images, artist and song info derived from mp3 files.
it would allso contain the theme functions for the file type to specify how this file type is presented.

During creation of a file node, the user could choose what kind of file type it is, or it could be determined from extension of mime type. Allthough I've never figured out how to get the mime type from a file that is not uploaded through a form.

This is a lot of thougths written in a rather clumsy way, I appologise for that. But I'll try to make it good with a snippet of pseudo php to try to illustrate my idea:

<?php
//NEW GENERATION DRUPAL FILE API

//DEFINE HELP AND OTHER BASIC NODE STUFF

hook_file-node-insert($node) {
 
db_query("INSERT BASIC FILE INFO SEE THE BASIC FILE TABLE FURTER DOWN")
  if (
$node->file-type HAS A MODULE) {
   
file-type-insert($node)
  }
}

hook_file-node-load($node) {
 
db_query("SELECT BASIC FILE INFO")
  if (
$node->file-type HAS A MODULE) {
   
//node passed by reference, returned with extra info
   
file-type_load($node)
  }
}

hook_file-node_view{
  if (
$node->file-type HAS A MODULE) {
   
$output = theme(file-type);
  }
  else {
   
$ouput = theme(standard file theme function)
  }
}
?>

STANDARD FILE TABLE
nid           #node id
vid           #version id
filename   
extension  #i.e. jpg, mp3, doc
mimetype 
filesize     
store        #where it is stored, remote ftp / http, local folder, database blob
path         #some sort of path to the file
//statistics could be a separate table, but for simplicity I include them here
total_downloads
last_downloaded

In short:
- All files are nodes
- A file node i not static, it depends on what kind of file it is
- Files can come from anywhere and be stored anywhere
- A file node is easy to include or attach to your content (example: it should be easy to add a player with your latest mp3 production to the articel where you introduce it.
- It is easy to create a file node from the same form that you creates other nodes, at the same time, either with upload or import.
- You know your files, how many they are, who created them, when they where created, how many downloads they have, where they are attached
- It's easy to manage file nodes

Again I'm sorry for the long post with the clumsy language, but at least now my idea wont die alone.

Comments

Database Filesystem

raintonr's picture

I agree somewhat with this... in particular would like to see at least an option to have files as blobs in the database.

Why? Well - at the moment if one wants to build a cluster/server farm/whatever of many machines serving up a Drupal site the 'files' directory has to be updated in real time between all the nodes. I know someone will say 'NFS' but if one wants to replicate files between two machines and have one of them take over in a failure, NFS won't do.

Most databases have the problem of replication/mirror/fault tolerance over a number of machines licked, so why not take advantage of that is my point?

After lots of searching I can't find anything about using database blobs instead of the filesystem in Drupal. Why is this?

I agree completely with this

bfdi533's picture

I agree completely with this assessment. I would like to be able to store files in a blob so they can be retrieved by multiple web applications, possibly by non-drupal apps as well.

Is there anyone working on this?

File API

Group organizers

Group notifications

This group offers an RSS feed. Or subscribe to these personalized, sitewide feeds: