Posted by dopry on April 27, 2006 at 8:17pm
I was hoping to have some of these questions answered in the poll, but most people have chosen to just cast a vote and not comment.
What dopry wants...
-- specific processing based on rulesets(mimetype, extension, nodetype, userroles, etc).
-- queued processing for long time functions (ex. transcoding media files)...
-- finegrained access control.
-- themable rendering.
I have other stuff kicking the back of my head, but its mostly architectrual stuff to make it more accessible.
Comments
faster private files and webdav
all your stuff is really important. also see http://www.tejasa.com/node/113. webdav and better doc mgmt are somewhat related.
rough api...
This is a rough API I'm planning...
-- file.inc-------------------------------------------------------------------
file_save($key, $dest) - save a file from $_FILES
file_get() - grab file from elsewhere...
file_remove()
file_copy()
file_move()
file_mkdir()
--filesystem.module----------------------------------------------------
filesystem_manage($file) - make a file managed through db
filesystem_unmanage($file) - remove a file from db
filesystem_privatize($file) - make file private (enable file access restrictions).
filesystem_publicize($file) - remove file access and place back in a public spot.
filesystem_load($attributes)
filesystem_copy($srcfile, $dstfile)
filesystem_move($srcfile, $dstfile)
filesystem_derive($srcfile, $dstfile, $derivecallback) - makes a copy, the runs derive callback on copy
filesystem_remove($file)
To handle private downloads I was thinking of some sort of key system, where a key is written to the filesystem
like $path/dlkeys/$hash_of_sessionid+fid+salt. So all we really need to do is ...
if (file_exists($path/dlkeys/$hash_of_sessionid+fid+salt)) {
//send the file
file_delete($path/dlkeys/$hash_of_sessionid+fid+salt);
}
l
Still publically accessible?
The method of storing a private file with a unique path seems okay, until you think about logs or sniffing... If you knew the URL you could get to the file, which sort of defeats the purpose in my eyes... What you would technically want with a private file is to deny access to anyone who doesn't have perms to access the file... So I guess that would mean that there were public private files, meaning a unique URL that possibly makes the file expire after it has been accessed at that url once... and then a private private file, meaning a file that belongs to a user and you must have a session open in order to get to the file...
Basically you would want to put the file in a private directory not accessible by HTTP, then use some sort of function to stream it out that determines the access rights that the user has in order to get to the file. Drupal already does the private file handling, there just needs to be something put in its way to control who can download what and how often... Download keys are cool, as long as they expire, but path based is a bad idea... the keys need to be stored in the db to limit access, and then possibly expired the file is accessed from them, or deleted via a cron after a certain amount of time.
no urls to files...
private files will not be stored in a publicly accessible location. The keys are just identifiers holding a path.. If someone sniffs your filekey and uses the url to it, they will get the download, but the keyfile is erased after the transfer. That is my current plan...
MIME types
Hi Darrel!
How about filetypes? It's important not only for restriction on upload but for correct theming.
how about??
braindump...
-- file.inc-------------------------------------------------------------------
file_save($key, $dst) - save a file from $_FILES
file_get() - grab file from elsewhere...
file_remove()
file_copy()
file_move()
file_mkdir()
--filesystem.module----------------------------------------------------
filesystem_manage($file) - make a file managed through db
filesystem_unmanage($file) - remove a file from db
filesystem_privatize($file) - make file private (enable file access restrictions).
filesystem_publicize($file) - remove file access and place back in a public spot.
filesystem_load($attributes)
filesystem_copy($srcfile, $dstfile)
filesystem_move($srcfile, $dstfile)
filesystem_derive($srcfile, $dstfile, $derivecallback) - makes a copy, the runs derive callback on copy
-- derived files have a parent fid, and a relationship field which describes their relationship to the parent...
filesystem_remove($file)
so webdav.module could
if ($file = file_save('mykey'), or file_get('file://path/to/file')) {
filesystem_manage($file);
}
To handle private downloads I was thinking of some sort of key system, where a key is written to the filesystem
like $path/dlkeys/$hash_of_sessionid+fid+salt. So all we really need to do is ...
if (file_exists($path/dlkeys/$hash_of_sessionid+fid+salt)) {//send the file
file_delete($path/dlkeys/$hash_of_sessionid+fid+salt);
}
Files as Nodes
Already mentioned here.
Daniel F. Kudwien
unleashed mind
Daniel F. Kudwien
netzstrategen
User-based paths
The ability to define the fixed path for a file based on owner/user, related og, mimetype, date, etc. So for instance define a rule that says that files are saved to files/$user/$mime/$filename
This isn't actually a hypothetical. My company just landed a client who wants files broken out by user, so that they can symlink the directories and use their existing disk/user quota system. :-)
Doable in 4.7
Here's code I'm writing as we speak to do what you want. You'll want to add in some user stuff, but it is basically the same idea:
<?php
function mymodule_nodeapi(&$node, $op, $teaser) {
if ($op == 'submit') {
if ($node->files) {
foreach($node->files as $key => $file) {
$node->files[$key]['filename'] = mymodule_file_path_munge($file['filename'], $file['filemime']);
}
}
}
}
function mymodule_file_path_munge($filename, $mime = 'misc') {
$fpath = check_plain($mime);
$fpath = file_create_path($fpath);
$parts = explode('/', $fpath);
$trail = '';
foreach ($parts as $place => $component) {
$trail .= $component;
$trail .= '/';
if (!file_check_directory($trail, true)) {
return $filename; // bail if an error occurs
}
$trail .= '/'; // stupid pass by reference!
}
// all good, return our mime based file name
return $fpath . '/' . $filename;
}
?>
This code comes with no warranty. ;-)
Another idea...
This was brought up in a phone conference with the folks from Advantage Labs... Mark Fredrickson suggested a driver to allow filesystem to save to a revision control system such as SVN/CVS -- this would all multiple revisions of files and also the ability to easily access the files from outside of Drupal.
Angie beat me to it...
I was just on my way to add a comment to this thread with the SVN/CVS idea. ;-)
From my brief investigation it looks like this functionality can be achieved using the .driver system in the fileapi. Ie. a svn.driver
Some initial thoughts:
Some other nice advantages of an SVN backend would be the user access controls (think private downloads), the ability to check out the files repo and make changes outside of drupal (which could also be painful for data integrity reasons), and the ability to rollback changes and make other versioning decisions on a live site (though again, I would probably leave this management to another app).
Thoughts?
-Mark
Scalable file storage
Currently all files are stored in a single directory, which is not a scalable approach at all. To see what I mean put 100 000 or more files in a directory and try working with (accessing programmatically) them (especially in the windows case). I think squid like approach would be better -
$dir1/$dir2/$dir3/$file_namewhere:<?php$dir1 = ($fid / 0x10000) % 0x100;
$dir2 = ($fid / 0x100) % 0x100;
$dir3 = $fid % 0x100;
?>
subscribing...
subscribing...
subscribing...
subscribing...
subscribing...
subscribing...