Posted by secoif on August 10, 2010 at 1:46am
I need to be able to control site users' bandwidth and diskspace usage.
At this point we are primarily concerned with bandwidth.
I thought I'd get some feedback on how I'm thinking about solving this issue:
- Create a module for Aegir that requires Apache module mod_cband (http://codee.pl/cband.html).
- Module provides a settings panel per site that allows for bandwidth limits to be set.
- Module implements hook_provision_apache_vhost_config which generates vhost config for each site that reflects the bandwidth settings.
And advanced features:
- Provide a Drupal module on the client site so they can monitor their bandwidth usage.
- Settings page to adjust CBandExceededURL (URL site goes to when bandwidth is exceeded)
- Cron job that monitors the bandwidth usage (via /cband-status?xml) and sends email alerts when bandwidth nears limits?
For the diskspace monitoring, I've not thought about it too much, but I'm thinking a drush script that runs du on the sites/site.com/files folder?
Another issue we might like to look at in the future is managing creation of mail & ftp accounts.
Feedback very welcome.
Comments
I have not run across this
I have not run across this need with my hosting service, but it sounds like a really useful feature for some installations.
Matt
Reinventing the wheel?
Is the Drupal stack really the best place to track bandwidth? Why not just use a server that has Plesk on it that supports all of this out of the box?
jemond
@jemond: I had thought about this exact solution and in fact, a plesk api powered aegir-like system is exactly what we deployed in my last workplace, except with a different cms. That made sense in the last workplace because the business was built around plesk. In my current workplace, aegir is our plesk, and we don't need all of the power of plesk, just some of the features. Plus, plesk doesn't know how to manage my drupal installs.
Perhaps a solution might be to use the remote server feature of aegir to manage drupal deploys onto a server with plesk. Hrm.
Now I think about it, the main reason we need this is probably nothing to do with limiting, but monitoring. Currently, we have no way to track what sites are using up what bandwidth, which makes planning for future infrastructure difficult.
Sounds like something i will
Sounds like something i will be working on soon. I would recommend two alterations to the project your undertaking. First i think that instead of just per user quota settings, you should setup a Role level quota level ability. So that you can set a quota level that each user of that role would have. This way it would be very easy to setup different hosting packages based on bandwidth/disk usage as well as be able to possibly add an ability to add additional 'flex' space or bandwidth.
I would recommend against a cron job that checks the bandwidth as this would eat up(small i know) bandwidth in itself. Maybe something onthe aegir side of the operation would be better, and same for the monitor of the site side bandwidth/file quota panel. Just some thoughts though
Just adding that the roles
Just adding that the roles things would be cool. It means you could charge people different amounts for different roles (which would be bandwidth options).
Subscribing - The idea of
Subscribing - The idea of user role pricing based on amount of site resource use sounds very interesting. Seems like this might be a way that I could identify and attribute my cloud server costs directly to the users that utilize it.
"Do or do not, there is no try" -- yoda
HI secoif, great idea. I'm
HI
secoif, great idea. I'm thinking the same thing. Can you tell me if you already started with implementation or is still just an idea. I would love to help with developing.
Subscribing to
And willing to help testing the module.
Has devellopment started yet?
what
about nginx support?
Very interesting discussion
We are looking for a similar solution. Ideally i would like to track bandwidth, disk usage and memory usage per webserver. We would probably need something like snmp installed on those hosts.
Has any development been done yet? Would love to collaborate.
i have this completed the
i have this completed the code is available in the Hostmaster Plus module. Let me know if you have any questions about implementing it. There is a bit of work to do with your templates for new sites as well. I modified mine so that the quota information is stored in each vhost's file.
I placed this drush script in my .drush folder for aegir:
<?phpfunction hostmaster_resources_provision_apache_vhost_config($uri, $data){
$CBandLimit = drush_get_option('hostmaster_resources_bandwidth_quota_'.$uri, '1G');
$CBandPeriod = drush_get_option('hostmaster_resources_bandwidth_term_count_'.$uri, '30');
$CBandPeriodType = drush_get_option('hostmaster_resources_bandwidth_term_type_'.$uri, '0');
switch($CBandPeriodType){
case '0':
$CBandPeriod.='M';
break;
case '1':
$CBandPeriod = (intval($CBandPeriod) * 60 * 24);
$CBandPeriod .= 'M';
break;
case '2':
$CBandPeriod = (intval($CBandPeriod) * 60 * 24 * 30);
$CBandPeriod .= 'M';
break;
case '3':
$CBandPeriod = (intval($CBandPeriod) * 60 * 24 * 30 * 12);
$CBandPeriod .= 'M';
break;
}
$output = "
CBandScoreboard /var/aegir/config/server_master/apache/scoreboards/$uri.scoreboard
CBandLimit 100G
CBandPeriod 30M
<Location /cband-status.html>
SetHandler cband-status
</Location>";
return $output;
}
?>
The module handles adding some fields to the Site node type, and adds the information about use onto the site's page under aegir. If you connect services you can have remote access to the information along with adding / removing resources. All of this is up and functional at http://wembassy.com
url for hostmaster plus:
url for hostmaster plus: http://drupal.org/project/hostmaster_plus
Im not sure if i have ram use
Im not sure if i have ram use tracking enabled as of yet, for me that was not as big an issue.
Thank you
Thank you very much cmcintosh for the elaborate response. This is awesome! I just stumbled upon hostmaster plus and will gladly check it out tomorrow. I will let you know how it fares.