High performance
This group is dedicated to solutions and approaches for high traffic, high performing Drupal sites. As such, it will deal with a lot of information around the rest of a typical Drupal "stack" -- the operating system, web server, database, and PHP tweaks that combine to support the Drupal application.
This is not a support channel or a place to post bug reports.
If you think you have found a bug, use the issue queue for the appropriate project.
If you need support, see http://drupal.org/support for options.
getimage vs direct url from apache
Assuming images gets re-sized when uploaded, why drupal uses getimage to include images and thumbs
if it is much faster to access the images directly through apache?
I always notice high cpu utilization when I load a page full of images.
How can I avoid this and include direct url to all my images?
thanks for any comments or suggestions.
Nginx static file cache or Varnish in front?
I have used varnish quite a bit on my LAMP stacks and this deff helps quite a bit out for static files to serve up and keeps the load off of apache.
I hear good things about nginx static file caching so is their a need to put varnish up a head of it?
Apache + Varnish : having both on port 80
Hi,
Turned out I can't replicate the problem defined down there, but I guess it would've been solved thanks to Advagg. I now have an entirely different problem.
I have several virtualhosts on Apache, some of them will not be served with Varnish, but all listen to port 80.
So, the question is, how can I have Apache listening on port 80 for some virtualhosts, and Varnish listening on port 80 for other virtualhosts?
I'm having a weird problem and I can't get to solve it.
Firstly, here is the setup :
<
ul>
Optimizing MySQL (indexes) for non-developers
Environment
I’m in the process of setting up a CentOS VPS running a moderately used D6 site (approximately 1000 visits/day). I’ve installed the entire stack including Apache, MySQL 5.5, and PHP 5.3. The VPS has a 3.6Ghz dedicated CPU, and 2.2GB Memory (provided through VPS.net)
I’ve installed Memcache and APC, which seem to be working fine. I’ve tuned MySQL using mysqltuner.pl and tuning-primer.sh. My current my.cnf is posted below. All tables are InnoDB.
Problem
My current issue is that when logged in, various pages tend to be very slow (occasionally taking over 10 seconds to generate). I have devel’s query log functionality enabled and it does not seem to be one specific query that causes the problem- it seems to be random queries that sometimes take several seconds. This especially seems to happen with admin pages.
Indexes?
After doing a bit of research, I believe I need to index various database tables. This site has 22 content types and 73 fields defined. My questions is how do I know what tables to index and what fields to index? I've looked into the DBtuner module, but that only shows core fields (as opposed to the CCK fields).
Does anyone have any tips or suggestions?
Below is my current my.cnf
[mysqld]
#added MH 1-17-12
#comment this out after troubleshooting
log-queries-not-using-indexes
## General
datadir = /var/lib/mysql
tmpdir = /var/lib/mysqltmp
socket = /var/lib/mysql/mysql.sock
skip-name-resolve
sql-mode = NO_ENGINE_SUBSTITUTION
#event-scheduler = 1
## Cache
thread-cache-size = 16
table-open-cache = 4096
table-definition-cache = 2048
query-cache-size = 96M
query-cache-limit = 5M
## Per-thread Buffers
sort-buffer-size = 1M
read-buffer-size = 1M
read-rnd-buffer-size = 1M
join-buffer-size = 2M
## Temp Tables
tmp-table-size = 256M
max-heap-table-size = 256M
## Networking
back-log = 100
max-connections = 25
max-connect-errors = 10000
max-allowed-packet = 16M
interactive-timeout = 3600
wait-timeout = 600
bind-address = 127.0.0.1 #added mh 1-24-12 for phpmyadmin
### Storage Engines
#default-storage-engine = InnoDB
innodb = FORCE
## MyISAM
key-buffer-size = 64M
myisam-sort-buffer-size = 128M
## InnoDB
innodb-log-file-size = 100M
innodb-log-buffer-size = 4M
innodb-file-per-table = 1
#innodb-open-files = 300
innodb_flush_log_at_trx_commit = 2
innodb_additional_mem_pool_size = 20M
# InnoDB caches. Increase if needed (defaults to 8M, should use as much as is available). MH 1-25-12
innodb_buffer_pool_size = 1024M
innodb_flush_method = O_DIRECT
## Replication
server-id = 1
#log-bin = /var/lib/mysqllogs/bin-log
#relay-log = /var/lib/mysqllogs/relay-log
relay-log-space-limit = 16G
expire-logs-days = 7
#read-only = 1
#sync-binlog = 1
#log-slave-updates = 1
#binlog-format = STATEMENT
#auto-increment-offset = 1
#auto-increment-increment = 2
## Logging
log-output = FILE
slow-query-log = 1
slow-query-log-file = /var/lib/mysqllogs/slow-log
log-slow-slave-statements
long-query-time = 2
[mysqld_safe]
log-error = /var/log/mysqld.log
open-files-limit = 65535
[mysql]
no-auto-rehashHigh Performance Drupal 7 Authenticated User Benchmarks
Hi All,
I just thought you might be interested to see some performance benchmarks that I've produced recently whilst doing some work porting the Authcache module to D7.
I think the results are pretty "encouraging". What do you think :) ?
|Benchmark|Requests|Concurrency|Total
Time Taken/s|Requests/s|Avg
Time Per
Request/ms|Time Per
Request
(across all)/ms|Longest
Request/ms|Completed
Requests|Failed
Requests|
Confused LEMP vs LAMP same vps hardware LEMP is slower?
My Nginx setup, granted very basic start to it, no real caching set up yet... LAMP has really no special additions to it kinda a out of the box install.
Drupal sets ups are clones on both systems.
command for testing on both ab -c10 -n1000 http://localhost/
nginx result
Server Software: nginx/1.0.9
Server Hostname: localhost
Server Port: 80
Document Path: /
Document Length: 3255 bytes
Concurrency Level: 10
Time taken for tests: 78.190 seconds
Complete requests: 1000
Failed requests: 0
Drupal Performance Optimization Specialist | Cognisync, Inc
We are looking for a Drupal engineer to review an existing site and offer performance and scalability recommendations to improve site performance.
The existing architecture includes Drupal 6, pound load balancing, MySQL database, memcache and APC. Existing site traffic levels include: 1.5-1.75 million pages, 10-15 million hits and 140-150GB bandwidth per month. Further details to be provided upon request.
The candidate should have a proven track record in this area and be able to furnish references upon request.
Thanks!
Measuring Memory and CPU usage during a benchmark
This is not exclusively "Drupal related", but it's still useful if you want to properly find bottlenecks during a benchmark.
I want to do a test with Ab (Apache Benchmark) and I would like to monitor the memory and CPU usage during that test. What do you use for that? top command line tool?
Senior Drupal Developer (6 Month Contract) | High Performing Company - SW London
Experienced Drupal Developer required to develop & maintain Drupal based sites
Agile / Scrum Methodology experience
Experienced in other frameworks desirable (but not essential)
Project experience of applying 'continuous integration' process (highly beneficial)
Good communication
6 Month Contract with high chance of extension
Please contact Amit Phakey on 0203 326 4110 / amit.phakey@parallelconsulting.com for more information.
[ANN] MongoDrop: A simple distribution that uses MongoDB
I created a simple distro to play with MongoDB. Introducing MongoDrop
It uses the features for blocks, cache, field storage, and sessions.
I found problems with mongodb_watchdog that I couldn't figure out right away. I haven't looked at mongodb_queue yet.
Let me know what you think :)
High Performance Drupal meetup at Filter Digital in Culver City on February 7, 2012
The High Performance Drupal meetup now meets on the 1st Tuesday of the month. We're meeting again on February 7, 2012 and will be at Filter Digital, a digital solutions agency in Culver City located just two blocks from Media Temple, a familiar venue of Drupal meetups on the Westside.
LA Drupal meetups follow a basic format and start off with Drupal news and announcements, local job announcements, raffle prizes and full-length presentations. As always, we'll also have community Q&A, during which time anyone in attendance can ask questions, divine answers from the crowd and give brief, 5-10 minute-long presentations known as lightning talks.
We'll finish off the night with more networking and a Drupal After Dark at 9:30pm.
For any changes to our agenda, stay tuned to this meetup announcement or click the Sign up button below (or both!) to be notified when the agenda has been updated.
About High Performance Drupal
What is the High Performance Drupal meetup about? The "high performance" part of the name is a catch-all to describe Drupal scalability, performance and benchmarking but we also discuss hosting, infrastructure, deployment, monitoring and highly repeatable processes. Also, this meetup has been held at coffeeshops in the past and the addition of caffeine makes our meetings feel as if they go faster and faster as the night goes on.
This is an LA Drupal meetup that focuses exclusively on advanced topics and there will not be a Users Helping Users portion to the evening.
Thank you to our sponsors!
Light food and refreshments will be served throughout the evening courtesy of our food, drink and venue sponsor, Filter Digital, a digital solutions agency.
Our client list spans global brands and Fortune 100 companies to startups and boutique firms. Let us help you create a digital solution that fits your needs!
Our projector for the evening is being provided by Exaltation of Larks, a Drupal strategy, consulting, development and training company with a team of Drupal experts in Los Angeles.
We like doing hard things on the web and excel at creating and maintaining exceptional Drupal sites with unexceptional budgets.
Amazon DynamoDB
Interesting announcement from Amazon today: managed NoSQL via their DynamoDB: http://aws.amazon.com/dynamodb
More details:
http://allthingsdistributed.com/2012/01/amazon-dynamodb.html
The interesting part of this is the data being stored on SSD, low latency, and the ability to pre-provision capacity and usage. My initial thought about this as it applies to Drupal would be along the lines of the mongodb project. IE, a Drupal module and API to enable certain tables to be stored and queried from a DynamoDB database.
Drupal + High Concurrent Users and Node.js or Twisted?
We are in the process of using drupal with our new game server, this server will read and write a lot of the users game data, We have chosen to go with the basic pressflow and minimal modules enabled, some custom ones that work with the services api (its how we communicate with the client and server with JSON).
Someone mentioned to me that and im still trying to figure out the formula of well with a pure drupal site optimized you can only have so many concurrent users hitting the db before apache/php bottle neck comes in.
Memcache configuration options, using multiple memcache servers
I'm interested in finding out how people are configuring Drupal sites to make the best use of multiple memcache servers and distinct machines.
Topologies I've seen so far include:
- replicating the caches to multiple memcache servers
- storing separate caches on each memcache server.
I would also be interested in your experience or thoughts on using moxi with Drupal.
Some relevant discussions I've found so far include
Running out of tricks on how to optimize my Drupal site
Hello,
I would like to seek your advice on what else that I need to do in order to speed up Drupal site. Its really slow for logged in users . The only thing that is good now is Boost , which also has some problems on its own that i worked around.
My site is on a dedicated server on HostGator. Here's what I have so far:
Installed Boost - running ok for anonymous users. Problems with the login block being visible after logging out. I solved this (almost) by ajaxifying the login block (i just wrote my custom code)
Varnish Page Cache + Pressflow 6 IP Based Roles
So we use page level caching on our Pressflow 6 multisite installation and some of the sites use a module for IP based role assignment.
There is an issue with using page caching and IP based role assignment in which a user will get a cached page from Varnish despite having an IP that will give them an authenticated role.
Senior Drupal/PHP Developer - Wellington, New Zealand | HeadFirst Limited
HeadFirst are a web development company based in Wellington, New Zealand. We provide enterprise-grade Drupal-based solutions for public sector, corporate and select start-ups. We are looking for a full time Senior Developer/Technical Lead proficient with PHP and Drupal. This is a hands on position - you will be responsible for leading a team of 3-4 developers building Drupal solutions for our clients.
Access Denied, You are not authorized to access this page
I am going crazy because of this problem ... Become and you get a lot for no reason ...
I have this problem since several months have been circumvented by deleting www .... Noted after several days of normal operation of the website ... But in recent days has become a continuously and, more disturbingly ... when I add new node or edit node ... I normally logged I can add the node and after that I have to log-in again
I have had this problem and prevented me from entering the admin panel until I updated to latest version 6.22 but still happened to me
Regards,
Drupal Scalability and Performance training at SANDCamp 2012 on January 26, 2012
Join us at SANDcamp, the San Diego Drupal Camp, on Thursday, January 26, 2012 for a full day of scalability and performance training! This class is being offered at a 50% discount off our normal prices. Sign up today or contact us with any questions.
| Sign up today at http://www.larks.la/training |
In this all-day workshop, top experts with years of front-line experience building large-scale Drupal deployments show you the best known practices for running fast sites for a large volume of users. Starting with a blank Linux virtual server, we will work through the setup, configuration and tuning of Drupal using Varnish, Pressflow, Apache, MySQL, Memcache and Apache Solr.
Through this process, we will explain common pitfalls and how to avoid them. By the end of this course you should feel comfortable with the basics of how to setup, configure and tune your Drupal site. You’ll learn the most robust solutions the community has discovered.
Which Linux flavour to choose?
Hi,
Found VPS provider for aegir instalation (up to 15 D6-D7 sites, 1/2 of them corporate brochure type), it will be openVZ with guaranteed 2048Mb RAM, mysql+php+apc+apache+smtp, and other basic services, maybe Apache Solr. Memcahed/Nginx/Varnish is a possibility if performance benefits over simple Boost module+APC are worth pain having another service on same vps.
Which linux to choose from this list:
CentOS 4
CentOS 5
CentOS 6
Debian 4
Debian 5
Debian 6
Fedora 14
Fedora 15
openSUSE 11.3
openSUSE 11.4
Ubuntu 10.04
