Hello All,
Our new Top Websites site is live (though still in beta).
http://top-websites.burtronix.co.za
Some initial countries: UAE, South Africa, International, Japan and the Czech Republic (capital is Prague, where the next DrupalCon will be held).
It still features the top Drupal websites in South Africa (http://top-websites.burtronix.co.za/drupal/south-africa/2013-08-05), but now also lists several other countries and top websites charts for Joomla! and WordPress.
For the technically inclined:
I'm now running the statistics gathering scripts as PHP CLI scripts that are much faster that the previous Drupal-based solution. I'm also storing data YAML format which is the new configuration file storage format for Drupal 8. The previous statistics gathering scripts ran for several days per chart, the new scripts finished the job for all the countries and CMSes in just a couple of hours.
Essentially, I obtain the list of top websites per country from Amazon through their web services, I then strip out all domains not in the ccTLD of the country and proceed with analysis. For analysis I grab the logic from the Wappalizer project and programmatically turn that into PHP logic; I then grab each top domain's http://, http://www, https:// and https://www versions and analyse the first that presents a 200 HTTP code. Finally, I run all of the found websites that were detected to run one of the CMSes past PhantomJS to get a screenshot for each.
The scripts aren't perfect, but since this was a weekend project, they can be made better in time.
Finally, the YAML files those scripts spit out are imported in one Drupal text field with a custom formatter to turn them into top charts. In time we'll gittify the source scripts, use Composer to get supporting libraries, base a lot more on Symfony (which supports newer YAML), write the scripts with more robust handling of sites (it simply skips a site if it can't connect to it the first try right now) and much more automation (still have a workflow around the generation of new charts right now).
The Amazon top list is based on usage of the Alexa toolbar, so if your site is not in the top list and you think it should be, simply install the Alexa toolbar for your browser.
Suggestions are very welcome!
Kind regards,
Riaan Burger
Comments
Added stats for China, India and Brazil
I added the top (other than the 'States) three internet using countries' statistics.
We now have the new stats for China, India and Brazil on top of the existing one for International, South Africa, Japan, the United Arab Emirates and the Czech Republic:
http://top-websites.burtronix.co.za/
Limitations to the script and to my IPv4 link may be what's causing so few Chinese sites to be listed.
Notable is India where Drupal websites outnumber Joomla! ones.
New Stats Added
This weekend I added stats for Germany, Russia, Switzerland and Nigeria.
http://top-websites.burtronix.co.za/
United Kingdom, Egypt, France, Mexico, Turkey and Vietnam
I'll update the South African statistics monthly, but if any of you are interested, I added the following countries to the list of statistics this week:
http://top-websites.burtronix.co.za/drupal/united-kingdom/2013-08-29
http://top-websites.burtronix.co.za/drupal/egypt/2013-08-28
http://top-websites.burtronix.co.za/drupal/france/2013-08-28
http://top-websites.burtronix.co.za/drupal/mexico/2013-08-28
http://top-websites.burtronix.co.za/drupal/turkey/2013-08-28
http://top-websites.burtronix.co.za/drupal/vietnam/2013-08-28
Interestingly, the additional traffic generated by the top sites list means that, by the addition of South African traffic alone, the burtronix.co.za domain is now ranked #1 629 in SA and #269 303 worldwide on the internet.
http://www.alexa.com/siteinfo/burtronix.co.za
That would put the burtronix.co.za domain at the #15 Drupal site in South Africa on the next run (if it was run today), incidentally replacing one of the other websites we manage: http://top-websites.burtronix.co.za/drupal/south-africa/2013-08-05
Having run several Google Ad campaigns before and managing several clients' websites that have valuable informational resources online, this just illustrates how much more valuable a good service can be in highlighting one's brand online. The ad campaign was paid for and its results pale in comparison to the return on investment while the top sites also gives back to the community. We'll definitely be doing more of this and have several good ideas int he pipeline.
Links not working
Hey Ryan,
That's awesome work you are doing.
I was trying to get to your website but the links in it are not working. I was wondering if you had shifted the website?
Nope
Hello Hars,
Nope, they are still in the same place:
http://top-websites.burtronix.co.za/
http://topdrops.org/
I did change servers several times and just now realised the first site uses some YAML library I'll have to go install quickly to get it's parsing to work. I'll do that just now.
Also, the data for the first site is very old by now. The second site still updates once a week, but it was also a weekend job some years ago. I'm slowly working on a very fast, much more functional version that will keep data up to date with weekly charts and is built with Node.js. Should be up in the next couple of months.
Kind regards,
Riaan
There we go
The YAML lib installed, all should be working perfectly now.
Working Perfectly
Thanks for the update, both the websites are now working perfectly.
I wanted to see how scalable a drupal project can be. I am starting a project which would have 20k+ active users. Your site's data is very helpful to see what other such drupal websites are like.
I look forward to your new website, should be awesome.
Thank You
What level of visitors are needed?
Hi Riaan,
Great work and good to show prospective clients. How many visitors does one need to get onto the list? Are there other criteria?
There are 10 kinds of people in the world, those who understand binary and those who don't.
Not that many...
If you regularly have a couple of hundred sessions reported per month on your Google Analytics, you should start seeing the site on topdrops.org.
The only other thing you need to do it to, well, run a Drupal website ;-)
The upcoming new site will once again also track Wordpress and Joomla! sites and you'll be able to select a TLD to only see one's statistics. It will have an open API and the main website, as well as several widgets will run on that API, so it will be easy to create your own site badges, for example, to put on sites you make or your portfolio.
Just checked...
Burtronix.co.za had around 3 000 per month when it was doing well on the charts. The problem Hars reported may have been around a long time, or a Google update may have caused the site to drop to roughly 700 per month from May 2014 and it fell off the carts when it dipped below that a few months later.
I'll have to pay some attention to that some time ;-)
:-)
My site www.claybrick.org gets around 8 500 sessions a month - I will look for it! It is a Drupal 6 website - we have budget for an upgrade to D7 in 2015.
There is also :
cbi-lowvoltage.co.za (over 6 000)
clearline.co.za (just under 4 000)
osborn.co.za (just over 3 000)
And I have quite a few between 1000 and 2000.
I have obviously not been looking hard enough. Are you sure I don't have to register them somewhere first? Or switch on a special module?
There are 10 kinds of people in the world, those who understand binary and those who don't.
Alexa's stats
Easiest to check on Alexa:
http://www.alexa.com/siteinfo/claybrick.org
http://www.alexa.com/siteinfo/cbi-lowvoltage.co.za
http://www.alexa.com/siteinfo/clearline.co.za
http://www.alexa.com/siteinfo/osborn.co.za
If they're in the top 1mil, they will start being listed on the topdrops.org website.
If you install the Alexa toolbar (and I generally don't, for privacy reasons) Alexa will become aware of sites you visit, but it won't really boost your sites in the ranking. They're working hard to prevent artificial boosting and give an even rank and they're aware of all your sites.
I speculate here, but there can be two reasons for them not appearing higher that I can think of:
Perhaps you use the services of some SEO firm. Unless you're forking out big money on that to a reputable firm, most of them sell snakeoil and more often than not just want your money in exchange for altering some metric (Google Analytics hits or higher for keywords in Google search results). Intelligent algorithms will ignore their work.
Maybe the sites serve a small audience who visits regularly. Gain rank with a larger audience. Usually sites that make it higher in the ranks serve wide audiences. It's not very likely that a SME site not offering an online service or data resource that's widely used will make it high on the list.
Quality
I may add, ranking doesn't necessarily speak to the quality of the website. I've seen some awful sites on there. Your sites look really good.
One thing you can do that will help your SEO a bit in every way is to serve only one copy of each site. For example, you currently serve these two:
http://www.cbi-lowvoltage.co.za/
http://cbi-lowvoltage.co.za/
So in effect, two copies. To solve that, edit your .htaccess file. You'll see a well commented section that will help you redirect www.cbi-lowvoltage.co.za to cbi-lowvoltage.co.za (without www). It will take a while to be spidered but for Google you can use their Webmaster Tools to help the process along a bit.
htacess
SEO consultants make used car salesmen look honest. Actually, they make Zuma look honest :-)
Unfortunately clients so much want to believe in snake oil!
There are 10 kinds of people in the world, those who understand binary and those who don't.