Hi all,
So I've been trying Phalcon the last few days and also started making my own extensions today using Zephir. Zephir is basically a high-level programming language that makes compiled PHP extensions. So it's supposed to be like the best of two worlds - compiled performance and easy development.
Read more here: http://zephir-lang.com/
I started using it today and since I had nothing better to compare it with I started porting some of the functions found in Drupals common.inc, just for learning. And it is very easy to use.
I did some of the functions and then basically looped each of the functions 10000 times with randomized input variables. I also verified that the functions where giving the same output. And this is the result I got in comparison to built in functions with PHP + APC:
- drupal_add_region_content() - Zephir extension is around 40% slower then PHP.
- drupal_get_region_content() - Zephir extension is around 40% slower then PHP.
- drupal_get_query_parameters() - Zephir extension is around 30% faster then PHP.
- drupal_get_region_content() - Zephir extension is around 35% faster then PHP.
- drupal_http_build_query() - Zephir extension is around 35% faster then PHP.
- drupal_get_destination() - Zephir extension is around 500% (!) slower then PHP.
- drupal_parse_url() - Zephir extension is around 25% faster then PHP.
- drupal_goto() - Zephir extension is around 100% slower then PHP.
- drupal_strip_dangerous_protocols() - Zephir extension is around 15% slower then PHP.
Basically it was just a test for me to learn Zephir, but my small conclusion from this is that even the good numbers does not come close to the figures that I have seen in other compiled extensions like creating native C PHP extensions or using HipHop.
But it is easy to learn though.
I have attached the zep file to this post as well as the common.inc that replaces the functions above if anybody wants to try it out.
Thought I would post it here as reference if anyone in the future wants to try out Zephir + Drupal.
Edit: Note that I am in no way an expert at Zephir, this is the first time I use it, so the script might be more optimized by an expert.
Edit 2: There is now a repo: https://github.com/ivanboring/Drupal-Zephir
Edit 3: I wrote some thoughts here: http://www.drupaldare.com/compiled-drupal-core-zephir
| Attachment | Size |
|---|---|
| common.tar_.gz | 86.68 KB |
| common2.tar_.gz | 86.6 KB |
Comments
nice work, as I am really
nice work, as I am really interested in performance I'd like to jump on this as well.
how did you choose the functions you tried to replace? ever considered looking at a xhprof result to see where the most impact might be done?
Not really, I just took
Not really, I just took common.inc because I though it would give a multitude of different type of functions, which would be good for learning.
So the performance test was really a secondary part of why I did it, but from what I can understand looped computations is where Zephir (and most compiled stuff) does really matter, and that is what I could see in my tests as well.
I know this exists also:
https://www.drupal.org/project/drupal_php_ext
I guess I could rewrite those in Zephir as well (to get some more training) just to compare figures to "real" compiled extensions. I'm guessing they choose those functions because they are "good" functions to see performance gains.
it would be interesting to
it would be interesting to figure out why the performance drops are so huge on some methods, maybe because of context switches between the zephir extension and native php? I saw a couple of slow implementations calling to drupal_parse_url() for instance - will this use the static one from the module or will it use the user space one? (right now I would think the later)
The thing is actually
The thing is actually regarding to context switching is that I tried drupal_get_destination() with both using the native php function (as it is written in the file dump)
let query = "" . drupal_http_build_query(drupal_get_query_parameters(NULL, ["q"], "", _GET));and using the extensions function (since I had made those two)
let query = "" . self::drupal_http_build_query(self::drupal_get_query_parameters(NULL, ["q"], "", _GET));I saw no difference between those two implementations in regards of computation speed, so I don't think it's because of that. One pattern I saw though was that any function using regex got a lot slower in Zephir since Zephir had to use PHP regex function for this. I tried valid_url and just copy&pasted more or less, and it was a lot slower.
how did you compile the
how did you compile the extension?
see https://github.com/phalcon/zephir/issues/694
I tried compiling with both
I tried compiling with both static type interference and local context pass and without and saw no real difference.
http://zephir-lang.com/optimizations.html#static-type-inference
I also tried with default build and prod mod without any difference.
Zray instead of xhprof
Zray debugging may be useful for this sort of thing. Give it a try, I use it a lot to eek out performance issues in Magento extensions. Zend Server has a built in plugin for debugging and tracing drupal.
The function tab can really help sort out the slow bits. Give it a try.
Can be better
Zephir author here. The implementation can be better, for instance, Zephir extensions have different challenges than PHP. Allowing the option to override a method makes Zephir lose an option to produce a better code, marking methods as final should allow Zephir to produce inline caches that should perform better.
This line is calling userspace functions so probably there are no caches there:
let query = "" . drupal_http_build_query(drupal_get_query_parameters(NULL, ["q"], "", _GET));These lines:
if isset _GET["destination"] {let destination = _GET["destination"]];
}
can be replaced by:
fetch destination, [_GET["destination"];This:
if isset parts["fragment"] {let options["fragment"] = parts["fragment"];
}
By:
if fetch fragment, parts["fragment"] {let options["fragment"] = fragment;
}
That's awesome having the
That's awesome having the creator here :)
I did the changes that you wrote - it totally makes sense that the class and it's methods should be final since there is no need to extend them. Also I learned what the fetch function is for, thanks! Regarding user space functions, I tried calling them within the namespace as well without any difference.
I also tried Zephir's type_of == "array" instead of in_array(), but saw no difference in performance or memory usage.
So doing the final thing and userspace/namespace really did no difference for performance what-so-ever. Is there something else that I'm doing wrong or could be optimized? Sorry for asking but it's quite literately the first time that I am working with Zephir. I attached the code (common2) with the changes in the original post. (And I do understand if you don't have time answering)
I did install xhprof as suggested by Andre-B so I will have a quick look at what functions seems most likely to be able to be improved (high computation, low i/o if I understood correct).
In the tests besides iterating the functions thousands of times, I also did test with running the functions the "normal" amount of time for a Drupal platform just to make sure that it does not exists some extra load that only shows itself on extreme multiple calls. But that was not the case.
Some examples of xhprof dumps just to see after I added final:
drupal_build_http_query()
PHP
Calls: 10000
Incl. wall time (ms): 305,811
Excl. wall time (ms): 136,689
Incl. CPU (ms): 302,180
Excl. CPU (ms): 67,295
Incl. Memuse (bytes): 16928
Excl. Memuse (bytes): -3748352 (releasing more then allocating?)
Incl. PeakMemUse (bytes): 12600
Excl. PeakMemUse (bytes): 5008
Zephir
Calls: 10000
Incl. wall time (ms): 190,206
Excl. wall time (ms): 20,319
Incl. CPU (ms): 199,577
Excl. CPU (ms): 16,527
Incl. Memuse (bytes): 10808
Excl. Memuse (bytes): -953416 (releasing more then allocating?)
Incl. PeakMemUse (bytes): 7896
Excl. PeakMemUse (bytes): 1168
drupal_get_region_content()
PHP
Calls: 10000
Incl. wall time (ms): 21,585
Excl. wall time (ms): 11,398
Incl. CPU (ms): 29,786
Excl. CPU (ms): 16,730
Incl. Memuse (bytes): 1856
Excl. Memuse (bytes): -1358936 (releasing more then allocating?)
Incl. PeakMemUse (bytes): 1616
Excl. PeakMemUse (bytes): 944
Zephir
Calls: 10000
Incl. wall time (ms): 42,273
Excl. wall time (ms): 21,467
Incl. CPU (ms): 52,174
Excl. CPU (ms): 23,484
Incl. Memuse (bytes): 2952
Excl. Memuse (bytes): -2238648 (releasing more then allocating?)
Incl. PeakMemUse (bytes): 2544
Excl. PeakMemUse (bytes): 1232
Too many PHP function calls = not much difference
Hi,
If you have too many PHP function calls, it wouldn't be faster. You should write complex logic with Zephir to be faster.
http://blog.ohgaki.net/phalcon-zephir-language
(It's written by Japanese, but you may try to use translation services. You can understand code at least.)
This is a OWASP recommended javascript string escape function implementation.
As you can see, original attempt didn't get much performance improvement. (10% faster) If I eliminate PHP function calls, it became 400% faster. (Note: there is a little cheat to eliminate PHP function calls)
大垣 靖男 / Yasuo OHGAKI / yohgaki@ohgaki.net
http://www.ohgaki.net/ / http://www.es-i.jp//http://www.provephp.com/
Yeah, you are completely
Yeah, you are completely correct on this. I transferred the drupal_strip_dangerous_protocols() from:
while(true) {
let before = uri;
let colonpos = strpos(uri, ":");
if colonpos > 0 {
let protocol = substr(uri, 0, colonpos);
if(preg_match("![/?#]!", protocol)) {
break;
}
if !isset allowed_protocols[strtolower(protocol)] {
let uri = substr(uri, (colonpos + 1));
}
}
if(before == uri) {
break;
}
}
return uri;
which was basically a rip-off of the drupal core code and did this instead that has the same logic, but without substr and preg_match which are PHP functions (I wrote this quick, so I'm sure I can make the code more beautiful, but it works):
let uri = strtolower(uri);
for ch in uri {
if allok {
let returnuri .= ch;
}
elseif ch == '?' || ch == '#' || ch == '/' {
if foundcolon == true {
if isset allowed_protocols[tempuri] {
let returnuri .= tempuri . ":";
let tempuri = "";
} else {
let tempuri = "";
}
let foundcolon = false;
} else {
let allok = true;
}
let returnuri .= ch;
}
elseif ch != ':' {
if foundcolon == true {
if !isset allowed_protocols[tempuri] {
let tempuri = "";
}
let foundcolon = false;
}
let tempuri .= ch;
}
else {
let foundcolon = true;
}
}
let returnuri .= tempuri;
return returnuri;
And the result was that it was 45% faster instead of 15% slower. But there is more that can be fixed on top of that, because right now I'm sending the allowed_protocols variable in from PHP via a static variable that is run through an array_flip in PHP. If I were to move all this and create an array_flipish function in Zephir I'm sure it would be even faster. The actual Zephir code is 300% faster then PHP not including the static variable.
But the problem as I see it is that you have to write pretty low-level logical code here anyway. And if you can't use PHP functions it will be fairly slow to rewrite stuff, so why would you not make it in native C instead of writing limited PHP code only using the Zephir methods (which are like 16)? Maybe someone should write a PHP class for Zephir :)
I will see if I have time to rewrite all those that is possible from the classes I made already that does not touch the Drupal API and do benchmarks again (drupal_goto wont work for instance without remaking a lot of other classes).
nice progress here. I don't
nice progress here.
I don't have a xhprof dump locally right now, but I guess you have, maybe there are further "easy" candidates that are called a lot of times that could be optimized? drupal_alter maybe - though this probably calls a lot of other stuff.. something theme related?
I started looking around with
I started looking around with XHProf and found that drupal_attributes() that is run fairly often on a vanilla Drupal installation has only one child in check_plain(). So I fixed them and the gains were as follows:
drupal_attributes()
PHP
Calls: 106
Incl. wall time (ms): 2,509
Excl. wall time (ms): 1,382
Incl. CPU (ms): 3,667
Excl. CPU (ms): 1,232
Incl. Memuse (bytes): 26352
Excl. Memuse (bytes): -18616 (releasing more then allocating?)
Incl. PeakMemUse (bytes): 8584
Excl. PeakMemUse (bytes): 7024
Zephir
Calls: 106
Incl. wall time (ms): 883
Excl. wall time (ms): 230
Incl. CPU (ms): 0 (!)
Excl. CPU (ms): 0 (!)
Incl. Memuse (bytes): 58768
Excl. Memuse (bytes): 17784
Incl. PeakMemUse (bytes): 8888
Excl. PeakMemUse (bytes): 1024
So it was much faster (>60%) but also ran much more memory. Shows that you can speed things up at least, that's almost 2ms cut on a total of 122 :)
Repo ?
Is there a repo for this ? I feel like many of us could contribute to speeding up Drupal that way.
would be nice if Marcus would
would be nice if Marcus would put it on github, drupal sandbox would probably lack the pull requests and forking...
Here you go
Here you go guys:
https://github.com/ivanboring/Drupal-Zephir
Awesome, thanks !
Awesome, thanks !
php to zephir conversion tool
Hi
i created this tool to allow building php extensions with just php.
I believe combine this tool and the tips found here, we can produce better code
https://github.com/jimthunderbird/php-to-c-extension