Application Server in PHP? well… Yes!

Interesting project,I need to start checking this out. I found one “problem”in your above post:

“It needs to have two methods:public function __construct and protected function requestHandler.”

Time for an Interface? Instructions like this will then be of the past. You simply say:“Create your class by extending SCGI_Application and implementing SCGI_Interface.”

Stefan:yup. there will be interface at some point

All this thing is very “adhoc”at this moment

Nice. I´ve written a “quick and dirty draft-script”for fcgi.
Just reading the packets and so on,so that the passed data can be used in a script.
While it´s basically no problem to write this in PHP,I´m not sure if the performance will be “good enough”.
My primary intention for this is an authorizer and an fcgi-alternative should be much faster than a legacy “.htaccess-db-auth”.
So I´m thinking about writing an extension instead.
(If I can find some time)

Thilo:fastcgi extension is a “big aim”for me. I hope to get there sooner or later.

Can you explain or show example hot to get it work with Apache2 and mod_scgi?

Sergey:
I believe you should add something like this to your config:

SCGIMount /scgi 127.0.0.1:9999

this will forward all http://www.yourhost.com/scgi queries to the application. You still have to start app manually (with “php runner.php”,if you are trying my example-app)

Why would anyone want to do this? It sure offers great,new possibilities to php-devs and makes nonestop php-rocking possible,but why would I want to do this? You people have to stop trying to do everything with php. Learn other languages and use them for such cases,there are better solutions than this. It’s not my intention to flame,I’m just making a point:In theory I could wipe my ass with a piece of sandpaper,but that’s not what it was made for.

And to leave some more useful comment,since noone’s gonna change after reading this:
If you want to do it with php,why not write a server in php and set up a proxy in apache? I think it would be faster than any (s/f)cgi-solution

Yes its working
I see information page and Memory usage:
number is increase with every request. This is a PHP memory leak or is this the way it should be?

Wow,I just started looking for a fcgi/scgi implementation a few days ago,and here it is. It’s something we need at the company I work and we’ve even decided to switch to Java or Python,just because php didn’t seem ready. Now we may have to reconsider.

The project is very interesting but what I want to know is how much useful would it prove,given that “the php-way”(if there is such thing) is to optimize loading,cache everything and be in general very quick-and-dirty about what you do. It’ll all be cleaned up at the end of the execution anyway. I expect not many frameworks would switch over to fcgi/scgi,just because it would cause too much trouble. Even ORMs,like Propel,aren’t ready for this approach. I’d like to see it,anyway.

It’s quite different for applications with a very specific purpose,of course.

PHP is freeing up the memory after php script finishes unless it is done manualy,that’s the reason why memory usage increases.

about an half a year i thought about controller which is looped into infinity. let’s say do…while and can be broke by somethig and inside is controller. in php.ini we set max-execution time to hmm.. big numer and try with streams? i send query (action=get_names) to port 123456
and i get response [mike,stefen,alex] etc.
I stil doesn’t tried this. maybe in the future

Daniel:actually I do know (and use) other languages. C++,Objective-C,Smalltalk,Haskell,Ruby to name a few

The reason I do this,because I really understand benefits.

I am definitely not a newbie

And no,proxying http is not a better approach than FastCGI (while quite comparable to SCGI). FastCGI uses a constant connection and has an effective binary protocol

Sergey:If you watch it for a longer time,then you would notice,that mempory stops growing at some point — that’s a point of balance reached by garbage collector. At least,that’s what I see.

If memory keeps leeking at your side,probably that is some buggy php-extension — try disabling ones which are not used.

gasper:why do you think,that Propel wouldn’t work well sith SCGI/FastCGI?

I think it will work quite nice indeed

kaszu:php cleans all “leaked on php-level”memory on the end of request and cleans “officially”unused memory at runtime using garbage-collector…

So,my SCGI solution shouldn’t bring any abnormally higher memory-usage.

Something like this has been on my todo list for a few months now. My idea was to take libevent and use it’s http server to handle the requests,then embed PHP directly so it’s all executing in the same namespace for an app server. The way I was considering it,it wouldn’t have the benefit of running all of the time,but should have been about as fast as you can get a one-off execution of PHP to respond to an HTTP request.

[...] Para quem ficou interessado,esse é o link:http://blog.milkfarmsoft.com/?p=51 [...]

Interesting indeed. One note about memory management in PHP:PHP uses reference counting to account for memory allocation,so values not used anymore are immediately reclaimed. However there’s two catches to that:

1. Refcounting does not like reference cycles. If you have something like $a->b = $b;$b->a = $a;recounting won’t be able to reclaim the memory once $a and $b are not in use anymore. It’s no big deal for regular scripts since it’s all cleaned up at the end of the request anyway,but for long-living scripts it may be a problem,so extra care should be taken to avoid it. Python has cycle-breaking garbage collector AFAIK but PHP doesn’t have one.

2. PHP has own memory manager,meaning freed memory is not returned to the system (well,occasionally it is if you free very large consecutive block) but reused for later allocations. Meaning the system memory usage may grow but it doesn’t mean if would grow forever –only to max size of the data used simultaneously.

You can also disable PHP memory manager by setting USE_ZEND_ALLOC=0 in environment,then you could see everything going to system malloc/free.

Travis:the thing which you propose sounds like “mod_php on drugs”… I guess you can get similiar effect by writing php-module for lighttpd or nginx

This is very promising,thanks a lot! I remember I started a thread http://forum.agiledev.ru/index.php?t=msg&th=821&start=0&(russian) at agiledev.ru looking for possibility to have application servers with PHP but to no avail at that time…But now things seem to be getting much better in this direction,please keep your good work!

As for “smarter”GC in PHP,I remember I asked Derick Rethans at PHPConf2007 in Moscow about this issue and he promised to present a better GC implementation(being one of GSOC projects) by the beginning of fall 2007.

Pavel:
hi
Yup,I’ve been at PHPConf too. I am the guy who helped you with answering questions about AOP

What a small world

P.S. your blog rocks!

Isn’t this like Vulcan Logic SRM (Script Running Machine)
http://www.vl-srm.net/doc/faq.introduction.php

Yann:it is similiar in some aspects,yes
thanks for the link

Does it support multiprocessing? I was wondering how 1 php instance can handle several request at the same time. Since I can’t get mod_scgi to work with apache 2.2.4 there was no possibility for testing this

regards

Alexey Zakhlestin’s Blog:Application Server in PHP? well…Yes!…

…

Daniel:not yet. At some point I will add multi-processing using pcntl_fork

I would prefer threads,but we don’t have those in PHP

[...] In a new post today,Alexey Zakhlestin talks about “limited implementation”of the SCGI protocol that he created in PHP. I finally found some time (and inspiration) to do something in direction of implementing FastCGI the way I see it. Initially,I was going to implement FastCGI-functions in php-extension,but that would require more time than I currently have,so I started with a simplier task:I implemented SCGI protocol (which is way simplier than FastCGI) in pure php-code (which is easier,again,and let’s me change API faster,during development). [...]

I’m actually working on a project with a slightly different approach (with writing a complete server API,using pcntl indeed for multiprocess). At the moment I’m looking for a solution for MS platform process handling.
As this will be a diploma work I will make it public after I have my paper in my hand.

[...] Alexey Zakhlestin escribió en su blog acerca de su proyecto,“Application server in PHP“. [...]

Peter:what do you mean by “complete server API”? You are making http-server written in PHP?

Well,about garbage collection:PHP cleans up when execution finished. Well,with fcgi/scgi it actually doesn’t finish after a request. The service remains running and ready to server another request. All the memory that gets used by global variables and singletons remains in use and isn’t cleaned up. If you have such a memory leak in an ordinary php script you’d probably never noticed,while fcgi is a different kind of animal. A very simple example would be a singleton or a static class that stores data,which isn’t actually important enough to persist throughout the application,but eats up memory anyway. With mod_php,this is never a problem.

As for the Propel;I don’t think Propel wouldn’t work with memory-resident application,I just think it isn’t well suited. For example,two queries for a same record return different objects,because there is not Identity Map or some sort of global object repository. Again,this is rarely an issue for an ordinary php script (although I’ve had this problem already),but it’s suboptimal for an application server which could benefit substantially from these approaches.

I actually don’t think these issues are a blocker,but I’m pretty sure writing frameworks would noticeably change (ie bye-bye to the class/config loading caches).

“I would prefer threads,but we don’t have those in PHP”

Not complete,yes. But I have a working version of pecl/threads on windows where the execution of included script-files in threads doesn’t crash the runtime at shutdown. But there is a real big problem in synchronising those threads in the current version and I don’t know how to deal with it at the moment. It looks like the main thread needs to control the childthreads destruction manuelly and also the PHP-internal functions for destroying executed scripts have to be reimplemented in this extension I think.

regards

gasper_k:
well,global object repository is easily implemented for those propel-classes which really need it. proper caching is never an easy task,but it can be done.
I believe,my approach lets people to implement more ways of caching

[...] Thanks for all the comments to my “Application Server in PHP? well… Yes!”posting. I was really amazed to see that many interested people. Your support gives me a great motivation. Thanks,guys! [...]

indeyets:well,your approach doesn’t in general limit anybody in implementing any kind of feature,that’s for sure. The question I was asking was if fcgi even could be interesting for php developers since it brings some issues with it? I’m quite aware it can be very useful for specific solutions,but it may be too much of a hassle for an “ordinary”php application (script,page,whatever) to be run that way. A lot of code has to be changed to weed out the aforementioned issues.

Anyway,I’m glad you’ve written this code,it’s a very interesting experiment with a lot of potential,no matter the issues.

I really like your approach,even if it is just a “proof of concept”.
It proves me one thing:your little PHP program –with nothing useless in it –is already leading to memory leaks:

Hello world! #21
Memory usage:208224
–
Hello world! #51
Memory usage:239632
–
This,just with F5 ! That’s why,now,PHP cannot be really used as a program:you just cannot launch it for hours …

Une application serveur en PHP…

Je viens de tester une solution d’application serveur en PHP. Le principe est simple:une application php est lancée en mémoire,de manière continue (elle reste en mémoire). On économise donc le temps de chargement des fichiers par ex….

Olivier:did memory-usage still grow at 51 or was jumping up’and’down?

I see it stabilizing at some poing

[...] There was a comment by Olivier to one of my previous posts mentioning,that php was still leaking memory even on the simpliest example. That’s not exactly true. I added some more memory-information output to that example and made “hard”testing by ab. Here are some results:[...]

indeyets:By complete server API I mean,that I want to create an application server API,which can be used to create (theoretically) any type of servers,and it would enable to do this on the same machine. Something like in in IBM WebSphere’s world.

indeyets:
Very similar indeed I actually tried the solid PHP route to see if it would work but performance was dismal. Hitting more than 10 concurrent requests using HTTP_Server and ext/pcntl would die at the 9th request. The actual HTTP request handling needed to be something else –that’s where libevent came in.

In order to get persistence I was going to have to go with a FastCGI type implementation to keep PHP running and have libevent just pass off to running processes –though the mod_php kind of route where the PHP script is executed within the server and returned probably would have been my first pass. I can live with single execution and not keeping things in memory –memcache can help me with that persistence –but I can’t live with the overhead of something like Apache or even lighttpd (so long as it uses FastCGI).

[...] In a new post,Derick Rethans talks about circular references (a reference of a memory structure back to itself) and how they relate to PHP. Circular-references has been a long outstanding issue with PHP. They are caused by the fact that PHP uses a reference counted memory allocation mechanism for its internal variables. This causes problems for longer running scripts (such as an Application Server or the eZ Components test-suite) as the memory is not freed until the end of the request. But not everybody is aware on what exactly the problem is,so here is a small introduction to circular references in PHP. [...]

[...] PHP 的 Application 層 Published in October 26th,2007 Posted by siko in PHP,轉載 請參考： Application Server in PHP? well… Yes! [...]

Implementing SCGI is a great idea indeed. You could have your PHP App start up,load all those classes,initialize some shared memory and then start some processes with pcntl_fork() for multiprocessing which run the accept()…If you add SHM to the mix you can have persistence even accross processes. The sad thing is that this isn’t supported on a variable level,that’s where __get() and __set() could come in in your classes.

The PHP socket API also supports select(),that’s what I used once to create a single process PHP Chat Server. The Problem is that you usually want all the other parts to be non-blocking,like MySQL queries and so on.

Want to thank you for your idea. I am basing my master thesis on it. It’s a great pity there is no FCGI extension from you yet SCGI does not multiplex,and I would use it.

BTW,reading the comments to this post …Guys,forking leads to nowhere,there is better approach,called request multiplexing.

Would like to show you what I’ve done so far,but this stuff is rather complex. According to my time schedule I am going to have some documentation to the project in about a month. So if you still want to create a real application server in PHP,you could eventually join the project when I’m done with thesis.

In my case,the expected advantage of using the SCGI application is to minimize PHP code load time.

So,we could implement the following method:

1. Load all PHP classes (e.g. 5MB).
2. Fork using pcntl_fork() and keep a number of always running processes.
3. On each request,pass it to a process.
4. After the request is processed,exit() within the process to clean the memory (it solves problems with circular references).

Because in Unix fork() is wise (it performs memory “copy-on-write”),it should be quite cheap.

What do you thing about such method?

Sounds like a sane solution. True multithreading is impossible for php in any reasonable timeframe,so,this is a way to go.
I still think,that proper application-server would be much cooler (shared objects,fast global cache),but this is better than nothing

I think it's a quite good idea! We can run few scgi php application (like mongrels in Ruby On Rails) and balance them via Nginx and we can use MemCached for fast global cache! I'll try this solution maybe next week!

Just remember,that you will need to adapt application to use my SCGI class. It is not a direct replacement for usual PHP's SAPIs

Hello Alexey,

I have started to use your library and has been working great.
I have found some problems with the HTTP/PostRequest.class.php
apparently you have to replace throw new RuntimeException to throw new RuntimeException else the system looks for it in the MFSAppServerHTTP namespace.

PS:I would like to join on your google project for appserver in php.

Similarly,I believe throw new LogicalException should be throw new LogicException (Logic not Logical),and for these to be caught in MyApp,_construct needs try/catch:
try{
parent::__construct($socket_url);
}
catch (LogicalException $e){
echo 'Caught exception:',$e->getMessage(),“n”;
}

With Cheroke http server you can define multiple instances (Information Sources —type=Remote Host) of the AppServer running on different ports and let its built-in Round Robin capability distribute the load.

I'm new but open to the idea of running PHP this way. Keep up the great work!

Thanks for the heads up.
I am doing the same the cherokee+10 instances of the app per server.
The PostRequest.class.php has to be sorted out though as it is not accepting requests from pear's HTTP::Request class (from php4)
PS:This is only when the request is using multipart/form-data.
we will see what can be done to fix it.

best regards and thanks for your input.

I just moved project to github. You can find it here:http://github.com/indeyets/appserver-in-php/
feel free to make github-fork of it. I would be glad to pull relevant patches from your tree.

issue with exceptions is fixed there. tell me if I missed anything

I fixed the issue with exceptions. new code can be found at github here:http://github.com/indeyets/appserver-in-php/

regarding catching LogicalExceptions — I am not 100% sure,that is a good idea. LogicalException means something really bad,so should be treated as such. Anyway,you are free to do whatever you want in your App-classes

Thanks. I will check it out.
The next thing would be to fix the multipart post handling.
We are getting many error when posting from pear_http_client in php4 to the
scgi app.

I will see when I can give you more informations and test case for it.

I am totally new to Exceptions,so any such “tips”are appreciated.
In my case the LogicException was trying to tell me “You have to run this from the command line!”Oh,you mean it's actually running all the time like a…server. Heh.
It's so counterintuitive —I love it.
Loading all your variables,objects,arrays,XML,etc. before receiving a single request. Brilliant! Leaves me wondering why this approach is not more common.

I recently coded fcgi php application server! It works fine with NGINX web server on FreeBSD. I tested it on sample application and got no memory leakage at all. Here is the link to project's google code page http://code.google.com/p/ultramilk/

As I promised 5 months ago,I have successfully finished my master thesis Distributed web framework. Basically,in your terms it is a distributed application server for PHP. It is currently using the SCGI protocol to communicate with web servers and the MCGI protocol (multiplexed SCGI) for inter node communication.

Check it out on http://bondari.net/?p=133

For those who interested I recommend to start with the presentation and not this the paper.

Big thanks to vintikzzz for the FCGI implementation.

I read your presentation. That's quite a beast. Reminded me of http://xscript.opensource.yandex.net/

Do you plan to make any user-level tutorials for you project? I mean:how difficult would it be to write “hello world”app?

Cool. I will look at it.
I noticed,that you use LGPL license. Is that a prinicipal moment?
My code is BSD-licensed,so,I won't be able to use your code in my project,for example

Paper chapter Evaluation contains the description of several examples,short but functional applications,including the very basic content management system. These applications are very small and pretty well documented.

General audience won't read paper. Then need short and easy step-by-step examples

Hmmm…I want to use this code with framework that under LGPL licence too!

You can use BSD-licensed code in any project (BSD,LGPL,GPL or even Close-sourced),as long as license is mentioned.
BSD-license is really simple an gives maximum number of freedoms,which is,why I prefer it for “core”components.

http://en.wikipedia.org/wiki/BSD_License

I read it. I found that BSD license is GPL compatible! Maybe I miss something?

It means that BSD code can be used in GPL-project. Because BSD code can be used in ANY project.
Unfortunately,it doesn't work other way round. You can't use (L)GPL components in BSD project,because (L)GPL is a viral license

Thesis paper is usually less formal and more user friendly rather than a typical research paper. It assumes reader has only some basic-to-average knowledge of the area. It also contains some very well illustrated and explained subsystems and example applications.

I think PHP is not an ideal thing for a process that:
- holds ONE connection with the webserver
- forks off it's own children and watches them in an intelligent way,so that their number matches the current load for instance
- bridge communication between every child and webserver connection (IMHO that's the worst part for doing in php)

What would be cool is a general SCGI multiplexer (ideally written in C or derivatives) that listens on a scgi port,forks off child processes and forwards all comunication between children and web server. Actually as far as I see it,this program does not have to know anything about SCGI protocol. To use your php app server without modification,the multiplexer could simply let every child start your runner.php listening on a different TCP port,immediately connect to this port and blindly relay communication with web server. Anyone knows of some general software capable of acomplishing such a thing? This is not classical tcp server,because tcp is used instead of IPC

BTW,just to be sure,do I see it right,that your php-app solution as it is –is only capable of processing all requests in a sequential order within one process,that is runner.php?

You are right. Current app-server components work sequentially.

Regarding multiplexer:Actually,I think,that such multiplexer should be the part of the server. And,I believe,lighttpd can do this (it is possible to specify pool of scgi-daemons,instead of one daemon). Anyway,I didn't try it,but that sounds like a proper way of doing things.

If not.. Well,it should be trivial to write one using your description

Hey,thanks for fast reply!

You people have to stop trying to do everything with php. Learn other languages and use them for such cases,there are better solutions than this. It's not my intention to flame,I'm just making a point:In theory I could wipe my ass with a piece of sandpaper,but that's not what it was made for.

You are underestimating both me and php
I know quite a lot of programming languages. Enough to stop thinking about learning new language as about the achievement.
And php really can be used for much more than simple web-scripting. It made a long way since php3-days and is suitable for this kind of tasks

[...] which is similar Ruby’s Rack or Python’s WSGI this is not it. But AiP (formerly “AppServer in PHP“) is:it lets your application pre-initialize classes (and keep them in memory between [...]

Why you so angry about php? It really gives much more opportunities than other languages. For example many web forms are done in php.