Help needed to configure Lighttpd and Apache with Django
I'm using Django as my web framework and then Apache and Lighttpd as my web server and static media server respectively. Lighty supports all of my static content nice and well, but I need to tweak it to serve new files uploaded by the user. Lighttpd runs on a different Apache machine (Django). My django code for creating a directory and then creating an image file is executed on my Apache machine, making it currently persistent on the same machine. I want this directory and file creation to happen on my static media server, which then has to be served by the media server itself. I am using os.mkdir and urllib.urlretrieve functions respectively to create a directory and save files on a Django machine (Apache). Is there anyway,that I can tweak some settings to make these things work, or do I have to write scripts on the media server and call them from the Django machine?
source to share
The simplest answer is that the user uploads to a shared directory that both web servers can access. Then it is available instantly. If you are using unix (sounds like this) then NFS is a possible solution. If you think your site will scale across multiple servers, and la flickr, then using rsync to go to multiple edge servers and maybe even using a sharding scheme is another solution.
Just be careful. There are many security issues that you should consider depending on your application.
If all files go to the public directory, users will be able to guess other peoples file names and download them. In this case, you will need to serve them from Django with a thin layer of security on top.
Never trust your users! Make sure they are loaded into a specific valid set. Under no circumstances should you let them download whatever you want. Unless, of course, your users trust little. Even then, you have to do some checks. They probably shouldn't be uploading .php files for one. The last thing you want to give them is the ability to run arbitrary scripts on your server. At the very least, set up the directory to just serve files and do nothing.
Good luck.
source to share
This is the stuff I use for rsync. Do whatever you want on the main server and then periodically (or on demand) rsync push to the static server. Rsync is faster (and more functional) than anything you can do with a quick hack.
Just because I'm paranoid, I do an hourly rsync of all my client sites to 2 backup servers, one of which is in my garage. I just timed "rsync -a" to a custom 1.7GB site (which had no changes) and it took 9.92 seconds of wall clock including 3 network handshakes to sync three different directories. If anything changed, presto bango, it was done, complete with timestamps, owner / group, etc.
It's amazing how blasé you can become about a server crash when you come back working with multiple machines not working on people. I sleep very well.
source to share