Isn't it crazy not to rely on a caching system like memcached currently (for dynamic sites)?
I was just looking at one of my client applications that uses the old legacy php framework that doesn't rely on caching at all and is pretty much completely database dependent.
I suppose I just rewrote it from scratch because it is really outdated and in this rewrite I want to implement a caching system. It would be nice if I could get multiple pointers if someone has done this before.
- The rewriting will be done in PHP or Python
- It would be nice if I could profile before and after this implementation
- I have my own server so I am not limited to shared hosting
source to share
Caching, when it works correctly (== high attack speed), is one of the few general-purpose techniques that can actually help with latency - the more complex part of the problem is generally described as "performance". You can get better QPS (requests per second) performance metrics simply by throwing more hardware into the problem - but latency doesn't work that way (i.e. it only takes one month to make babies if you install nine mothers him; -).
However, the main resource used by caching is usually memory (RAM or disk, as it may be). As you noted in the comment that the only performance issue you see is memory usage, caching would not help: it would simply allocate some of the memory for caching purposes, leaving it even less available as a "shared fund". As a California resident, I am a first-hand witness of what happens when too many resources are committed and I could not recommend such a course of action with a clear conscience! -)
source to share
If your site's performance is fine, there is no reason to add caching. Many sites can do without cache altogether, or by switching to a filesystem based cache. These are only super high traffic sites that need memcached.
What is "crazy" is the architecture of the code (or lack of architecture) which makes it difficult to add caching to the latter.
source to share
Since Python is one of your options, I would go with Django. Built in caching engine and I used this debug_toolbar to help me with development / profiling.
By the way, memcached does n't work as you described. It maps unique keys to values in memory, it has nothing to do with .csh files or database queries. What you store in the value is what will be cached.
Oh, and caching is only worthwhile if there are (or will be) performance issues. There is nothing wrong with "don't rely" on caches if you don't need them. Premature optimization is 99% evil!
source to share
Depending on the specifics of your code and traffic patterns, you may not even need to rewrite your entire site. The horribly inefficient code isn't that big if it can be cached for 99.9% of page requests.
When choosing PHP or Python, make sure you figure out where you are going to host the site (or if you even get this call). Many of my clients are already configured on a web server and Python is not an option. You should also make sure that any databases / external programs you want to interact with are well supported in PHP or Python.
source to share