Updating shared libraries without restarting processes
If my process is loading the .so library, and if a new version of the library is available, is it possible to switch to the new library without reloading the process? Or does the answer depend on whether there is a parameter change in one of the existing functions in the library?
I work on a fairly large system that starts 100 processes and every load of 10 libraries. Libraries provide specific functionality and are provided by separate commands. So when one of the changes in the library is (to fix bugs lets say) the ideal thing is to publish it under the hood without affecting the current process. Is it possible?
EDIT Thanks! In my case, when a new library is available, all running processes should start using it. His variant prevents them from working with the old version and building a new one later. So it looks like the safer option is to just restart processes.
source to share
You cannot update a linked library on the fly with a running process. You can even try, but if you succeed (and you do not fail with the "text file" error message), you will have to restart the process for it to map the new library into memory.
You can use the lsof command to check which libraries are linked to (runtime or link time):
lsof -p <process_pid> | grep ' mem '
source to share
One interesting technique, although somewhat prone to failure during the checkpoint recovery phase, is to do an invisible restart.
Your server process, or whatever, saves all the information it needs to files on disk. Including file descriptor numbers and current states. The server process then makes a system call exec
to execute itself, replacing its current version. It then reads its state from files on disk and resumes working with file descriptors as if nothing had happened.
If all goes well, the restart is invisible and the new process uses all the updated libraries.
source to share
At the very least, you should make sure that the library interface does not change between versions. If guaranteed, I would try to dynamically load the libraries with dlopen / dlsym and see if dlclose can reload.
I never did it myself, but this is the path I followed first. If you go like this, can you post the results?
source to share
ldd binary of your process is one way to find out. While theoretically possible, it is not recommended to tinker with the current process, although I'm sure there are utilities like ksplice that tinker with running linux kernels.
you can simply upgrade and the running process will continue with the old version and pick up the new version on restart, assuming your package management system is good and knows what to install.
source to share
You may want to know about shared library versioning and the ld parameter -h
.
One way to use it is as follows:
You maintain a version counter on your build system. You create a shared library with:
ld ..... -h mylibrary.so.$VERSION
however, you put it in your lib tree file as simple mylibrary.so
. (There's also a hack to include all the .so in the .a file).
Now, at runtime, processes using the library look for the fully versioned name. To release a new version, you simply add the new version to the image. Launching programs associated with the old version continues to use it. Since the programs are reconnected and tested against the new one, you run new executables.
source to share
Sometimes you can upgrade to use .so and sometimes you can't. It depends mainly on how you are trying to do it, as well as the security guarantees of the kernel you are running in.
Don't do this: cat new.so> old .so ... because eventually your process might try to claim something on the page and find that it's not in the right place anymore. This is a problem because the addresses of things can change, and it's still the same inode; you are just overwriting the bytes in the file.
However, if you: mv new.so old.so On most systems, you will be fine, because your running processes can hold the now unnamed index for the old library, and new calls to your processes get the new file. BUT, some kernels don't like to let you use in use.so, perhaps out of caution, perhaps for simplicity.
source to share
If you expect libraries to change fairly regularly, and you expect to keep up with the timing, I think your system should be redesigned so that such libraries actually become loosely coupled components (like services).
Having said that, my answer to the question is yes: under certain circumstances, it is possible to update shared libraries without restarting processes. Most of the time I expect this is not possible, such as when your library API changes, when your data segments change, when the library supports internal threads. The list is quite long.
For very small bug fixes to the code, you can still use ptrace to write to the process's memory space and from there redo that /lib/ld-linux.so in terms of dynamic linking. To be honest, this is an extremely difficult activity.
source to share