Clever Geek Handbook

Bright, safe, efficient and multilingual sandbox

I'm working on building an online judgment system where maybe 100 or so unreliable executables will run concurrently and evaluate the same input.

I would like each executable to be limited to an equal proportion of CPU, memory, disk space, etc. predefined pool of resources. For example, if the resource pool was installed in 3/4 of the processor computer, 3 GB of memory, 300 GB of disk and 2 executables, each of them will receive 3/8 of the processor, 1.5 GB of memory, 150 GB of disk. If another has joined, the resource will be reconfigured into three equal chunks. This is to ensure that a malicious or buggy executable does not steal resources from others, and also provides equal resources to everyone.

Ideally, I would also like the executables not to be tied to one language (for example, so that users develop in what they feel comfortable with - C, C ++, Java, Python, etc.).

Using an entire virtual machine or something like OpenVZ seems like overkill. Are there lighter weight alternatives that essentially use a separate process for each executable, limiting its resources, disabling things like network access, spawning process, etc.? Part of the reason I'm looking for an easy solution is that there is quite a bit of input - you probably won't have to copy it into every executable, but let it read from shared memory.

+3


source to share


1 answer


It might be sufficient to create different user IDs for each process and then restrict them with "ulimit". Manpage from bash ulimit to Debian:

ulimit [-HSTabcdefilmnpqrstuvx [limit]]

Provides control over the resources available to the shell and the processes it starts on systems that allow such control. The -H and -S options indicate whether a hard or soft limit is set for this resource. The hard limit cannot be increased by a non-root user after installing it; the soft limit can be increased up to the hard limit value. If neither -H nor -S is specified, both soft and hard limits are specified. The limit value can be a number in the unit specified for the resource, or one of the special values, hard, soft, or unlimited, which stands for the current hard limit, the current soft limit, and no limit, respectively. If the limit is omitted, the current soft resource limit is printed,if only the -H parameter is given. When more than one resource is specified, the name and the constraint unit are printed to the value. Other options are interpreted as follows:



          -a     All current limits are reported
          -b     The maximum socket buffer size
          -c     The maximum size of core files created
          -d     The maximum size of a process data segment
          -e     The maximum scheduling priority ("nice")
          -f     The maximum size of files written by the shell and its children
          -i     The maximum number of pending signals
          -l     The maximum size that may be locked into memory
          -m     The **maximum resident set size** (many systems do not honor this limit)
          -n     The maximum number of open file descriptors (most systems do not allow this value to be set)
          -p     The pipe size in 512-byte blocks (this may not be set)
          -q     The maximum number of bytes in POSIX message queues
          -r     The maximum real-time scheduling priority
          -s     The maximum stack size
          -t     The maximum amount of cpu time in seconds
          -u     The maximum number of processes available to a single user
          -v     The maximum amount of virtual memory available to the shell and, on some systems, to its children
          -x     The maximum number of file locks
          -T     The **maximum number of threads**

I think limiting threads to 1 allows for a fair distribution of CPU computation time.

Also limit the maximum RSS.

0


source






More articles:


All Articles