Clever Geek Handbook

Performance of Read () and Write () to / from Linux SKB

Based on a standard Linux system with a user application and a kernel networking stack. I've read that moving frames from user space to kernel space (and vica-versa) can be costly in terms of CPU cycles.

My questions:

  • Why? and moves the frame in one direction (i.e. from user to kernel) have a stronger impact.
  • Also how are things when you go to TAP based interfaces. Since the border will still be between user / kernel space. Is space being used, or is there some form of zero-copy in the game?
+3


source to share


2 answers


Addressing questions in line:

Why? and moves the frame in one direction (that is, from the user to the kernel) have a stronger impact.

Moving to / from user / kernel space is expensive because the OS must:

  • Confirm the pointers for the copy operation.
  • Transmission of actual data.
  • Include the usual costs associated with switching between user / kernel mode.


There are some exceptions to this, for example if your driver implements a strategy such as "page flipping" that effectively reassigns a chunk / page of memory so that it is available to a user-space application. This is "close enough" to a zero-copy operation.

In terms of performance copy_to_user

/ copy_from_user

, the performance of the two functions seems to be comparable.

Also how things change when you move to TAP based interfaces. In view, the frame will still move between user / kernel space. Do the problem space, or is there some form of zero copy in the game?

With TUN / TAP based interfaces, the same considerations apply unless you are using any kind of DMA, page flipping, etc .; logics.

+4


source


Context Switch

Moving frames from user space to kernel space is called a context switch, which is usually triggered by a system call (which causes an interrupt int 0x80

).

  • The interruption occurs by introducing kernel space;
  • When an interrupt occurs, os retain all the register values in the kernel thread stack: ds

    , es

    , fs

    , eax

    , cr3

    etc.
  • It then goes to the IRQ handler like a function call;
  • After some common path of IRQ execution, it will choose the next thread to run according to some algorithm;
  • Runtime information (all registers) is loaded from the next thread;
  • Return to user space;

As we can see, we do a lot of work when you move the frame to / from the kernel that much more work than a simple function call (only installation ebp

, esp

, eip

). This is why this behavior takes a relatively long time.

Virtual devices

As virtual network devices, recording to TAP is no different from recording to /dev/xxx

.



If you write in TAP, os will be interrupted like the above description, then it will copy your arguments to the kernel and block the current thread (in IO blocking). The kernel driver thread will be notified in some way (like a message queue) to receive the arguments and use it.

There is some null syscall instance in Andorid and in my demo implementations this can be done by converting addresses between user and kernel. Since the kernel and user thread do not share the same address space and user thread data, we can copy the data to the kernel. Therefore, if we satisfy the condition, we can avoid copying:

  • this system call must be blocked, i.e. data will not be changed;
  • translate between addresses on page tables, that is, the kernel can refer to the correct data;

Code

Below are the codes from my demo related to this question if you are interested in the details:

+3


source






More articles:


All Articles