Recently I noticed that when I simultaneously do 'up' to many network
(many is ~15) netlink drops part of the messages about interface state change
my userspace tools don't know that some interfaces are in up state now. The
I get from netlink socket is "No buffer space available".
After looking at the code I saw that the only way I can get such error from
is if sk->rmem_allock is bigger than sk->rcvbuf. I can enlarge sk->rcvbuf, but
interface I receive six messages and each of this messages is smaller then 200
the default size of sk->rcvbuf is 65535 bytes, so why messages about 15
fit in default buffer size?
So I looked at rtmsg_ifinfo function. We allocate skb of size NLMSG_GOODSIZE
(NLMSG_GOODSIZE appears to be one page size), fill only ~200 bytes and broadcast
the message to all netlink sockets that should receive it. When we actually
skb to the socket we add skb->truesize (4096 bytes) to sk->rmem_allock and not
size of the actual message (200 byte). So the number of messages that can be in
sk->receive_queue simultaneously is about 16 only!
Now, I understand that we have to use skb->truesize for accounting and not
waste 4000 bytes for each NEWLINK message is to much IMO. I see two solutions
to the problem:
First is to define NLMSG_GOODSIZE to something more reasonable (small) and
second to aggregate
many small messages to one big multipart message and store it in one skb. Only
when the skb is
full we will add another skb to the receive_queue.
I've implemented second approach for netlink_broadcast_deliver just to explain
what I mean (see
attached patch against 2.4.0). The same thing can be done in netlink_unicast
Is there other way to avoid such waste of space in netlink socket's rcvbuf?
Comments are welcom!
Description: Text document