网络控制台¶
由 Ingo Molnar mingo@redhat.com 于 2001年9月17日 创建
2.6 内核的网络轮询API移植，由 Matt Mackall <mpm@selenic.com> 于 2003年9月9日添加
IPv6的支持，由 Cong Wang <xiyou.wangcong@gmail.com> 于 2013年1月1日添加
扩展的控制台支持，由 Tejun Heo <tj@kernel.org> 于 2015年5月1日 添加
发布版本前置信息支持，由 Breno Leitao <leitao@debian.org> 于 2023年7月7日 添加
用户数据附加支持，由 Matthew Wood thepacketgeek@gmail.com 于 2024年1月22日 添加
系统数据附加支持，由 Breno Leitao <leitao@debian.org> 于 2025年1月15日 添加
简介：¶
该模块通过UDP记录内核的printk消息，从而允许在磁盘日志记录失败且串行控制台不实用的情况下进行调试。
它既可以编译为内置组件，也可以作为模块使用。作为内置组件时，网络控制台（netconsole）会在网卡初始化后立即启动，并尽快启用指定的网络接口。尽管这无法捕获早期内核崩溃信息，但能记录大部分的启动过程。
发送端与接收端配置：¶
该模块接收一个名为“netconsole”的字符串配置参数，其格式如下：
netconsole=[+][r][src-port]@[src-ip]/[<dev>],[tgt-port]@<tgt-ip>/[tgt-macaddr]
where
+ if present, enable extended console support
r if present, prepend kernel version (release) to the message
src-port source for UDP packets (defaults to 6665)
src-ip source IP to use (interface address)
dev network interface name (eth0) or MAC address
tgt-port port for logging agent (6666)
tgt-ip IP address for logging agent
tgt-macaddr ethernet MAC address for logging agent (broadcast)
示例：
linux netconsole=4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc
或者：
insmod netconsole netconsole=@/,@10.0.0.2/
或者使用 IPv6：
insmod netconsole netconsole=@/,@fd00:1:2:3::1/
或通过 MAC 地址指定出口网络接口，格式如下：
linux netconsole=4444@10.0.0.1/22:33:44:55:66:77,9353@10.0.0.2/12:34:56:78:9a:bc
该功能同样支持向多个远程代理进行日志记录，只需为各个代理指定参数，参数之间用分号分隔，并将整个参数字符串用引号包裹，格式如下：
modprobe netconsole netconsole="@/,@10.0.0.2/;@/eth1,6892@10.0.0.3/"
内置的网络控制台（netconsole）在 TCP 协议栈初始化完成后立即启动，并尝试在指定的地址上启用提供的网络设备。
远程主机可通过多种方式接收这些内核消息，例如：
syslogd
netcat
在使用基于BSD的netcat版本（如Fedora、openSUSE和Ubuntu）的发行版上，监听端口必须在不使用-p参数的情况下指定，格式如下：
nc -u -l -p <port>' / 'nc -u -l <port>
或者：
netcat -u -l -p <port>' / 'netcat -u -l <port>
socat
socat udp-recv:<port> -
动态重配置：¶
动态重配置是网络控制台（netconsole）的一项实用增强功能，允许在运行时通过基于 configfs 的用户空间接口，动态添加或移除远程日志记录目标，或重新配置其参数。
要启用此功能，请在构建网络控制台（netconsole）模块（若网络控制台（netconsole）为内置，则指构建内核）时，选择 CONFIG_NETCONSOLE_DYNAMIC 配置选项。
以下是一些示例（假设 configfs 已挂载于 /sys/kernel/config 挂载点）。
添加远程日志记录目标（目标名称可任意指定）：
cd /sys/kernel/config/netconsole/
mkdir target1
请注意，新创建的目标使用默认参数值（如上所述），且默认处于禁用状态——必须按照下文说明，向“enabled”属性写入“1”才能启用它（通常在相应设置参数后）。
移除目标：
rmdir /sys/kernel/config/netconsole/othertarget/
该接口向用户空间暴露了网络控制台（netconsole）目标的以下参数：
enabled
当前是否启用此目标？
（读写）
extended
扩展模块已启用
（读写）
release
在消息前附加内核版本
（读写）
dev_name
本地网络接口名称
（读写）
local_port
要使用的源UDP端口
（读写）
remote_port
远程代理的UDP端口
（读写）
local_ip
要使用的源IP地址
（读写）
remote_ip
远程代理的IP地址
（读写）
local_mac
本地接口的MAC地址
（只读）
remote_mac
远程代理的MAC地址
（读写）
transmit_errors
数据包发送错误数
（只读）
“enabled” 属性也用于控制是否可以更新目标的参数——您只能修改处于禁用状态的目标的参数（即当“enabled”为0时）。
更新目标参数的方法如下：
cat enabled # check if enabled is 1
echo 0 > enabled # disable the target (if required)
echo eth2 > dev_name # set local interface
echo 10.0.0.4 > remote_ip # update some parameter
echo cb:a9:87:65:43:21 > remote_mac # update more parameters
echo 1 > enabled # enable target again
您也可以动态更新本地接口。这在您希望使用新上线的接口时特别有用（这些接口可能在网络控制台（netconsole）加载/初始化时尚未存在）。
在启动时（或模块加载时）通过 netconsole= 参数定义的目标，会被分配名称 cmdline<index>。例如，参数中的第一个目标被命名为 cmdline0。您可以通过创建具有匹配名称的configfs目录来控制和修改这些目标。
假设您在启动时定义了两个网络控制台（netconsole）目标：
netconsole=4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc;4444@10.0.0.1/eth1,9353@10.0.0.3/12:34:56:78:9a:bc
您可以在运行时通过创建以下目标来修改这些设置：
mkdir cmdline0
cat cmdline0/remote_ip
10.0.0.2
mkdir cmdline1
cat cmdline1/remote_ip
10.0.0.3
附加用户数据¶
启用网络控制台（netconsole）的动态配置后，可将自定义用户数据附加到消息末尾。修改用户数据条目时，无需更改目标的“enabled”属性。
userdata 下的目录（键名）长度限制为 53 个字符，userdata/<键名>/value 中的数据限制为 200 字节：
cd /sys/kernel/config/netconsole && mkdir cmdline0
cd cmdline0
mkdir userdata/foo
echo bar > userdata/foo/value
mkdir userdata/qux
echo baz > userdata/qux/value
Messages will now include this additional user data:
echo "This is a message" > /dev/kmsg
Sends:
12,607,22085407756,-;This is a message
foo=bar
qux=baz
Preview the userdata that will be appended with:
cd /sys/kernel/config/netconsole/cmdline0/userdata
for f in `ls userdata`; do echo $f=$(cat userdata/$f/value); done
If a userdata entry is created but no data is written to the value file, the entry will be omitted from netconsole messages:
cd /sys/kernel/config/netconsole && mkdir cmdline0
cd cmdline0
mkdir userdata/foo
echo bar > userdata/foo/value
mkdir userdata/qux
The qux key is omitted since it has no value:
echo "This is a message" > /dev/kmsg
12,607,22085407756,-;This is a message
foo=bar
Delete userdata entries with rmdir:
rmdir /sys/kernel/config/netconsole/cmdline0/userdata/qux
警告
When writing strings to user data values, input is broken up per line in configfs store calls and this can cause confusing behavior:
mkdir userdata/testing
printf "val1\nval2" > userdata/testing/value
# userdata store value is called twice, first with "val1\n" then "val2"
# so "val2" is stored, being the last value stored
cat userdata/testing/value
val2
It is recommended to not write user data values with newlines.
Task name auto population in userdata¶
Inside the netconsole configfs hierarchy, there is a file called taskname_enabled under the userdata directory. This file is used to enable or disable the automatic task name population feature. This feature automatically populates the current task name that is scheduled in the CPU sneding the message.
To enable task name auto-population:
echo 1 > /sys/kernel/config/netconsole/target1/userdata/taskname_enabled
When this option is enabled, the netconsole messages will include an additional line in the userdata field with the format taskname=<task name>. This allows the receiver of the netconsole messages to easily find which application was currently scheduled when that message was generated, providing extra context for kernel messages and helping to categorize them.
Example:
echo "This is a message" > /dev/kmsg
12,607,22085407756,-;This is a message
taskname=echo
In this example, the message was generated while "echo" was the current scheduled process.
Kernel release auto population in userdata¶
Within the netconsole configfs hierarchy, there is a file named release_enabled located in the userdata directory. This file controls the kernel release (version) auto-population feature, which appends the kernel release information to userdata dictionary in every message sent.
To enable the release auto-population:
echo 1 > /sys/kernel/config/netconsole/target1/userdata/release_enabled
Example:
echo "This is a message" > /dev/kmsg
12,607,22085407756,-;This is a message
release=6.14.0-rc6-01219-g3c027fbd941d
备注
This feature provides the same data as the "release prepend" feature. However, in this case, the release information is appended to the userdata dictionary rather than being included in the message header.
CPU number auto population in userdata¶
Inside the netconsole configfs hierarchy, there is a file called cpu_nr under the userdata directory. This file is used to enable or disable the automatic CPU number population feature. This feature automatically populates the CPU number that is sending the message.
To enable the CPU number auto-population:
echo 1 > /sys/kernel/config/netconsole/target1/userdata/cpu_nr
When this option is enabled, the netconsole messages will include an additional line in the userdata field with the format cpu=<cpu_number>. This allows the receiver of the netconsole messages to easily differentiate and demultiplex messages originating from different CPUs, which is particularly useful when dealing with parallel log output.
Example:
echo "This is a message" > /dev/kmsg
12,607,22085407756,-;This is a message
cpu=42
In this example, the message was sent by CPU 42.
备注
If the user has set a conflicting cpu key in the userdata dictionary, both keys will be reported, with the kernel-populated entry appearing after the user one. For example:
# User-defined CPU entry
mkdir -p /sys/kernel/config/netconsole/target1/userdata/cpu
echo "1" > /sys/kernel/config/netconsole/target1/userdata/cpu/value
Output might look like:
12,607,22085407756,-;This is a message
cpu=1
cpu=42 # kernel-populated value
Message ID auto population in userdata¶
Within the netconsole configfs hierarchy, there is a file named msgid_enabled located in the userdata directory. This file controls the message ID auto-population feature, which assigns a numeric id to each message sent to a given target and appends the ID to userdata dictionary in every message sent.
The message ID is generated using a per-target 32 bit counter that is incremented for every message sent to the target. Note that this counter will eventually wrap around after reaching uint32_t max value, so the message ID is not globally unique over time. However, it can still be used by the target to detect if messages were dropped before reaching the target by identifying gaps in the sequence of IDs.
It is important to distinguish message IDs from the message <sequnum> field. Some kernel messages may never reach netconsole (for example, due to printk rate limiting). Thus, a gap in <sequnum> cannot be solely relied upon to indicate that a message was dropped during transmission, as it may never have been sent via netconsole. The message ID, on the other hand, is only assigned to messages that are actually transmitted via netconsole.
Example:
echo "This is message #1" > /dev/kmsg
echo "This is message #2" > /dev/kmsg
13,434,54928466,-;This is message #1
msgid=1
13,435,54934019,-;This is message #2
msgid=2
Extended console:¶
If '+' is prefixed to the configuration line or "extended" config file is set to 1, extended console support is enabled. An example boot param follows:
linux netconsole=+4444@10.0.0.1/eth1,9353@10.0.0.2/12:34:56:78:9a:bc
Log messages are transmitted with extended metadata header in the following format which is the same as /dev/kmsg:
<level>,<sequnum>,<timestamp>,<contflag>;<message text>
If 'r' (release) feature is enabled, the kernel release version is prepended to the start of the message. Example:
6.4.0,6,444,501151268,-;netconsole: network logging started
Non printable characters in <message text> are escaped using "xff" notation. If the message contains optional dictionary, verbatim newline is used as the delimiter.
If a message doesn't fit in certain number of bytes (currently 1000), the message is split into multiple fragments by netconsole. These fragments are transmitted with "ncfrag" header field added:
ncfrag=<byte-offset>/<total-bytes>
For example, assuming a lot smaller chunk size, a message "the first chunk, the 2nd chunk." may be split as follows:
6,416,1758426,-,ncfrag=0/31;the first chunk,
6,416,1758426,-,ncfrag=16/31; the 2nd chunk.
Miscellaneous notes:¶
警告
the default target ethernet setting uses the broadcast ethernet address to send packets, which can cause increased load on other systems on the same ethernet segment.
小技巧
some LAN switches may be configured to suppress ethernet broadcasts so it is advised to explicitly specify the remote agents' MAC addresses from the config parameters passed to netconsole.
小技巧
to find out the MAC address of, say, 10.0.0.2, you may try using:
ping -c 1 10.0.0.2 ; /sbin/arp -n | grep 10.0.0.2
小技巧
in case the remote logging agent is on a separate LAN subnet than the sender, it is suggested to try specifying the MAC address of the default gateway (you may use /sbin/route -n to find it out) as the remote MAC address instead.
备注
the network device (eth1 in the above case) can run any kind of other network traffic, netconsole is not intrusive. Netconsole might cause slight delays in other traffic if the volume of kernel messages is high, but should have no other impact.
备注
if you find that the remote logging agent is not receiving or printing all messages from the sender, it is likely that you have set the "console_loglevel" parameter (on the sender) to only send high priority messages to the console. You can change this at runtime using:
dmesg -n 8
or by specifying "debug" on the kernel command line at boot, to send all kernel messages to the console. A specific value for this parameter can also be set using the "loglevel" kernel boot option. See the dmesg(8) man page and The kernel's command-line parameters for details.
Netconsole was designed to be as instantaneous as possible, to enable the logging of even the most critical kernel bugs. It works from IRQ contexts as well, and does not enable interrupts while sending packets. Due to these unique needs, configuration cannot be more automatic, and some fundamental limitations will remain: only IP networks, UDP packets and ethernet devices are supported.