Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

netlink receive: no such file or directory on Android 12 #347

Closed
kittyzero520 opened this issue Apr 15, 2023 · 27 comments · Fixed by #358
Closed

netlink receive: no such file or directory on Android 12 #347

kittyzero520 opened this issue Apr 15, 2023 · 27 comments · Fixed by #358
Assignees
Labels
bug Something isn't working enhancement New feature or request help wanted Extra attention is needed

Comments

@kittyzero520
Copy link

  • OS: [安卓12]
  • Arch: [e.g. arm_aarch64]
  • Kernel Version: [ 5.10.110]
  • Version: [ecapture-v0.5.1-android-aarch64.tar.gz]
    使用ecapture-v0.5.1-android-aarch64.tar.gz在安卓12上运行命令报错,报错信息,如下:

ecapt tls -w save_android.pcapng -i eth0

module run failed, [skip it]. error:couldn't start bootstrap manager error:2 errors occurred:
* error:error:netlink receive: no such file or directory , couldn't add a ", err clsact" qdisc to interface 5, {UID:, EbpfFuncName:egress_cls_func}
* error:error:netlink receive: no such file or directory , couldn't add a ", err clsact" qdisc to interface 5, {UID:, EbpfFuncName:ingress_cls_func}

, probes activation validation failed .
tls_2023/04/14 01:45:40 ECAPTURE :: No runnable modules, Exit(1)

ecapt tls -w save_android.pcapng -i wlan0
module run failed, [skip it]. error:route ip+net: no such network interface
tls_2023/04/14 01:48:57 ECAPTURE :: No runnable modules, Exit(1)

@cfc4n cfc4n added bug Something isn't working help wanted Extra attention is needed labels Apr 15, 2023
@cfc4n cfc4n changed the title 安卓12执行命令失败 netlink receive: no such file or directory on Android 12 Apr 15, 2023
@cfc4n
Copy link
Member

cfc4n commented Apr 15, 2023

similar #331

@cfc4n cfc4n self-assigned this Apr 15, 2023
@cfc4n
Copy link
Member

cfc4n commented Apr 16, 2023

#331 里也提到了这个报错,按照他的环境,我无法重现。

你可以自己先多测试测试,尝试给出其他更多环境不同的信息吗?

发一下 tc qdisc add dev eth0 clsact的结果


In #331, this error was also mentioned. According to their environment, I am unable to reproduce it.

Can you please do more testing yourself and try to provide additional information about different environments?

upload result please ,shell : tc qdisc add dev eth0 clsact

bin/ecapture tls -i eth0 -w a.pcapng
tls_2023/04/16 03:59:22 ECAPTURE :: ecapture Version : linux_x86_64:0.5.1-20230415-fffcd0f:[CORE]
tls_2023/04/16 03:59:22 ECAPTURE :: Pid Info : 9095
tls_2023/04/16 03:59:22 ECAPTURE :: Kernel Info : 6.2.8
2023/04/16 03:59:22 read keylogger :/etc/ld.so.conf.d/*.conf error .
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	module initialization
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	Module.Run()
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	TC MODEL
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	OpenSSL/BoringSSL version not found from shared library file, used default version:linux_default_3_0
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	HOOK type:2, binrayPath:/lib/libssl.so.3
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	Ifname:eth0, Ifindex:2,  Port:443, Pcapng filepath:/root/ecapture/a.pcapng
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	Hook masterKey function:SSL_write
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	target all process.
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	target all users.
tls_2023/04/16 03:59:22 EBPFProbeOPENSSL	BPF bytecode filename:user/bytecode/openssl_3_0_0_kern.o
tls_2023/04/16 03:59:24 EBPFProbeOPENSSL	module started successfully.
tls_2023/04/16 03:59:24 EBPFProbeGNUTLS	module initialization
tls_2023/04/16 03:59:24 EBPFProbeGNUTLS	Module.Run()
tls_2023/04/16 03:59:24 EBPFProbeGNUTLS	BPF bytecode filename:user/bytecode/gnutls_kern.o
tls_2023/04/16 03:59:24 EBPFProbeGNUTLS	HOOK type:2, binrayPath:/usr/lib/libgnutls.so.30
tls_2023/04/16 03:59:24 EBPFProbeGNUTLS	target all process.
tls_2023/04/16 03:59:25 EBPFProbeGNUTLS	module started successfully.
tls_2023/04/16 03:59:25 EBPFProbeNSPR	module initialization failed. [skip it]. error:stat /usr/lib/libnspr4.so: no such file or directory
tls_2023/04/16 03:59:25 ECAPTURE :: 	cant found module EBPFProbeGoTLS config info.
tls_2023/04/16 03:59:25 ECAPTURE :: 	start 2 modules

@kittyzero520
Copy link
Author

blueline:/ # tc qdisc add dev eth0 clsact
RTNETLINK answers: No such file or directory

@SeeFlowerX
Copy link

歪个楼,请问你的blueline是怎么用上5.10的内核的,可以展开一下吗 😃

@kittyzero520
Copy link
Author

17万刀片服务器自带的

@SeeFlowerX
Copy link

了解了,那应该是 redroid 或者 cuttlefish 之类的技术吧?

@cfc4n
Copy link
Member

cfc4n commented Apr 17, 2023

blueline:/ # tc qdisc add dev eth0 clsact RTNETLINK answers: No such file or directory

应该是你的内核不支持network emulation。 内核编译需要启用相关配置,你可以参考如下链接。


It should be that your kernel does not support network emulation. Enabling relevant configurations is required during kernel compilation, and you can refer to the following link.

https://itecnote.com/tecnote/linux-rtnetlink-answers-no-such-file-or-directory-error/
https://cateee.net/lkddb/web-lkddb/NET_SCH_NETEM.html
https://itecnote.com/tecnote/linux-rtnetlink-answers-no-such-file-or-directory-error/

@cfc4n cfc4n added wontfix This will not be worked on and removed bug Something isn't working labels Apr 17, 2023
@kittyzero520
Copy link
Author

重新编译内容后,发现抓到的pcap中没什么请求内容
68571681978326_ pic
Uploading 68541681977066_.pic_hd.jpg…

@kittyzero520
Copy link
Author

抓包文件
save_pcap.pcapng.zip

@cfc4n
Copy link
Member

cfc4n commented Apr 21, 2023

你使用的启动命令、测试shell分别是什么? 我觉得大概是你监听的网卡不对,不是流量经过的网卡


What are the startup commands and test shells you are using? I think it's probably because you're monitoring the wrong network interface, not the one that the traffic is passing through.

@kittyzero520
Copy link
Author

我想把抓包的数据写到数据库,将记录保存下来

@kittyzero520
Copy link
Author

应该是eth0

130|blueline:/data/local/tmp # ifconfig
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:13 errors:0 dropped:0 overruns:0 frame:0
TX packets:13 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:600 TX bytes:600

eth0 Link encap:Ethernet HWaddr 48:ad:08:45:1c:01
inet addr:192.168.127.152 Bcast:192.168.127.255 Mask:255.255.255.0
inet6 addr: fe80::1b60:bd0c:a877:bcaf/64 Scope: Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:92078896 errors:0 dropped:1213 overruns:0 frame:0
TX packets:54925595 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:157061853928 TX bytes:4915228285

@kittyzero520
Copy link
Author

使用命令 tc qdisc add dev eth0 clsact

@cfc4n
Copy link
Member

cfc4n commented Apr 23, 2023

使用命令 tc qdisc add dev eth0 clsact

嗯? 这条命令有什么问题吗?

你这个问题的错误原因,在 #347 (comment) 描述的很详细了。

@kittyzero520
Copy link
Author

执行命令不反馈空记录了,到记录无法写入pcap

@kittyzero520
Copy link
Author

_2023/04/23 06:51:44 EBPFProbeOPENSSL saving pcapng file /data/local/tmp/test.pcapng
tls_2023/04/23 06:51:44 EBPFProbeOPENSSL save pcanNP failed, error:Can't send statistics for non existent interface 8; have only 3 interfaces.
tls_2023/04/23 06:51:44 EBPFProbeOPENSSL save 1 packets into pcapng file.
tls_2023/04/23 06:51:44 EBPFProbeOPENSSL close.
tls_2023/04/23 06:51:44 EBPFProbeOPENSSL close

@kittyzero520
Copy link
Author

文件都1kb大小

@cfc4n
Copy link
Member

cfc4n commented Apr 23, 2023

_2023/04/23 06:51:44 EBPFProbeOPENSSL saving pcapng file /data/local/tmp/test.pcapng

tls_2023/04/23 06:51:44 EBPFProbeOPENSSL save pcanNP failed, error:Can't send statistics for non existent interface 8; have only 3 interfaces.

tls_2023/04/23 06:51:44 EBPFProbeOPENSSL save 1 packets into pcapng file.

tls_2023/04/23 06:51:44 EBPFProbeOPENSSL close.

tls_2023/04/23 06:51:44 EBPFProbeOPENSSL close

完整的命令行发出来,别总发不全的信息。沟通成本很高

@kittyzero520
Copy link
Author

blueline:/data/local/tmp # ./ecapt tls -i eth0 -w test.pcapng
tls_2023/04/23 14:37:54 ECAPTURE :: ecapture Version : androidgki_aarch64:0.5.1-20230408-e1afbb8:[CORE]
tls_2023/04/23 14:37:54 ECAPTURE :: Pid Info : 24515
tls_2023/04/23 14:37:54 ECAPTURE :: Kernel Info : 5.10.110
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL module initialization
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL Module.Run()
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL TC MODEL
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL OpenSSL/BoringSSL version not found, used default version :android_default
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL HOOK type:2, binrayPath:/apex/com.android.conscrypt/lib64/libssl.so
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL Ifname:eth0, Ifindex:8, Port:443, Pcapng filepath:/data/local/tmp/test.pcapng
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL Hook masterKey function:SSL_in_init
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL target all process.
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL target all users.
tls_2023/04/23 14:37:54 EBPFProbeOPENSSL BPF bytecode filename:user/bytecode/boringssl_1_1_1_kern.o
tls_2023/04/23 14:37:56 EBPFProbeOPENSSL module started successfully.
tls_2023/04/23 14:37:56 ECAPTURE :: start 1 modules
tls_2023/04/23 14:38:11 TLS1_3_VERSION: save CLIENT_RANDOM c819e15dbd9b64b9583a643c9fee242d03219e97eadaf18d29f6ee8525aa1448 to file success, 778 bytes
tls_2023/04/23 14:39:00 TLS1_3_VERSION: save CLIENT_RANDOM 3c9fe3beda23815ce1e7d85002324f17dacff3de1ec92d591f4cc14aae836e64 to file success, 778 bytes

^Ctls_2023/04/23 14:39:16 EBPFProbeOPENSSL saving pcapng file /data/local/tmp/test.pcapng
tls_2023/04/23 14:39:16 EBPFProbeOPENSSL save pcanNP failed, error:Can't send statistics for non existent interface 8; have only 3 interfaces.
tls_2023/04/23 14:39:16 EBPFProbeOPENSSL save 1 packets into pcapng file.
tls_2023/04/23 14:39:16 EBPFProbeOPENSSL close.
tls_2023/04/23 14:39:17 EBPFProbeOPENSSL

@cfc4n
Copy link
Member

cfc4n commented Apr 23, 2023

Can't send statistics for non existent interface 8; have only 3 interfaces.

这里抱错了,我第一次遇到,租需要debug一下。 你的运行环境可以给一下吗? 详细信息。

@kittyzero520
Copy link
Author

可以的,给个邮箱我发你

@cfc4n
Copy link
Member

cfc4n commented Apr 23, 2023

版本信息贴在这呗,不用发邮箱

@kittyzero520
Copy link
Author

OS: [安卓12]
Arch: [e.g. arm_aarch64]
Kernel Version: [ 5.10.110]
Version: [ecapture-v0.5.1-android-aarch64.tar.gz]
硬件设备:刀片服务器

这是环境信息,我的意思你要不远程设备看下

@cfc4n
Copy link
Member

cfc4n commented Apr 24, 2023

嗯,这硬件配置,我确实准备不来。 你在我微信公众号里留言吧,我到时加你。

@kittyzero520
Copy link
Author

感谢大神的支持

@kittyzero520
Copy link
Author

已关注公众号,todesk远程信息已发

@cfc4n
Copy link
Member

cfc4n commented Apr 28, 2023

blueline:/data/local/tmp # ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
8: eth0@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
    link/ether 48:ad:08:45:1c:01 brd ff:ff:ff:ff:ff:ff link-netnsid 0

eth0网卡ID是8,实际上一共就2个网卡。 在gopacket包里,判断不对,报错了。

// pcapgo/ngwrite.go
func (w *NgWriter) WritePacket(ci gopacket.CaptureInfo, data []byte) error {
	if ci.InterfaceIndex >= int(w.intf) || ci.InterfaceIndex < 0 {
		return fmt.Errorf("Can't send statistics for non existent interface %d; have only %d interfaces", ci.InterfaceIndex, w.intf)
	}
// ...

@cfc4n cfc4n added bug Something isn't working enhancement New feature or request and removed wontfix This will not be worked on labels Apr 28, 2023
cfc4n added a commit that referenced this issue Apr 28, 2023
cfc4n added a commit that referenced this issue Apr 30, 2023
@cfc4n cfc4n pinned this issue Nov 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working enhancement New feature or request help wanted Extra attention is needed
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants