编译链接共享库路径问题

今天在编译可执行文件时，报错

$ mips-unknown-linux-gnu-gcc userspace/private/phyreg/phyreg_user.c  -I output/staging/include/ -Loutput/staging/lib  -lphyreg
。。。ld: warning: libnl-3.so.200, needed by output/staging/lib/libphyreg.so, not found (try using -rpath or -rpath-link)
。。。ld: warning: libnl-genl-3.so.200, needed by output/staging/lib/libphyreg.so, not found (try using -rpath or -rpath-link)
output/staging/lib/libphyreg.so: undefined reference to `genl_ctrl_alloc_cache@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `genl_connect@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nl_cache_free@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nl_socket_alloc@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nl_send_auto_complete@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nl_cb_alloc@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `genlmsg_put@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `genlmsg_valid_hdr@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nl_recvmsgs@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nl_cb_put@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nl_socket_free@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `genlmsg_attrdata@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `genl_family_get_id@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nla_next@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nla_type@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `genl_family_put@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nla_len@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `genlmsg_hdr@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nla_data@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nlmsg_hdr@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nla_put@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `genlmsg_attrlen@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nl_cb_set@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nlmsg_alloc_size@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `nla_ok@libnl_3'
output/staging/lib/libphyreg.so: undefined reference to `genl_ctrl_search_by_name@libnl_3'
collect2: error: ld returned 1 exit status

依赖是这样的，libphyreg.so依赖与libnetlink的几个库，然后可执行文件phyreg，依赖于libphyreg。共享库全部放在output/staging/lib里面了。编译时，使用了-Loutput/staging/lib，为啥libphyreg.so可以找到，而它依赖的netlink的库为啥找不到呢。

使用strace来调试发现，ld根本没去-L指定的路径去搜，只搜索了几个默认路径：

$ strace /work/toolchain/mips-unknown-linux-gnu/bin/mips-unknown-linux-gnu-ld a.o -L output/staging/lib/ -lphyreg -lc
open("/work/toolchain/mips-unknown-linux-gnu/bin/../mips-unknown-linux-gnu/sysroot/work/k2t/k2t-mesh/output/toolchain/mips-unknown-linux-gnu/etc/ld.so.conf", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/work/toolchain/mips-unknown-linux-gnu/bin/../mips-unknown-linux-gnu/sysroot/etc/ld.so.conf", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/work/toolchain/mips-unknown-linux-gnu/bin/../mips-unknown-linux-gnu/sysroot/work/k2t/k2t-mesh/output/toolchain/mips-unknown-linux-gnu/mips-unknown-linux-gnu/lib/libnl-3.so.200", O_RDONLY) = -1 ENOENT (No s
uch file or directory)
open("/work/toolchain/mips-unknown-linux-gnu/bin/../mips-unknown-linux-gnu/sysroot/usr/local/lib/libnl-3.so.200", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/work/toolchain/mips-unknown-linux-gnu/bin/../mips-unknown-linux-gnu/sysroot/lib/libnl-3.so.200", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/work/toolchain/mips-unknown-linux-gnu/bin/../mips-unknown-linux-gnu/sysroot/usr/lib/libnl-3.so.200", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/work/toolchain/mips-unknown-linux-gnu/bin/../mips-unknown-linux-gnu/sysroot/work/k2t/k2t-mesh/output/toolchain/mips-unknown-linux-gnu/mips-unknown-linux-gnu/lib/libnl-3.so.200", O_RDONLY) = -1 ENOENT (No s
uch file or directory)
open("/work/toolchain/mips-unknown-linux-gnu/bin/../mips-unknown-linux-gnu/sysroot/usr/local/lib/libnl-3.so.200", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/work/toolchain/mips-unknown-linux-gnu/bin/../mips-unknown-linux-gnu/sysroot/lib/libnl-3.so.200", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/work/toolchain/mips-unknown-linux-gnu/bin/../mips-unknown-linux-gnu/sysroot/usr/lib/libnl-3.so.200", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, "/work/toolchain/mips-unknown-lin"..., 68/work/toolchain/mips-unknown-linux-gnu/bin/mips-unknown-linux-gnu-ld) = 68

如果手动在命令行选项加上：-lnl-3 -lnl-genl-3，则是可以链接ok的。

1	`$ /work/toolchain/mips-unknown-linux-gnu/bin/mips-unknown-linux-gnu-gcc userspace/private/phyreg/phyreg_user.c -I output/staging/include/ -Loutput/staging/lib/ -lnl-3 -lnl-genl-3 -lphyreg`

搜索发现了-rpath和-rpah-link选项。这里使用-rpath-link指定路径就可以了。此时不需要手动加上netlink的-l。

$ /work/toolchain/mips-unknown-linux-gnu/bin/mips-unknown-linux-gnu-gcc userspace/private/phyreg/phyreg_user.c -I output/staging/include/ -Wl,-rpath-link=output/staging/lib -Loutput/staging/lib/ -lphyreg

思考：

phyreg依赖libphyreg.so，那么链接的时候，libphyreg.so再依赖的其它共享库是按照运行时路径来搜索的。libphyreg.so是按照链接路径来搜索的。-L可以指定libphyreg.so的搜索路径。而运行时路径通过 -rpath和-rpath-link来指定，前者将路径写到到生成的可执行文件中，后者不会，只是告诉ld运行时搜索路径。
如果不用-rpath-link，那么我们可以使用-L -l将libphyreg.so依赖的so显示指定进来，这样ld发现这些so已经加载了，就不会再去运行时路径搜索了。
注意-rpath-link不影响链接搜索路径，因此上面去掉-L，将导致libphyreg.so搜索失败。

$ /work/toolchain/mips-unknown-linux-gnu/bin/mips-unknown-linux-gnu-gcc userspace/private/phyreg/phyreg_user.c -I output/staging/include/ -Wl,-rpath-link=output/staging/lib -lphyreg

/work/toolchain/mips-unknown-linux-gnu/bin/../lib/gcc/mips-unknown-linux-gnu/6.3.0/../../../../mips-unknown-linux-gnu/bin/ld: cannot find -lphyreg

collect2: error: ld returned 1 exit status

ILD