Can't load linux64.ko module

[Steve Kargl]

Booting a kernel from
% uname -a
FreeBSD sleepdirt 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r331370: \
Thu Mar 22 13:41:30 AKDT 2018 \
***@sleepdirt:/usr/obj/usr/src/amd64.amd64/sys/SLEEPDIRT amd64

gives the following from dmesg

% dmesg | grep linux
link_elf_obj: symbol elf64_linux_vdso_fixup undefined
linker_load_file: /boot/kernel/linux64.ko - unsupported file type

Mine kernel config file contains

options COMPAT_LINUX32
options COMPAT_LINUXKPI
options LINPROCFS

Is something else missing? A glimpse at /sys/conf/NOTES and
/sys/amd64/conf/GENERIC do not reveal anything obvious.

[Ed Maste]

Post by Steve Kargl
Booting a kernel from
% uname -a
FreeBSD sleepdirt 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r331370: \
Thu Mar 22 13:41:30 AKDT 2018 \
gives the following from dmesg
% dmesg | grep linux
link_elf_obj: symbol elf64_linux_vdso_fixup undefined
linker_load_file: /boot/kernel/linux64.ko - unsupported file type

Are you loading the linuxulator bits from modules or trying to compile
it into the kernel? Did your case work in the past but break recently?

As a point of reference, my laptop is at r331538+0a541b719b64 (my WIP
branch), and loading linux.ko and linux64.ko is successful.

[Steve Kargl]

Post by Ed Maste

Post by Steve Kargl
Booting a kernel from
% uname -a
FreeBSD sleepdirt 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r331370: \
Thu Mar 22 13:41:30 AKDT 2018 \
gives the following from dmesg
% dmesg | grep linux
link_elf_obj: symbol elf64_linux_vdso_fixup undefined
linker_load_file: /boot/kernel/linux64.ko - unsupported file type

Are you loading the linuxulator bits from modules or trying to compile
it into the kernel? Did your case work in the past but break recently?
As a point of reference, my laptop is at r331538+0a541b719b64 (my WIP
branch), and loading linux.ko and linux64.ko is successful.

kernel and world are from r331370.

kernel config file contains

options COMPAT_LINUX32
options COMPAT_LINUXKPI
options LINPROCFS

When booting, the kernel tries to load the module. A manual
loading of the module results in

% kldload /boot/kernel/linux64.ko
kldload: an error occurred while loading module /boot/kernel/linux64.ko.
Please check dmesg(8) for more details.
sleepdirt:fvwm:root[203] dmesg | tail -2
link_elf_obj: symbol elf64_linux_vdso_fixup undefined
linker_load_file: /boot/kernel/linux64.ko - unsupported file type

Now, that I look at /sys/amd64/conf/NOTES again, I find that
there is a COMPAT_LINUX as well as the COMPAT_LINUX32. I must
have conflated that two options into being the same thing.

I have to update the system for the recent security announcement,
so I update everything and change my kernel config file.

[Steve Kargl]

This post might be inappropriate. Click to display it.

[Steve Kargl]

Post by Steve Kargl
kernel config file contains
options COMPAT_LINUX32
options COMPAT_LINUXKPI
options LINPROCFS
When booting, the kernel tries to load the module. A manual
loading of the module results in
% kldload /boot/kernel/linux64.ko
kldload: an error occurred while loading module /boot/kernel/linux64.ko.
Please check dmesg(8) for more details.
sleepdirt:fvwm:root[203] dmesg | tail -2
link_elf_obj: symbol elf64_linux_vdso_fixup undefined
linker_load_file: /boot/kernel/linux64.ko - unsupported file type
Now, that I look at /sys/amd64/conf/NOTES again, I find that
there is a COMPAT_LINUX as well as the COMPAT_LINUX32. I must
have conflated that two options into being the same thing.

OK, so where is elf64_linux_vdso_fixup suppose to come from?

cd /boot/kernel
foreach i (*.ko)
nm $i | grep linux_vdso_fixup
end

0000000000018f40 t elf32_linux_vdso_fixup
0000000000017cd0 t elf32_linux_vdso_fixup
U elf64_linux_vdso_fixup

nm kernel | grep linux_vdso
ffffffff80f3cb88 d __set_sysinit_set_sym_elf_linux_vdso_init_sys_init
ffffffff80f3e140 d __set_sysuninit_set_sym_elf_linux_vdso_uninit_sys_uninit
ffffffff80a3eae0 T elf32_linux_vdso_fixup
ffffffff80a3ebe0 T elf32_linux_vdso_reloc
ffffffff80a3e9e0 T elf32_linux_vdso_sym_init
ffffffff81180e70 B elf32_linux_vdso_syms
ffffffff80f32ae0 d elf_linux_vdso_init_sys_init
ffffffff80f32af8 d elf_linux_vdso_uninit_sys_uninit
ffffffff80a292d0 t linux_vdso_deinstall
ffffffff80a29210 t linux_vdso_install

[Steve Kargl]

Post by Steve Kargl
OK, so where is elf64_linux_vdso_fixup suppose to come from?

The answer is compat/linux/linux_vdso.c where we find

#if defined(__i386__) || (defined(__amd64__) && defined(COMPAT_LINUX32))
#define __ELF_WORD_SIZE 32
#else
#define __ELF_WORD_SIZE 64
#endif

having COMPAT_LINUX32 in my kernel config file gives me
elf32_linux_vdso_fixup. It seems that one cannot have
a kernel that supports both 32 and 64-bit linux software.

linux(4) states

for an amd64 kernel use:

options COMPAT_LINUX32

Alternatively, to load the ABI as a module at boot time, place the
following line in loader.conf(5):

linux_load="YES"

It turns out that I have 'linux_load=YES" in /etc/loader.conf.
When I boot the kernel built with COMPAT_LINUX32 prevents
the kldload of linux64.ko.

Oh well, learn something new everyday.

[Benjamin Kaduk]

Post by Steve Kargl
The answer is compat/linux/linux_vdso.c where we find
#if defined(__i386__) || (defined(__amd64__) && defined(COMPAT_LINUX32))
#define __ELF_WORD_SIZE 32
#else
#define __ELF_WORD_SIZE 64
#endif
having COMPAT_LINUX32 in my kernel config file gives me
elf32_linux_vdso_fixup. It seems that one cannot have
a kernel that supports both 32 and 64-bit linux software.
linux(4) states
options COMPAT_LINUX32
Alternatively, to load the ABI as a module at boot time, place the
linux_load="YES"
It turns out that I have 'linux_load=YES" in /etc/loader.conf.
When I boot the kernel built with COMPAT_LINUX32 prevents
the kldload of linux64.ko.

Yes, building the linuxulator statically into the kernel forces only
a single ABI to be possible. If dynamic modules are used for all
three relevant modules, then it is possible to simultaneously
support 32- and 64-bit linux code.

This is (perhaps obliquely) documented at
https://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/linuxemu-lbc-install.html
which I added after a similar thread on -current last July.

-Ben

[John Baldwin]

Post by Steve Kargl

Post by Steve Kargl
OK, so where is elf64_linux_vdso_fixup suppose to come from?

The answer is compat/linux/linux_vdso.c where we find
#if defined(__i386__) || (defined(__amd64__) && defined(COMPAT_LINUX32))
#define __ELF_WORD_SIZE 32
#else
#define __ELF_WORD_SIZE 64
#endif
having COMPAT_LINUX32 in my kernel config file gives me
elf32_linux_vdso_fixup. It seems that one cannot have
a kernel that supports both 32 and 64-bit linux software.
linux(4) states
options COMPAT_LINUX32
Alternatively, to load the ABI as a module at boot time, place the
linux_load="YES"
It turns out that I have 'linux_load=YES" in /etc/loader.conf.
When I boot the kernel built with COMPAT_LINUX32 prevents
the kldload of linux64.ko.
Oh well, learn something new everyday.

The Right Way to fix this is probably to have linux_vdso32.c and
linux_vdso64.c that #include linux_vdso.c after setting
ELF_WORD_SIZE similar to how sys/kern/imgact_elf.c works.
Then the COMPAT_LINUX and linux64.ko modules would include
linux_vdso64.c and COMPAT_LINUX32 and linux32.ko modules
(and linux.ko on i386) would include linux_vdso32.c.