VirtualBox

Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#19145 closed defect (fixed)

linux: kernel 5.5-rc1/rc2 - we need changes -> fixed in 6.1.4

Reported by: Frank Batschulat (Oracle) Owned by: Frank Batschulat (Oracle)
Component: other Version: VirtualBox 6.0.14
Keywords: linux kernel 5.5 5.5-rc1 5.5-rc2 Cc:
Guest type: other Host type: other

Description (last modified by Frank Batschulat (Oracle))

Compiling trunk against the current new mainline kernel: 5.5-rc1 2019-12-08 https://git.kernel.org/torvalds/t/linux-5.5-rc1.tar.gz fails with the following errors for which we need to make adjustments:

In file included from /home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/hgsmi_base.c:26:0:
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_drv.h:101:10: fatal error: drm/drmP.h: No such file or directory
 #include <drm/drmP.h>
          ^~~~~~~~~~~~
compilation terminated.
make[2]: *** [/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/hgsmi_base.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_fb.c:40:10: fatal error: drm/drmP.h: No such file or directory
 #include <drm/drmP.h>
          ^~~~~~~~~~~~
compilation terminated.
make[2]: *** [/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_fb.o] Error 1
In file included from /home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vboxvideo_guest.h:33:0,
                 from /home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/modesetting.c:26:
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_drv.h:101:10: fatal error: drm/drmP.h: No such file or directory
 #include <drm/drmP.h>
          ^~~~~~~~~~~~
compilation terminated.
make[2]: *** [/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/modesetting.o] Error 1
kBuild: Generating Java interface files
filesplitter: Out of 323 files: 323 rewritten, 0 unchanged. (/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/vboxjxpcom-gen/jxpcomgen/java/interfaces)
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_drv.c:35:10: fatal error: drm/drmP.h: No such file or directory
 #include <drm/drmP.h>
          ^~~~~~~~~~~~
compilation terminated.
make[2]: *** [/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_drv.o] Error 1
make[1]: *** [/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod] Error 2
make: *** [vboxvideo] Error 2
kmk: *** [/home/ws/vbtrunk/trunk/src/VBox/Additions/linux/Makefile.kmk:354: /home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_site_ws_linux-5.5-rc1_mod.run] Error 2
kmk: *** Waiting for unfinished jobs....

That header file does not exist anymore:

root@lserver:/site/ws/linux-5.5-rc1# pwd
/site/ws/linux-5.5-rc1
root@lserver:/site/ws/linux-5.5-rc1# find . -name drmP.h -ls
root@lserver:/site/ws/linux-5.5-rc1# 

but was in the past:

# find /usr -name drmP.h -ls
   269962      4 -rw-r--r--   1 root     root         3474 Mär  4  2019 /usr/src/linux-headers-5.0.0-27/include/drm/drmP.h
  1933958      4 -rw-r--r--   1 root     root         3474 Mär  4  2019 /usr/src/linux-headers-5.0.0-36/include/drm/drmP.h
  2257654     12 -rw-r--r--   1 root     root        11103 Jan 28  2018 /usr/src/linux-headers-4.15.0-72/include/drm/drmP.h
  2292256      4 -rw-r--r--   1 root     root         3474 Mär  4  2019 /usr/src/linux-headers-5.0.0-37/include/drm/drmP.h

and was still present in 5.4.1 and 5.4.2:

root@lserver:/site/ws/linux-5.4.1# find . -name drmP.h -ls
  3243422      4 -rw-rw-r--   1 root     root         3121 Nov 29 10:10 ./include/drm/drmP.h
# pwd
/site/ws/linux-5.4.2
root@lserver:/site/ws/linux-5.4.2# find . -name drmP.h -ls
  3337858      4 -rw-rw-r--   1 root     root         3121 Dez  4 22:31 ./include/drm/drmP.h

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=71866a56bc7594068d894bacd9ac957878c6816a
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=0005cbda438fa846650ae52ce693eeaff6d16b92

[v3,06/12] drm: remove include of drmP.h from bridge/dw_hdmi.h https://lore.kernel.org/patchwork/patch/1030070/

[v4,4/8] drm/hisilicon/kirin: prepare for drmP.h removal from drm_modeset_helper.h https://lore.kernel.org/patchwork/patch/1031546/

https://lkml.org/lkml/2019/1/26/140

and more, see:

https://www.collabora.com/news-and-blog/news-and-events/linux-kernel-5.4.html

also affected is 5.5.0-rc2 https://git.kernel.org/torvalds/t/linux-5.5-rc2.tar.gz

Attachments (1)

errors-1.txt (45.7 KB ) - added by Frank Batschulat (Oracle) 5 years ago.

Download all attachments as: .zip

Change History (24)

comment:1 by Frank Batschulat (Oracle), 5 years ago

Owner: set to Frank Batschulat (Oracle)
Status: newaccepted

comment:2 by Frank Batschulat (Oracle), 5 years ago

These are going to be our inspection targets:

/vbox/src/VBox/Additions/linux/drm/
H A D    vbox_drv.c    39 #include <drm/drmP.h>
H A D    vbox_drv.h    106 #include <drm/drmP.h>
H A D    vbox_fb.c    45 #include <drm/drmP.h> 

comment:3 by Frank Batschulat (Oracle), 5 years ago

if we disable that header file:

$ svn diff
Index: src/VBox/Additions/linux/drm/vbox_drv.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_drv.c	(revision 135411)
+++ src/VBox/Additions/linux/drm/vbox_drv.c	(working copy)
@@ -36,7 +36,10 @@
 #include <linux/console.h>
 #include <linux/vt_kern.h>
 
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
 #include <drm/drmP.h>
+#endif
+
 #include <drm/drm_crtc_helper.h>
 
 #include "vbox_drv.h"
Index: src/VBox/Additions/linux/drm/vbox_drv.h
===================================================================
--- src/VBox/Additions/linux/drm/vbox_drv.h	(revision 135411)
+++ src/VBox/Additions/linux/drm/vbox_drv.h	(working copy)
@@ -103,7 +103,9 @@
 #define S64_MIN         ((s64)(-S64_MAX - 1))
 #endif
 
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
 #include <drm/drmP.h>
+#endif
 #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 11, 0) || defined(RHEL_75)
 #include <drm/drm_encoder.h>
 #endif
Index: src/VBox/Additions/linux/drm/vbox_fb.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_fb.c	(revision 135411)
+++ src/VBox/Additions/linux/drm/vbox_fb.c	(working copy)
@@ -42,7 +42,9 @@
 #include <linux/fb.h>
 #include <linux/init.h>
 
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
 #include <drm/drmP.h>
+#endif
 #include <drm/drm_crtc.h>
 #include <drm/drm_fb_helper.h>
 #include <drm/drm_crtc_helper.h>

We get a lot of follow up compilation errors we have to fix for the LINUX version macros:

/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_drv.c:35:5: error: "LINUX_VERSION_CODE" is not defined, evaluates to 0 [-Werror=undef]
 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
     ^~~~~~~~~~~~~~~~~~
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_drv.c:35:26: error: "KERNEL_VERSION" is not defined, evaluates to 0 [-Werror=undef]
 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
                          ^~~~~~~~~~~~~~
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_drv.c:35:40: error: missing binary operator before token "("
 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
                                        ^
kBuild: xpidl XPCOM - /home/ws/vbtrunk/trunk/src/libs/xpcom18a4/xpcom/base/nsIConsoleService.idl
kBuild: xpidl XPCOM - /home/ws/vbtrunk/trunk/src/libs/xpcom18a4/xpcom/base/nsIErrorService.idl
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_fb.c:40:5: error: "LINUX_VERSION_CODE" is not defined, evaluates to 0 [-Werror=undef]
 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
     ^~~~~~~~~~~~~~~~~~
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_fb.c:40:26: error: "KERNEL_VERSION" is not defined, evaluates to 0 [-Werror=undef]
 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
                          ^~~~~~~~~~~~~~
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_fb.c:40:40: error: missing binary operator before token "("
 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
                                        ^
 $ svn diff
Index: src/VBox/Additions/linux/drm/vbox_drv.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_drv.c	(revision 135411)
+++ src/VBox/Additions/linux/drm/vbox_drv.c	(working copy)
@@ -36,11 +36,12 @@
 #include <linux/console.h>
 #include <linux/vt_kern.h>
 
+#include "vbox_drv.h"
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
 #include <drm/drmP.h>
+#endif
 #include <drm/drm_crtc_helper.h>
-
-#include "vbox_drv.h"
-
 #if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 1, 0) || defined(RHEL_81)
 #include <drm/drm_probe_helper.h>
 #endif
Index: src/VBox/Additions/linux/drm/vbox_drv.h
===================================================================
--- src/VBox/Additions/linux/drm/vbox_drv.h	(revision 135411)
+++ src/VBox/Additions/linux/drm/vbox_drv.h	(working copy)
@@ -103,7 +103,9 @@
 #define S64_MIN         ((s64)(-S64_MAX - 1))
 #endif
 
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
 #include <drm/drmP.h>
+#endif
 #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 11, 0) || defined(RHEL_75)
 #include <drm/drm_encoder.h>
 #endif
Index: src/VBox/Additions/linux/drm/vbox_fb.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_fb.c	(revision 135411)
+++ src/VBox/Additions/linux/drm/vbox_fb.c	(working copy)
@@ -42,12 +42,15 @@
 #include <linux/fb.h>
 #include <linux/init.h>
 
+#include "vbox_drv.h"
+
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
 #include <drm/drmP.h>
+#endif
 #include <drm/drm_crtc.h>
 #include <drm/drm_fb_helper.h>
 #include <drm/drm_crtc_helper.h>
 
-#include "vbox_drv.h"
 #include <VBoxVideo.h>
 
 #if LINUX_VERSION_CODE < KERNEL_VERSION(4, 7, 0) && !defined(RHEL_74)

And then we get to the real errors we have to fix after disabling the use of drmP.h:

see atatched file errors-1.txt

https://www.virtualbox.org/attachment/ticket/19145/errors-1.txt

Last edited 5 years ago by Frank Batschulat (Oracle) (previous) (diff)

by Frank Batschulat (Oracle), 5 years ago

Attachment: errors-1.txt added

comment:4 by Frank Batschulat (Oracle), 5 years ago

Description: modified (diff)
Keywords: 5.5-rc2 added
Summary: linux: kernel 5.5-rc1 - we need changeslinux: kernel 5.5-rc1/rc2 - we need changes

comment:5 by Szpak, 5 years ago

Trying to CC myself (can it be done in Trac without making a comment?)

comment:6 by Larry998, 5 years ago

You should be able to replace

#include <drm/drmP.h>

with

#include <drm/drm_file.h>
#include <drm/drm_drv.h>
#include <drm/drm_device.h>

comment:7 by Frank Batschulat (Oracle), 5 years ago

So after applying the following diffs we come to the true remaining problems to solve:

$ svn diff
Index: src/VBox/Additions/linux/drm/vbox_drv.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_drv.c	(revision 135493)
+++ src/VBox/Additions/linux/drm/vbox_drv.c	(working copy)
@@ -36,7 +36,6 @@
 #include <linux/console.h>
 #include <linux/vt_kern.h>
 
-#include <drm/drmP.h>
 #include <drm/drm_crtc_helper.h>
 
 #include "vbox_drv.h"
Index: src/VBox/Additions/linux/drm/vbox_drv.h
===================================================================
--- src/VBox/Additions/linux/drm/vbox_drv.h	(revision 135493)
+++ src/VBox/Additions/linux/drm/vbox_drv.h	(working copy)
@@ -48,6 +48,7 @@
 #include <linux/genalloc.h>
 #include <linux/io.h>
 #include <linux/string.h>
+#include <linux/pci.h>
 
 #if defined(RHEL_MAJOR) && defined(RHEL_MINOR)
 # if RHEL_MAJOR == 8 && RHEL_MINOR >= 1
@@ -103,7 +104,17 @@
 #define S64_MIN         ((s64)(-S64_MAX - 1))
 #endif
 
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
 #include <drm/drmP.h>
+#else /* > KERNEL_VERSION(5, 5, 0) */
+#include <drm/drm_file.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_device.h>
+#include <drm/drm_ioctl.h>
+#include <drm/drm_fourcc.h>
+#include <drm/drm_irq.h>
+#include <drm/drm_vblank.h>
+#endif /* < KERNEL_VERSION(5, 5, 0) */
 #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 11, 0) || defined(RHEL_75)
 #include <drm/drm_encoder.h>
 #endif
Index: src/VBox/Additions/linux/drm/vbox_fb.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_fb.c	(revision 135493)
+++ src/VBox/Additions/linux/drm/vbox_fb.c	(working copy)
@@ -42,7 +42,6 @@
 #include <linux/fb.h>
 #include <linux/init.h>
 
-#include <drm/drmP.h>
 #include <drm/drm_crtc.h>
 #include <drm/drm_fb_helper.h>
 #include <drm/drm_crtc_helper.h>

Now this eventually uncovers the real problematic changes we need to fix:

/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_mode.c: In function ‘vbox_best_single_encoder’:
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_mode.c:396:26: error: ‘struct drm_connector’ has no member named ‘encoder_ids’; did you mean ‘encoder’?
  int enc_id = connector->encoder_ids[0];
                          ^~~~~~~~~~~
                          encoder

in 5.4 we had:

‘struct drm_connector’

#define DRM_CONNECTOR_MAX_ENCODER 3
	/**
	 * @encoder_ids: Valid encoders for this connector. Please only use
	 * drm_connector_for_each_possible_encoder() to enumerate these.
	 */
	uint32_t encoder_ids[DRM_CONNECTOR_MAX_ENCODER];

https://elixir.bootlin.com/linux/v5.4/source/include/drm/drm_connector.h#L1613
/**
 * drm_connector_for_each_possible_encoder - iterate connector's possible encoders
 * @connector: &struct drm_connector pointer
 * @encoder: &struct drm_encoder pointer used as cursor
 * @__i: int iteration cursor, for macro-internal use
 */
#define drm_connector_for_each_possible_encoder(connector, encoder, __i) \
	for ((__i) = 0; (__i) < ARRAY_SIZE((connector)->encoder_ids) && \
		     (connector)->encoder_ids[(__i)] != 0; (__i)++) \
		for_each_if((encoder) = \
			    drm_encoder_find((connector)->dev, NULL, \
					     (connector)->encoder_ids[(__i)])) \

#endif

This accessor function changed in 5.5 because the structure and its members changed:

https://elixir.bootlin.com/linux/v5.5-rc2/source/include/drm/drm_connector.h#L1617

/**
 * drm_connector_for_each_possible_encoder - iterate connector's possible encoders
 * @connector: &struct drm_connector pointer
 * @encoder: &struct drm_encoder pointer used as cursor
 */
#define drm_connector_for_each_possible_encoder(connector, encoder) \
	drm_for_each_encoder_mask(encoder, (connector)->dev, \
				  (connector)->possible_encoders)

#endif

and we get this error:

/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_ttm.c: In function ‘vbox_mm_init’:
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_ttm.c:311:6: error: passing argument 4 of ‘ttm_bo_device_init’ makes pointer from integer without a cast [-Werror=int-conversion]
      true);
      ^~~~
In file included from /home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_drv.h:122:0,
                 from /home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_ttm.c:30:
./include/drm/ttm/ttm_bo_driver.h:605:5: note: expected ‘struct drm_vma_offset_manager *’ but argument is of type ‘int’
 int ttm_bo_device_init(struct ttm_bo_device *bdev,
     ^~~~~~~~~~~~~~~~~~
/home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_ttm.c:300:8: error: too few arguments to function ‘ttm_bo_device_init’
  ret = ttm_bo_device_init(&vbox->ttm.bdev,
        ^~~~~~~~~~~~~~~~~~
In file included from /home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_drv.h:122:0,
                 from /home/ws/vbtrunk/trunk/out/linux.amd64/debug/obj/tstvboxvideo-src_mod/vbox_ttm.c:30:
./include/drm/ttm/ttm_bo_driver.h:605:5: note: declared here
 int ttm_bo_device_init(struct ttm_bo_device *bdev,
     ^~~~~~~~~~~~~~~~~~

Again the function arguments changed in 5.5:

https://elixir.bootlin.com/linux/v5.5-rc2/source/include/drm/ttm/ttm_bo_driver.h#L605

int ttm_bo_device_init(struct ttm_bo_device *bdev,
		       struct ttm_bo_driver *driver,
		       struct address_space *mapping,
		       struct drm_vma_offset_manager *vma_manager,
		       bool need_dma32);

new member inserted in the middle:

struct drm_vma_offset_manager *vma_manager

which our code does not cover currently:

294 int vbox_mm_init(struct vbox_private *vbox)
295 {
296         int ret;
297         struct drm_device *dev = vbox->dev;
298         struct ttm_bo_device *bdev = &vbox->ttm.bdev;
299 
300 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 0, 0) && !defined(RHEL_77) && !defined(RHEL_81)
301         ret = vbox_ttm_global_init(vbox);
302         if (ret)
303                 return ret;
304 #endif
305         ret = ttm_bo_device_init(&vbox->ttm.bdev,
306 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 0, 0) && !defined(RHEL_77) && !defined(RHEL_81)
307                                  vbox->ttm.bo_global_ref.ref.object,
308 #endif
309                                  &vbox_bo_driver,
310 #if LINUX_VERSION_CODE >= KERNEL_VERSION(3, 15, 0) || defined(RHEL_71)
311                                  dev->anon_inode->i_mapping,
312 #endif
313 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 2, 0)
314                                  DRM_FILE_PAGE_OFFSET, true);
315 #else
316                                  true);
317 #endif

comment:8 by jsible, 5 years ago

Adding myself to CC. (Sorry for the noise, but according to https://www.virtualbox.org/ticket/1191 there's no other way for me to do this.)

comment:9 by David C Rankin, 5 years ago

Let me add an oddity with the 5.5 kernel that may be related here, or belong in a bug of it's own. Archlinux moved to the 5.5 kernel. I was currently building with testcase 5.2.35 (135458). All modules built. Attempting to run the Arch guest on the Arch host (with 5.5 kernel), starting the guest headless - reported it started. I opened the guest via-rdesktop and it was booting, but the boot process would not complete (could not create the network interface, udev kernel device manager will not start, and boot cannot create volatile files and directories (?)

So I built with 5.2.36 - same result, so I built with testcase 5.2.37 (135942), same result. The curiosity is that each time on Arch, the drivers built, but whatever changes were propagated to other packages for 5.5 support seem to have broken how host and (guest not yet updated to 5.5) interact.

I will downgrade and see if I can find any logs on the guest that shed light on this issue. If this belongs in a bug of it's own, feel free to migrate it, or let me know and I'll do it, but since it was kernel 5.5 related and this was the link I was given on the vbox-user mailing list -- I started here.

Last edited 5 years ago by David C Rankin (previous) (diff)

comment:10 by Hans-Peter Jansen, 5 years ago

cc me..

comment:11 by Jörg Skottke, 5 years ago

CC me

in reply to:  9 ; comment:12 by David C Rankin, 5 years ago

Replying to drankinatty:

Let me add an oddity with the 5.5 kernel that may be related here, or belong in a bug of it's own. Archlinux moved to the 5.5 kernel. I was currently building with testcase 5.2.35 (135458). All modules built. Attempting to run the Arch guest on the Arch host (with 5.5 kernel), starting the guest headless - reported it started. I opened the guest via-rdesktop and it was booting, but the boot process would not complete (could not create the network interface, udev kernel device manager will not start, and boot cannot create volatile files and directories (?)

So I built with 5.2.36 - same result, so I built with testcase 5.2.37 (135942), same result. The curiosity is that each time on Arch, the drivers built, but whatever changes were propagated to other packages for 5.5 support seem to have broken how host and (guest not yet updated to 5.5) interact.

I will downgrade and see if I can find any logs on the guest that shed light on this issue. If this belongs in a bug of it's own, feel free to migrate it, or let me know and I'll do it, but since it was kernel 5.5 related and this was the link I was given on the vbox-user mailing list -- I started here.

This is a problem with newer versions of packages in Archlinux that impact both the host and guest and leave I/O between the guest and host at a trickle. I downgraded both the host and guest multiple times to the package/kernel state they were in on 1/14/20, 1/21/20, 1/24/20, and 1/27/20 when all was working fine -- The host/guest communication remain broken. It is like changes were made to a config file that isn't reverted on downgrade continues to cause problem.

For example, with an Arch host booting a basic Arch guest to a text console login take over 600 seconds (10 minutes). Users cannot login because the 90 second login timeout expires between the time the user enters their user name and presses [Enter] and the display of the 'password:' prompt. root can login as there is no 90 second timeout (takes about 120 seconds for 'password:' to display after entering user name.

I have captured a strace of the activity, but am no expert in interpreting the strace output (other than to note each bit of I/O between host and guest takes 12-18 seconds to complete. Currently I have both host and guest with packages as of 1/24/20 with the 5.4.14 Arch kernel. I have the dmesg output from the guest as well.

It feels like this should be opened as it's own (related) but separate bug. I'll let the developers and admins make the call. If you want me to attach the strace and dmesg output here I will, but will wait until I get a decision on whether you want this is a separate bug before attaching.

in reply to:  12 ; comment:13 by Frank Batschulat (Oracle), 5 years ago

Replying to drankinatty:

It feels like this should be opened as it's own (related) but separate bug. I'll let the developers and admins make the call. If you want me to attach the strace and dmesg output here I will, but will wait until I get a decision on whether you want this is a separate bug before attaching.

Yes this should be filed as a separate bug, this does clutter up this bug and its content for no good reason with unrelated stuff.

comment:14 by Richlv, 5 years ago

Adding to cc.

comment:15 by Frank Batschulat (Oracle), 5 years ago

let's continue with where we left, the DRM changes related to the VMA manager task. Let's recap how we arrived there in the 5.5 kernel.

https://static.lwn.net/kerneldoc/gpu/drm-mm.html
https://www.kernel.org/doc/html/v4.10/gpu/drm-mm.html#

ttm_bo_device_init()
https://static.lwn.net/kerneldoc/gpu/drm-mm.html#the-translation-table-manager-ttm https://www.kernel.org/doc/html/v4.10/gpu/drm-mm.html#the-translation-table-manager-ttm

drm_gem_object_init()
https://static.lwn.net/kerneldoc/gpu/drm-mm.html#the-graphics-execution-manager-gem https://www.kernel.org/doc/html/v4.10/gpu/drm-mm.html#the-graphics-execution-manager-gem

drm_vma_offset_manager_init()
https://static.lwn.net/kerneldoc/gpu/drm-mm.html#vma-offset-manager https://www.kernel.org/doc/html/v4.10/gpu/drm-mm.html#vma-offset-manager

The vma-manager is responsible to map arbitrary driver-dependent memory regions into the linear user address-space. It provides offsets to the caller which can then be used on the address_space of the drm-device. It takes care to not overlap regions, size them appropriately and to not confuse mm-core by inconsistent fake vm_pgoff fields. Drivers shouldn’t use this for object placement in VMEM. This manager should only be used to manage mappings into linear user-space VMs.

We use drm_mm as backend to manage object allocations. But it is highly optimized for alloc/free calls, not lookups. Hence, we use an rb-tree to speed up offset lookups.

You must not use multiple offset managers on a single address_space. Otherwise, mm-core will be unable to tear down memory mappings as the VM will no longer be linear.

PATCH 1/8 drm/ttm: turn ttm_bo_device.vma_manager into a pointer
http://kernel1054.rssing.com/browser.php?indx=58795953&item=125227
https://patchwork.kernel.org/patch/11132229/

Rename the embedded struct vma_offset_manager, new name is _vma_manager.
ttm_bo_device.vma_manager changed to a pointer.

The ttm_bo_device_init() function gets an additional vma_manager
argument which allows to initialize ttm with a different vma manager.
When passing NULL the embedded _vma_manager is used.

All callers are updated to pass NULL, so the behavior doesn't change.
[...include/drm/ttm/ttm_bo_driver.h] 
> + * @vma_manager: A pointer to a vma manager or NULL.
[...ttm_bo_device_init()]
> +	if (!vma_manager)
> +		vma_manager = &bdev->_vma_manager;

but that has changed since:

[...include/drm/ttm/ttm_bo_driver.h]
 * @vma_manager: A pointer to a vma manager.
[...ttm_bo_device_init()]
	if (WARN_ON(vma_manager == NULL))
		return -EINVAL;

prep-work for removing TTMs internal vma manager entirely:

PATCH 2/8 drm/nouveau: switch to gem vma offset manager
http://kernel1054.rssing.com/browser.php?indx=58795953&item=125228

PATCH 3/8 drm/vram: switch to gem vma offset manager
http://kernel1054.rssing.com/browser.php?indx=58795953&item=125224

PATCH 4/8 drm/radeon: switch to gem vma offset manager
http://kernel1054.rssing.com/browser.php?indx=58795953&item=125231

PATCH 5/8 drm/amdgpu: switch to gem vma offset manager
http://kernel1054.rssing.com/browser.php?indx=58795953&item=125225

PATCH 6/8 drm/qxl: switch to gem vma offset manager
http://kernel1054.rssing.com/browser.php?indx=58795953&item=125226

PATCH 7/8 drm/vmwgfx: switch to own vma manager
http://kernel1054.rssing.com/browser.php?indx=58795953&item=125230

https://linuxlists.cc/l/1/linux-kernel/t/3298988/(patch_7_8)_drm_vmwgfx:_switch_to_own_vma_manager

and the final removal of the internal vma manager from TTM:

PATCH 8/8 drm/ttm: remove embedded vma_offset_manager
http://kernel1054.rssing.com/browser.php?indx=58795953&item=125229

No users left. Drivers either setup vma_offset_manager themself
(vmwgfx) or pass the gem vma_offset_manager to ttm_bo_device_init
(all other drivers). 

This removal went into the 5.5 linux kernel release:

https://www.fclose.com/linux-kernels/07/58/346-drm-ttm-remove-embedded-vma_offset_manager

https://www.fclose.com/linux-kernels/linux-5-5-release

https://elixir.bootlin.com/linux/v5.5/source/drivers/gpu/drm/drm_gem.c#L87

 * drm_gem_init - Initialize the GEM device fields
 * @dev: drm_devic structure to initialize
 */
int
drm_gem_init(struct drm_device *dev)
{
	struct drm_vma_offset_manager *vma_offset_manager;

	mutex_init(&dev->object_name_lock);
	idr_init_base(&dev->object_name_idr, 1);

	vma_offset_manager = kzalloc(sizeof(*vma_offset_manager), GFP_KERNEL);
	if (!vma_offset_manager) {
		DRM_ERROR("out of memory\n");
		return -ENOMEM;
	}

	dev->vma_offset_manager = vma_offset_manager;
	drm_vma_offset_manager_init(vma_offset_manager,
				    DRM_FILE_PAGE_OFFSET_START,
				    DRM_FILE_PAGE_OFFSET_SIZE);

	return 0;
}

will be setup as part of drivers call to drm_dev_init() or drm_dev_alloc().

VBOX is using the latter in trunk/src/VBox/Additions/linux/drm/vbox_drv.c:vbox_pci_probe().

So we will have to provide a VMA manager to ttm_bo_device_init() for kernel versions >= 5.5

Last edited 5 years ago by Frank Batschulat (Oracle) (previous) (diff)

in reply to:  13 comment:16 by David C Rankin, 5 years ago

Replying to fbatschu:

Yes this should be filed as a separate bug, this does clutter up this bug and its content for no good reason with unrelated stuff.

Done: https://www.virtualbox.org/ticket/19311 Thank You.

comment:17 by Frank Batschulat (Oracle), 5 years ago

the final changeset is:

$ svn status
M       src/VBox/Additions/linux/drm/vbox_drv.c
M       src/VBox/Additions/linux/drm/vbox_drv.h
M       src/VBox/Additions/linux/drm/vbox_fb.c
M       src/VBox/Additions/linux/drm/vbox_mode.c
M       src/VBox/Additions/linux/drm/vbox_ttm.c
M       src/VBox/HostDrivers/linux/Makefile
$ svn diff
Index: src/VBox/Additions/linux/drm/vbox_drv.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_drv.c	(revision 136094)
+++ src/VBox/Additions/linux/drm/vbox_drv.c	(working copy)
@@ -36,7 +36,6 @@
 #include <linux/console.h>
 #include <linux/vt_kern.h>
 
-#include <drm/drmP.h>
 #include <drm/drm_crtc_helper.h>
 
 #include "vbox_drv.h"
Index: src/VBox/Additions/linux/drm/vbox_drv.h
===================================================================
--- src/VBox/Additions/linux/drm/vbox_drv.h	(revision 136094)
+++ src/VBox/Additions/linux/drm/vbox_drv.h	(working copy)
@@ -48,6 +48,7 @@
 #include <linux/genalloc.h>
 #include <linux/io.h>
 #include <linux/string.h>
+#include <linux/pci.h>
 
 #if defined(RHEL_MAJOR) && defined(RHEL_MINOR)
 # if RHEL_MAJOR == 8 && RHEL_MINOR >= 1
@@ -103,7 +104,17 @@
 #define S64_MIN         ((s64)(-S64_MAX - 1))
 #endif
 
+#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 5, 0)
 #include <drm/drmP.h>
+#else /* >= KERNEL_VERSION(5, 5, 0) */
+#include <drm/drm_file.h>
+#include <drm/drm_drv.h>
+#include <drm/drm_device.h>
+#include <drm/drm_ioctl.h>
+#include <drm/drm_fourcc.h>
+#include <drm/drm_irq.h>
+#include <drm/drm_vblank.h>
+#endif /* >= KERNEL_VERSION(5, 5, 0) */
 #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 11, 0) || defined(RHEL_75)
 #include <drm/drm_encoder.h>
 #endif
Index: src/VBox/Additions/linux/drm/vbox_fb.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_fb.c	(revision 136094)
+++ src/VBox/Additions/linux/drm/vbox_fb.c	(working copy)
@@ -42,7 +42,6 @@
 #include <linux/fb.h>
 #include <linux/init.h>
 
-#include <drm/drmP.h>
 #include <drm/drm_crtc.h>
 #include <drm/drm_fb_helper.h>
 #include <drm/drm_crtc_helper.h>
Index: src/VBox/Additions/linux/drm/vbox_mode.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_mode.c	(revision 136094)
+++ src/VBox/Additions/linux/drm/vbox_mode.c	(working copy)
@@ -398,19 +398,26 @@
 static struct drm_encoder *vbox_best_single_encoder(struct drm_connector
 						    *connector)
 {
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 5, 0)
+        struct drm_encoder *encoder;
+
+        /* There is only one encoder per connector */
+        drm_connector_for_each_possible_encoder(connector, encoder)
+            return encoder;
+#else /* KERNEL_VERSION < 5.5 */
 	int enc_id = connector->encoder_ids[0];
 
 	/* pick the encoder ids */
 	if (enc_id)
-#if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 15, 0) || \
-    (defined(CONFIG_SUSE_VERSION) && \
-        LINUX_VERSION_CODE >= KERNEL_VERSION(4, 12, 0)) || \
-    defined(RHEL_76)
+# if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 15, 0) || \
+     (defined(CONFIG_SUSE_VERSION) && \
+         LINUX_VERSION_CODE >= KERNEL_VERSION(4, 12, 0)) || \
+     defined(RHEL_76)
 		return drm_encoder_find(connector->dev, NULL, enc_id);
-#else
+# else
 		return drm_encoder_find(connector->dev, enc_id);
-#endif
-
+# endif
+#endif /* KERNEL_VERSION < 5.5 */
 	return NULL;
 }
 
Index: src/VBox/Additions/linux/drm/vbox_ttm.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_ttm.c	(revision 136094)
+++ src/VBox/Additions/linux/drm/vbox_ttm.c	(working copy)
@@ -310,11 +310,12 @@
 #if LINUX_VERSION_CODE >= KERNEL_VERSION(3, 15, 0) || defined(RHEL_71)
 				 dev->anon_inode->i_mapping,
 #endif
-#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 2, 0)
-				 DRM_FILE_PAGE_OFFSET, true);
-#else
+#if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 5, 0)
+                                 dev->vma_offset_manager,
+#elif LINUX_VERSION_CODE < KERNEL_VERSION(5, 2, 0)
+				 DRM_FILE_PAGE_OFFSET,
+#endif
 				 true);
-#endif
 	if (ret) {
 		DRM_ERROR("Error initialising bo driver; %d\n", ret);
 #if LINUX_VERSION_CODE < KERNEL_VERSION(5, 0, 0) && !defined(RHEL_77) && !defined(RHEL_81)
Index: src/VBox/HostDrivers/linux/Makefile
===================================================================
--- src/VBox/HostDrivers/linux/Makefile	(revision 136094)
+++ src/VBox/HostDrivers/linux/Makefile	(working copy)
@@ -65,7 +65,7 @@
 	        cp vboxdrv/Module.symvers vboxnetflt; \
 	    fi; \
 	    echo "=== Building 'vboxnetflt' module ==="; \
-	    $(MAKE) KBUILD_VERBOSE=$(KBUILD_VERBOSE) -C vboxnetflt || exit 1; \
+	    $(MAKE) KBUILD_VERBOSE=$(KBUILD_VERBOSE) KBUILD_EXTRA_SYMBOLS=$(abspath vboxnetflt/Module.symvers) -C vboxnetflt || exit 1; \
 	    cp vboxnetflt/vboxnetflt.ko .; \
 	    echo; \
 	fi
@@ -76,7 +76,7 @@
 	        cp vboxdrv/Module.symvers vboxnetadp; \
 	    fi; \
 	    echo "=== Building 'vboxnetadp' module ==="; \
-	    $(MAKE) KBUILD_VERBOSE=$(KBUILD_VERBOSE) -C vboxnetadp || exit 1; \
+	    $(MAKE) KBUILD_VERBOSE=$(KBUILD_VERBOSE) KBUILD_EXTRA_SYMBOLS=$(abspath vboxnetadp/Module.symvers) -C vboxnetadp || exit 1; \
 	    cp vboxnetadp/vboxnetadp.ko .; \
 	    echo; \
 	fi
@@ -87,7 +87,7 @@
 	        cp vboxdrv/Module.symvers vboxpci; \
 	    fi; \
 	    echo "=== Building 'vboxpci' module ==="; \
-	    $(MAKE) KBUILD_VERBOSE=$(KBUILD_VERBOSE) -C vboxpci || exit 1; \
+	    $(MAKE) KBUILD_VERBOSE=$(KBUILD_VERBOSE) KBUILD_EXTRA_SYMBOLS=$(abspath vboxpci/Module.symvers) -C vboxpci || exit 1; \
 	    cp vboxpci/vboxpci.ko .; \
 	    echo; \
 	fi

and works with building against 5.5-rc7 (which is effectively 5.5.0) and 5.5.1 and 5.5.2.

actually a nit was luring which requires the following diff on the diffs above for the final changes to work with real old kernels (3.16.0)

$ svn diff
Index: src/VBox/Additions/linux/drm/vbox_drv.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_drv.c	(revision 136167)
+++ src/VBox/Additions/linux/drm/vbox_drv.c	(working copy)
@@ -36,10 +36,10 @@
 #include <linux/console.h>
 #include <linux/vt_kern.h>
 
+#include "vbox_drv.h"
+
 #include <drm/drm_crtc_helper.h>
 
-#include "vbox_drv.h"
-
 #if LINUX_VERSION_CODE >= KERNEL_VERSION(5, 1, 0) || defined(RHEL_81)
 #include <drm/drm_probe_helper.h>
 #endif
Index: src/VBox/Additions/linux/drm/vbox_fb.c
===================================================================
--- src/VBox/Additions/linux/drm/vbox_fb.c	(revision 136167)
+++ src/VBox/Additions/linux/drm/vbox_fb.c	(working copy)
@@ -42,11 +42,12 @@
 #include <linux/fb.h>
 #include <linux/init.h>
 
+#include "vbox_drv.h"
+
 #include <drm/drm_crtc.h>
 #include <drm/drm_fb_helper.h>
 #include <drm/drm_crtc_helper.h>
 
-#include "vbox_drv.h"
 #include <VBoxVideo.h>
 
 #if LINUX_VERSION_CODE < KERNEL_VERSION(4, 7, 0) && !defined(RHEL_74)

Last edited 5 years ago by Frank Batschulat (Oracle) (previous) (diff)

comment:18 by Frank Batschulat (Oracle), 5 years ago

fix in trunk in revision: r136116
fix in branch 6.0 in revision: r136121
fix in branch 5.2 in revision: r136124

Version 3, edited 5 years ago by Frank Batschulat (Oracle) (previous) (next) (diff)

comment:19 by Frank Batschulat (Oracle), 5 years ago

Resolution: fixed
Status: acceptedclosed

comment:20 by johnlee, 5 years ago

No win build in test builds for r136116 j

comment:21 by Frank Batschulat (Oracle), 5 years ago

Summary: linux: kernel 5.5-rc1/rc2 - we need changeslinux: kernel 5.5-rc1/rc2 - we need changes -> fixed in 6.1.4

comment:22 by Frank Batschulat (Oracle), 5 years ago

fix in official release 6.1.4

comment:23 by Frank Batschulat (Oracle), 5 years ago

also fixed in offical release 5.2.38 and 6.0.18

Note: See TracTickets for help on using tickets.

© 2024 Oracle Support Privacy / Do Not Sell My Info Terms of Use Trademark Policy Automated Access Etiquette