4.3.2 PATM VERR_SSM_LOADED_TOO_MUCH when resuming from snapshot => Fixed in SVN

[Jacob Klein]

I use BOINC on my Windows 8.1 x64 machine, to work on Test4Theory VM tasks (Linux 2.6 client VMs), and we're having a problem resuming snapshots.

Essentially, it looks like there's an error when PATM is loading, something about: !pSSM->u.Read.fEndOfData .... resulting in VERR_SSM_LOADED_TOO_MUCH

Log of error:

00:00:01.505325 AssertLogRel D:\tinderbox\win-4.3\src\VBox\VMM\VMMR3\SSM.cpp(6212) int __cdecl ssmR3DataReadBufferedV2(struct SSMHANDLE *,void *,unsigned __int64): !pSSM->u.Read.fEndOfData
00:00:01.505340 cbBuf=1SSM: LoadExec failed for 'PATM' instance #0 (version 55, pass 0xffffffff): VERR_SSM_LOADED_TOO_MUCH
00:00:01.505382 VMSetError: D:\tinderbox\win-4.3\src\VBox\VMM\VMMR3\SSM.cpp(8275) int __cdecl ssmR3LoadExecV2(struct VM *,struct SSMHANDLE *); rc=VERR_SSM_LOADED_TOO_MUCH
00:00:01.505385 VMSetError: Failed to load unit 'PATM'

I'm positive we did not have this error in 4.2.16, and I'm pretty sure that my testing shows we did not have it in 4.2.18 or 4.3.0 either.

Did the 4.3.2 release break something here?

[Jacob Klein]

Here's a little more log detail, which might be helpful, including the final error message which said:

Power up failed (vrc=VERR_SSM_LOADED_TOO_MUCH, rc=E_FAIL (0X80004005))

00:00:01.505325 AssertLogRel D:\tinderbox\win-4.3\src\VBox\VMM\VMMR3\SSM.cpp(6212) int __cdecl ssmR3DataReadBufferedV2(struct SSMHANDLE *,void *,unsigned __int64): !pSSM->u.Read.fEndOfData
00:00:01.505340 cbBuf=1SSM: LoadExec failed for 'PATM' instance #0 (version 55, pass 0xffffffff): VERR_SSM_LOADED_TOO_MUCH
00:00:01.505382 VMSetError: D:\tinderbox\win-4.3\src\VBox\VMM\VMMR3\SSM.cpp(8275) int __cdecl ssmR3LoadExecV2(struct VM *,struct SSMHANDLE *); rc=VERR_SSM_LOADED_TOO_MUCH
00:00:01.505385 VMSetError: Failed to load unit 'PATM'
00:00:01.505412 NAT: set redirect TCP host port 55246 => guest port 80 @ 10.0.2.15
00:00:01.505731 Changing the VM state from 'LOADING' to 'LOAD_FAILURE'.
00:00:01.505770 Changing the VM state from 'LOAD_FAILURE' to 'POWERING_OFF'.
...
...
...
00:00:01.505949 Changing the VM state from 'POWERING_OFF' to 'OFF'.
00:00:01.506052 Console::powerDown(): A request to power off the VM has been issued (mMachineState=Restoring, InUninit=0)
00:00:01.572160 VRDP: TCP server closed.
00:00:01.572838 Changing the VM state from 'OFF' to 'DESTROYING'.
...
...
...
00:00:01.633264 Changing the VM state from 'DESTROYING' to 'TERMINATED'.
00:00:01.702868 ERROR [COM]: aRC=E_FAIL (0x80004005) aIID={8ab7c520-2442-4b66-8d74-4ff1e195d2b6} aComponent={Console} aText={Failed to load unit 'PATM' (VERR_SSM_LOADED_TOO_MUCH)}, preserve=false
00:00:01.702998 Power up failed (vrc=VERR_SSM_LOADED_TOO_MUCH, rc=E_FAIL (0X80004005))

[Jacob Klein]

I believe it is possible that the solution for Ticket #12222 caused this ticket's problem.

[Jacob Klein]

Full VBox log when trying to reopen the 4.3.2 snapshot

[Jacob Klein]

I attached the full log, in case it gives any clues/hints as to the problem.

Also, the steps I use to replicate the problem are:

• Set BOINC so that "Leave applications in memory while memory is suspended" (LAIM) is unchecked
• Get a T4T task in BOINC
• Let it run for 25 minutes
• Open up Oracle VM VirtualBox Manager
• Verify that the T4T VM has, on the snapshots tab, a snapshot that says 1200 (indicating that 2 snapshots were taken, the 10-minute mark and the 20-minute mark)
• In BOINC, Activity -> Suspend
• Wait 30 seconds
• In BOINC, Activity -> Run based on preferences
• See if Oracle VM VirtualBox Manager shows that the task restarted and is running
• When it's not restarting properly, inspect the VBox.log file in the BOINC slot directory for that task

[Jacob Klein]

I have made some progress in identifying the source of the problem!

T4T tasks run VMs of Type "Linux" and Version "Linux 2.6 / 3.x", with the hardware virtualization setting "Enable VT-x/AMD-V" UNCHECKED. If I check that "Enable VT-x/AMD-V" setting before the snapshot is taken, I am then able to restore the snapshot and start the VM without getting the PATM VERR_SSM_LOADED_TOO_MUCH error.

Are snapshots taken WITHOUT "Enable VT-x/AMD-V" being somehow corrupted?

Is VirtualBox somehow not able to load such snapshots?

For additional reference, the full VM description of the T4T task's VM, is below. Looking forward to your response.

Thanks, Jacob

===============================================================
T4T VM Settings (Summary):

General:
Operating System: Linux 2.6 / 3.x

System:
Base Memory: 256 MB
Boot Order: Hard Disk
Acceleration: PAE/NX

Display:
Video Memory: 8 MB
Remote Desktpo Server Port: 6xxxx
Video Capture: Disabled

Storage:
Controller: Hard Disk Controller
IDE Primary Master: vm_image.vdi (Normal, 9.00 GB)
Controller: Floppy Controller
Floppy Device 0: vm_floppy_7.img (24.00 KB)

Audio:
Disabled

Network:
Adapter 1: Intel PRO/1000 MT Desktop (NAT)

USB:
Disabled

Shared Folder:
None
===============================================================
T4T VM Settings (Details):

General - Basic:
Type: Linux
Version: Linux 2.6 / 3.x

General - Advanced:
Snapshot Folder: Has one
Shared Clipboard: Disabled
Drag'n'Drop: Disabled
Removable Media: Remember Runtime Changes: Checked
Mini ToolBar: Show in Fullscreen/Seamless: Checked
Show at Top of Screen: Unchecked

General - Description:
[Blank]

System - Motherboard:
Base Memory: 256 MB
Boot Order: Hard Disk checked at the top, with unchecked: Floppy, CD/DVD, Network
Chipset: PIIX3
Pointing Device: PS/2 Mouse
Extended Features:
Enable I/O APIC: Checked
Enable EFI (special OSes only): Unchecked
Hardware Clock in UTC Time: Unchecked

System - Processor:
Processor(s): 1
Exection Cap: 100%
Extended Features:
Enable PAE/NX: Checked

System - Acceleration:
Hardware Virtualization:
Enable VT-x/AMD-V: Unchecked
Enable Nested Paging: Checked, disabled

Display - Video:
Video Memory: 8 MB
Monitor Count: 1
Extended Features:
Enable 3D Acceleration: Unchecked
Enable 2D Video Acceleration: Unchecked

Display - Remote Display:
Enable Server: Checked
Server Port: 63219
Authentication Method: Null
Authentication Timeout: 0
Extended Features:
Allow Multiple Connections: Unchecked and disabled

Display - Video Capture:
Enable Video Capture: Disabled
File Path: Has one, disabled
Frame Size: 1024 x 768 4:3, disabled
Frame Rate: 25 fps, disabled
Quality: 512 kbps, disabled
Screens: Screen 1 checked, disabled

Storage:
Tree:
- Controller: Hard Disk Controller (Type: PIIX4, Use Host I/O Cache checked)
-- vm_image.vdi: IDE Primary Master, Solid-state Drive unchecked
-- Type (Format): Normal (VDI)
-- Virtual Size: 9.00 GB
-- Actual Size: 1.58 GB
-- Details: Dynamically allocated storage
-- Location: Has one
-- Attached to: VM
- Controller: Floppy Controller (Type: I82078, Use Host I/O Cache checked)
-- vm_floppy_7.img: Floppy Device 0
-- Type: Image
-- Size: 24.00 KB
-- Location: Has one
-- Attached to: VM

Audio:
Enable Audio: Unchecked and disabled
Host Audio Driver: Windows DirectSound, disabled
Audio Controller: ICH AC97, disabled

Network - Adapter 1:
Enable Network Adapter: Checked, disabled
Attached to: NAT
Name: Blank, disabled
Adapter Type: Intel PRO/1000 MT Desktop (82540EM), disabled
Promiscuous Mode: Deny, disabled
MAC Address: 08002748E36E, disabled
Cable Connected: Checked
Port Forwarding: 1 Rule
Port Forwarding Rule:
Name: vboxwrapper
Protocol: TCP
Host IP: 127.0.0.1
Host Port: (Remote Desktop Server Port minus 1)
Guest IP: [Blank]
Guest Port: 80

Network - Adapter 2/3/4: Disabled

Serial Ports - Ports 1/2: Disabled

USB:
Enable USB Controller: Unchecked, disabled
Enable USB 2.0 (EHCI) Controller: Unchecked, disabled
USB Device Filters: None, disabled

Shared Folders:
Machine Folders: None
Transient Folders: None

[Jacob Klein]

Any word on when this issue (caused by resuming snapshots without the "Enable VT-x/AMD-V" feature enabled) ... will be looked at or resolved?

[Frank Mehnert]

Problem is known and we are working on a fix. No ETA though yet, sorry.

[Frank Mehnert]

Hopefullyl finally fixed in the next maintenance release.

[Jacob Klein]

Thanks for being prompt -- Looking forward to testing the fix!

[Jake Ochs]

same here

vbox 4.3.2. windows 7 64 bit host xubuntu 13.10 guest

Failed to open a session for the virtual machine saucy.

Failed to load unit 'PATM' (VERR_SSM_LOADED_TOO_MUCH).

Result Code: E_FAIL (0x80004005)
Component: Console
Interface: IConsole {8ab7c520-2442-4b66-8d74-4ff1e195d2b6}

[Frank Mehnert]

Fix in 4.3.4.

[Jacob Klein]

Now getting a new/different error resuming a non-VT-X BOINC Test4Theory snapshot.

The new error is:
SSM: LoadExec failed for 'PATM' instance #0 (version 56, pass 0xffffffff): VERR_SSM_STRUCTURE_MAGIC

See Ticket #12414 ==> https://www.virtualbox.org/ticket/12414

[sergeda]

Replying to frank:

Fix in 4.3.4.

I have 4.3.4-91027 installed on Windows 8 and have exactly this issue. I can't resume VM with Windows guest. It gives me this error "ERR_SSM_LOADED_TOO_MUCH". How can I fix this issue? I need access to my VM data.

[Frank Mehnert]

The bug should be finally fixed in VBox 4.3.6. Please continue the discussion in #12414.