Image taken differently for each eye (UE4 + VXGI + VR)

Project: I have a scene in Unreal Engine that uses VXGI for global space lighting. The scene is well optimized and I have a good FPS (> 90fps). The scene includes a movable direct light (sun) that makes a transition day and night, and this is the only light source in the scene. The game should work on HTC Vive.

Problem: If I run the scene as a standalone application, I have the correct reflections and shadows. If I work in VR, I see the correct image in the left eye, and on the right, I don't see any shadows, and everything in the scene is too lit.

Solution I found, but not suitable in my case: - If I make the sun still, I see the correct image in VR - If I turn off VXGI indirect lighting for the sun (and keep VXGI on in the post processing scope) I get the correct one image for both eyes.

Question: Is there a way to make the dynamic sun work with VXGI and have a decent image in VR? (even if the frame rate drops a little ...)

PS This is a research project for Lighting Design and VXGI is a must as well as VR.

Thank.