SetUndecorated (true) messing with toFront ()

I have these two JFrames that work fine when setUndecorated is set to false, but it doesn't work consistently when set to true;

frame = new JFrame("Name of the frame"); // main frame
frame.setAlwaysOnTop(false);
frame.setSize(width, height);
frame.add(canvas);
frame.setUndecorated(true);
frame.setVisible(true);

menu = new MenuUI(); // this is also a undecorated JFrame

      

        
        
        
      

    

triggered by a mouse event, but sometimes it appears in front and sometimes it appears in the back of the main frame ...

public static void checkMove(int action, MouseEvent e) {
    int x = e.getX();
    if(x == 0){
        menu.setVisible(true);
        menu.toFront();
    }else{
        menu.setVisible(false);
    }

      

        
        
        
      

    

I tried to do frame.toBack();

but it also sends the frame behind everything else ... How can I bring an unecorated JFrame in front of another unordered JFrame and make it consistent?

EDIT: Menu frame disappears on mouse event

contentPane.addMouseListener(new MouseAdapter() {
        @Override
        public void mouseExited(MouseEvent e) {
            setVisible(false);
        }
    }); 

      

        
        
        
      

    

EDIT2: Following mKorbel's answer, I tried to create a JDialog, but I get the same result as with a JFrame. How do I set the parent if I extend the JDialog?

package menu;

import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;

import javax.swing.GroupLayout;
import javax.swing.GroupLayout.Alignment;
import javax.swing.JDialog;

import openGL.main.openGLMain;

@SuppressWarnings("serial")
public class menuTest extends JDialog {

public menuTest() {
    setUndecorated(true);
    setBounds(0, 0, 250, frame.getHeight());
    GroupLayout groupLayout = new GroupLayout(getContentPane());
    groupLayout.setHorizontalGroup(
        groupLayout.createParallelGroup(Alignment.LEADING)
            .addGap(0, 450, Short.MAX_VALUE)
    );
    groupLayout.setVerticalGroup(
        groupLayout.createParallelGroup(Alignment.LEADING)
            .addGap(0, 700, Short.MAX_VALUE)
    );
    getContentPane().setLayout(groupLayout);
    getContentPane().addMouseListener(new MouseAdapter() {
        @Override
        public void mouseExited(MouseEvent e) {
                setVisible(false);
        }
    });
}
}