I am able to use a DVD but I am then not running in Live mode and it is very slow to boot.
Anyway, it turned out that my USB stick was bad. I got a new one and used the Linux Live USB Creator to create a bootable USB. I then didnt have the strange GRUB prompt!
However, I now have a new issue...