It is possible to view Elite Dangerous using Google Cardboard. Currently (as of early August 2014), head tracking does not work, but 3D does.
You will need a decent modern Android phone for this to work - we don't know about iOS devices; we suspect that they wont work. So far the only device we've tested is the Nexus 5 but anything fairly modern should be good enough.
Make sure the window is as central as possible on your desktop to get the best effect.
Google Chrome Remote Desktop from computer to phone is very low latency (even over wifi - pretty much indistinguishable from a normal screen on our network) and is very playable given Elite Dangerous's “smooth” gameplay. It is also very easy to set up and initiate a session (1 tap and you're running).
If you find the latency is not fast enough due to poor wifi, there might be apps on the play store that allow a direct USB sharing which may offer better latency.