Message Boards Message Boards

GROUPS:

[WSS18] Augmented & Virtual Reality implementation in Wolfram Language

Posted 2 months ago
507 Views
|
3 Replies
|
13 Total Likes
|

Augmented and Virtual reality functions in Mathematica

Overture

The goal of my project at Wolfram Summer School '18 was to implement VR and AR visualization features in the Wolfram Language. Conceptually, the underlying idea was to create two simple functions through which deploy in a VR or AR environment any 3D object created from Mathematica, spanning from a random solid (e.g. created using the ConvexHullMesh function):

ConvexHullMesh[RandomReal[5, {50, 3}]]

to a detailed view of 3D elevation data for a specific region:

ListPlot3D[GeoElevationData[GeoDisk[Entity["City", {"Tivoli", "Lazio", "Italy"}], 4 Quantity[1, "Kilometers"]], GeoZoomLevel -> 12]]

How

The two functions ARDeploy and VRDeploy work in conjunction with the WebVR BoilerPlate platform and AR.js library, which provide the required JavaScript tools to manage the VR/AR environments. The objects representation is translated from Mathematica into a js compatible format by Three.js and Aframe.js libraries. In particular, VRDeploy enables to investigate abstract mathematical (and any 3D Mathematica models) representations in a new environment leading to a clearer and easier understanding of them. In combination with mobile devices, equipped with a headset, enables the user full immersion experience. ARDeploy, on the other side, enables the "on-screen" exploration of the same objects compatible in VRDeploy, attaching it to a marker which could be visualized in the Mathematica notebook. The great difference between the two functions relies on the way the 3D object is explored. The VRDeploy offers a static full immersive point of view while ARDeploy offers a dynamic exploration.

These two functions (in form of Mathematica package) and all the required files can be downloaded at https://github.com/DomenicoRomano/ARVRDeploy.

VRDeploy Example Code

VRDeploy (as ARDeploy) works with any 3D surfaces from Plot3D or Example Data. In the example code below the 3D surface from Plot3D is translated in a VR object, VRDeploy returns a link to a webpage in which there is the WebVR environment with the Plot3D object visualised:

VRDeploy[Plot3D[0.5 *Sin[0.4*x y], {x, -4, 4}, {y, -4, 4}, MaxRecursion -> 4]]

Below a screenshot view of the webpage returned by VRDeploy as seen from mobile devices, this provides the full immersion experience. It is also possible to switch to normal view and then navigate the VR scene using the mobile device orientation hardware.

Mobile device view of the full immersive experience

Below a view of the same webpage as above returned by VRDeploy as seen from a laptop/desktop computer:

Web page visualizing the output of the above VRDeploy line of code

Another example is the VR visualization of the stars distribution around the Earth, using the StarData function. The following code extracts the position of the 35000 closest stars to our Sun, then converts the distances to parsec and VRDeploy them:

VRDeploy[Graphics3D[
  Sphere[QuantityMagnitude[
    DeleteMissing[StarData[EntityClass["Star", 
     List[Rule[EntityProperty["Star", "DistanceFromEarth"], 
       TakeSmallest[35000]]]], "HelioCoordinates"]]/3.26],
   .05], {
  Axes -> {True, True, True}, 
   AxesLabel -> {"pc", "pc", "pc"}, ImageSize -> Large}]]

Resulting in the picture below:

A full immersive view of the closest 35000 stars as seen from Earth, created with VRDeploy and the StarData function.


ARDeploy example

A simple example of ARDeploy function presented here use the Viking Lander model from the 3D Example data of Wolfram Mathematica, following is the code:

ARDeploy[ExampleData[{"Geometry3D", "VikingLander"}]]

The returned CloudObject is a link to a webpage, which can be easily converted to a QR code and scanned by mobile devices.

BarcodeImage[
 First[ARDeploy[ExampleData[{"Geometry3D", "VikingLander"}]]], "QR"]

The webpage opened in the mobile device will ask "to allow the application to access the camera". After granting the access to the camera the mobile device will recognize the marker, plotting on it the ARDeployed object:

The Viking Lander 3D model, taken from Mathematica Example data, visualized attached to a marker on a laptop screen.

Future Directions

Next steps require to enrich the visualization experience with interactive capabilities, displaying specific information or changing the object properties. Another required step is the marker independent placement of the object, using absolute GPS coordinates. The required mobile hardware improvements will be soon available. It will be also interesting to offer a virtual reality Mathematica notebook in which the user could experience the advantages to learn in a full-immersive three-dimensional virtual world.

3 Replies

This has great potential for education and entertainment at home and on the go. I hope to see more tech progress and applications in future!

Very nice topic, great work!

enter image description here - Congratulations! This post is now a Staff Pick as distinguished by a badge on your profile! Thank you, keep it coming!

Reply to this discussion
Community posts can be styled and formatted using the Markdown syntax.
Reply Preview
Attachments
Remove
or Discard

Group Abstract Group Abstract