Skip to content

XR

Jump to bottom
Philippe Coval edited this page Oct 11, 2019 · 55 revisions

WoTxR : WoT to XR(VR/AR)

Presentation

WORK IN PROGRESS

As a proof of concept, webthing-webapp can be also declined to use 3D widgets and provide immersive web experience:

Special thanks to Fabien, for introducing to A-Frame:

Work is still in progress at:

#wotxr-20190320rzr

DEMO

Usage:

Misc:

TWINS

COMPONENTS:

CONTROLLER:

Is there any X/Y track pad events ?

com.sec.android.app.svrbrowser.apk

WEBAPP:

webthings-webapp-aframe

More details on WebApp page

TODO

How to clean cache of sVRbrowser ?

XR

More hints to come at:

MISC

Demos:

Ideas:

  • visualize opensensemap's sensors
  • generic 3d browser

Log:

A-FRAME:

ANDROID:

adb="sudo adb" # TODO: setup udev

package=com.sec.android.app.sbrowser
url='http://is.gd/webxr'
activity='com.sec.android.app.sbrowser/.SBrowserMainActivity'

${adb} kill-server
${adb} devices -l # Plug USB before
#| List of devices attached
#| ad0ffffffffffffff      device

# Stop package
${adb} shell am force-stop $package

# Clean cache
${adb} shell pm clear $package

# Start package
${adb} shell monkey -p $package -c android.intent.category.LAUNCHER 1

# Start activity and open URL
${adb} shell am start \
-n $activity \
-a android.intent.action.VIEW \
-d "$url"

# Open URL (for default browsers)
${adb} shell am start -a android.intent.action.VIEW -d "$url"

${adb} shell dumpsys package $package

${adb} shell ip address show dev wlan0 # note ip

${adb} tcpip 5555 # then disconnect usb

${adb} connect 192.168.1.58 # Or adapt
#| connected to 192.168.1.58:5555

SVRBROWER

adb="sudo adb" # or setup udev

package=com.sec.android.app.svrbrowser
activity=com.sec.android.app.svrbrowser/.UnityPlayerNativeActivity
application=com.sec.android.app.svrbrowser.VrBrowserApplication
url='https://is.gd/webxr'

${adb} shell pm list package | grep ${package}
${adb} shell dumpsys package $package | grep ${activity}
${adb} shell am force-stop $package
${adb} shell pm clear $package 

# Plug in GearVR headset, it will switch to VR dashboard
${adb} shell am start -n "$activity"

# TODO: pass url
${adb} shell am start \
 -n $activity \
 -a android.intent.action.VIEW \
 -d "$url"


# Workaround:

text="is.gd/webxr"
# text="http%3A%2F%2F$text"

# 1st Focus on URL bar (TODO : use ${adb} shell input tap 300 700 ?)
${adb} shell input keyevent 67 && ${adb} shell input text "$text" && ${adb} shell input keyevent 66

LOGIN:

Automated login hack:

gw_url="https://sosg.mozilla-iot.org"
token="eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Ijg1NWEyYWVjLWI2NTMtNDQ3Mi04MDllLTQyNGI4ZTk1NjQyMiJ9.eyJjbGllbnRfaWQiOiJsb2NhbC10b2tlbiIsInJvbGUiOiJhY2Nlc3NfdG9rZW4iLCJzY29wZSI6Ii90aGluZ3M6cmVhZHdyaXRlIiwiaWF0IjoxNTUyNzM5MzQ4LCJpc3MiOiJodHRwczovL3Nvc2cubW96aWxsYS1pb3Qub3JnIn0.JLgc7Gd5R5IvG5AHdQb1zBG311sOyGjT4bPlqPUYma1IPaxDJYCJQvfurqkNVd2l7MtLP7IGZlBdN7B3pDLzMA"
base_url="https://webthings-webapp.samsunginter.net/"
base_url="${base_url}/login.html"
url="$base_url#?url=${gw_url}&token=${token}"

# On GNU/Linux:
xdg-open "$url"

# With exokit (TBC):
url='http://rzr.online.fr/x/vr/' # TODO
mldb launch com.webmr.exokit -v "ARGS=node --experimental-worker . ${url}"


#git clone ... 
base_url="file://$PWD//"
base_url="${base_url}/login.html"
url="$base_url#?url=${gw_url}&token=${token}"

# On GNU/Linux
xdg-open "$url"

# With exokit:
mldb launch com.webmr.exokit -v "ARGS=node --experimental-worker . ${url}"

AR:

Normally it should display a cube that goes from Red to Green and then Blue:

url=http://samsunginter.net/color-sensor-js/example/color-sensor-webthing/extra/aframe.html
mldb launch com.webmr.exokit -v "ARGS=node --experimental-worker . ${url}"

wotxr-aframe-20190320rzr

SETUP

See WebApp page

oauthClients.register(
  new ClientRegistry(new URL('https://webthings-webapp.samsunginter.net/login.html'),
                     'webthings-webapp.samsunginter.net',
                     'WebApp Token Service',
                     'webthings-webapp.samsunginter.net',
                     '/things:readwrite')
);


oauthClients.register(
  new ClientRegistry(new URL('https://webthings-webapp.samsunginter.net/index.html'),
                     'webthings-webapp.samsunginter.net',
                     'WebApp Token Service',
                     'webthings-webapp.samsunginter.net',
                     '/things:readwrite')
);

BABYLONJS: (WIP)

BabylonJs

ACTUATOR

NOTES

CORS

It appears that Glitch uses cookies, it would break CORS if rejected:

GLTF:

sudo snap install --classic  blender # 2.80
/snap/blender/current/blender assets/house/house.blend

LINKS:

LICENSE:

LOST ?

Check Concept page for overview, Gateway to get started, IotJs page to install runtime to build webthing as explained in Home page.

For further experiments check Social and Sensor, or Extra parts like WebApp (for Tizen or PWA) or MCU info about running on other microcontrollers not supported by TizenRT.

While Home focus mostly on using iotjs to build webthings (on GNU/Linux or TizenRT for ARTIK05X devices).

FEEDBACK: WELCOME

This document is still in draft state, but reviews are always welcome, if you try to replicate it and stuck on missing instructions I would appreciate that you file issues or even better make pull request (just edit in github) that insert "TODO marks" in following chapters, like:

  • TODO: please explain more this chapter and then remove this TODO line

Community contributions are welcome at:

Support is also possible, ask in:

DISCLAIMER:

WARNING: Developement branches could break over time.

Instead of maintaining "quick and dirty" demo code, I decided to split demo in smaller independents parts (which can reused) and I am upstreaming the most I can.

Then support can be done on mainline branches (or released versions).

Note that, Upstreaming can be a slow process, so snapshots links will remain until 100% of code is upstreamed.

Licence:

Reference documentation is at:

INDEX

Clone this wiki locally