Extension Requirement

loetzsch · February 22, 2021, 5:08pm

Can be that the UVW mapping of the TV does not fit to your movie. Better first try it with a siple box and apply the stream to this box.

wojtek · February 23, 2021, 9:02am

We have implemented a dedicated viewer extension to use an HLS video stream as a texture, but it’s not yet available in the desktop application, it will be included in the next release.

As a part of the implementation we’ve extended the new JavaScript API with a new function: ViewerApi.createStreamTextureFromHtmlVideo(video) that must be used instead of ViewerApi.createTextureFromHtmlVideo(video) to create a texture from a video element if the video element represents a live streamed video. @Nisarg_Mehta, could you use the former function and check if it fixes the black image issue for you? The new function is currently available only after uploading the scene to our hosting, ignore that it fails in the desktop application.

Nisarg_Mehta · February 23, 2021, 10:16am

Thanks a lot @wojtek. This could possibly be a gamechanger for us. The new function works succesfully as you mentioned. The only small quibble is that the user has to click twice to play the video. The first click only loads the audio. Upon the second click, the video starts playing.

Please find the link below. The video will play when we click on the Rug.
https://3d.xporium.com/example-room-demo/

wojtek · February 23, 2021, 1:00pm

I think you have to make the video texture playing (material.play() or videoTexture.play()) from the Hls.Events.MANIFEST_PARSED event handler, because only then the video is ready to be played, and calling video.play() makes the video DOM element play, but not the texture.

Vladan · February 24, 2021, 10:51pm

Hi @wojtek,

This topic also interests me. However, I’m not sure what needs to be replaced.
Can you give the complete code?
How many videos can be streamed at the same time without any problems?
Is it possible to activate 2 (or more) videos with one switch (what would the cod look like)?

Thanks for your help!

wojtek · February 25, 2021, 1:04pm

In the next version of Shapespark we will add an extension that enables you to use a live video stream as a texture without custom coding.

For now, I think you can start with the script from the Jumbe’s post replacing createTextureFromHtmlVideo with createStreamTextureFromHtmlVideo and using material names from your scene instead of the names from our example-room scene. However, currently it will work only when the scene is hosted on our infrastructure.

Vladan · February 25, 2021, 6:51pm

I look forward to a new update Shapespark with these features.

I did a demo scene and entered the code according to your instructions, but it takes two clicks to watch the video. I don’t know how to solve it.

Nisarg_Mehta · February 27, 2021, 12:23pm

Sorry for the late revert @wojtek. Thanks for your reply. I will try this solution out shortly and get back to you if its working or not.

jorgearq · March 30, 2021, 6:06pm

wojtek:

To use a live stream as a video texture you can try the following approach (JavaScript programming required):

Use live streaming service supporting the HLS protocol and offering M3U8 manifests for the live streams. I am aware of Dacast , but there should be many solutions in this area.

Write a custom JavaScript snippet in the scene’s body-end.html file that:

uses the hls.js library to make a <video> element play your live stream based on the M3U8 manifest,

uses the <video> element as an argument to the Shapespark’s API Viewer.createTextureFromHTMLVideo function to create a live stream texture,

assigns the texture to the baseColorTexture property of the material you want the live stream to appear on.

We haven’t tested the above approach, but it seems to be complete and should work.

Hi @wojtek, do you have a date for the new update where this implementation is already coming?

jan · March 31, 2021, 7:15am

We have this implemented as an extensions, but unfortunately during testing it turned out that Apple devices have a bug with handling of HLS streams that prevents our extensions and the manual solution provided by @wojtek from working on these devices. Apple is working on a fix, but unfortunately we will need to delay the release until the fix is released.

jorgearq · March 31, 2021, 2:34pm

Roger that @jan, I hope apple corrects soon so you guys can get this update out! Thanks for your prompt reply.

njay · May 7, 2021, 3:36pm

Is there any update on whether the element Apple need to fix has been done yet @jan

jan · May 10, 2021, 11:22am

A version with the fix is not yet released, but is is already available in iOS 14.6 Beta 2: 218637 – Textures Fail to Render in WebGL from HLS Stream on iPhone 12 [iOS 14.2]