Problems with dynamic video controls

Customization
1

Hi there!

I’m implemeting a dynamic room for my customer, where admin can customize colors, banners, tvs and so on. We managed to create the texture within Shapespark and body-end, but we’re struggling with some issues when we need to control the videos with chroma key enabled.

After updating the texture:

  • the material remains black even after setting a preload=“auto” in the video tag. Desired: first frame of video in pause;
  • when trying to toggle play, the video always rewinds. Desired: after pause and play again, continues from where it stoped;
  • this one could be due to the video size, but it’s keep freezing while playing. Wish I could preload it and just then display the material, but if not possible will compact the video in the future.

My code to this part:

function updateHumanMaterial(material, file) {
  if (file.thumb?.url) {
    updateImageMaterial(material, file.thumb.url);
  }

  const video = document.createElement("video");
  video.crossOrigin = "anonymous";
  video.preload = "auto";
  video.muted = false;
  video.loop = true;
  video.src = `${file.url}#t=0.5`;

  video.addEventListener(
    "canplay",
    () => {
      const texture = viewer.createTextureFromHtmlVideo(video);
      material.baseColorTexture = texture;

      viewer.requestFrame();

      HUMAN = texture; // global variable

      showHumans(); // this set visibility to the materials
      video.remove();
    },
    { once: true }
  );
}

function showHumans() {
  showObject(objKey?.human);
  showObject(objKey?.shadow);
  setPlayHumanButtonVisible(true);
}

function showObject(objName) {
  if (objName) {
    const objs = viewer.findNodesOfType(objName);
    objs?.forEach((obj) => obj.show());
  }
}

function playHuman() {
  HUMAN?.play();
  setPlayHumanButtonVisible(false);
}

function pauseHuman() {
  HUMAN?.pause();
}

function togglePlayHuman() {
  if (HUMAN) {
    if (HUMAN.isPlaying) {
      pauseHuman();
    } else {
      playHuman();
    }
  }
}

Thanks!

2

The scene: 3D scene

3

Anyone? I was hoping to solve this quickly.

4

Hi!

As for

the material remains black even after setting a preload=“auto” in the video tag. Desired: first frame of video in pause;

This is currently not supported in shapespark. You can try to make a workaround around this and first show an image and then when you want to start the video change the texture to the video and play it. We will however investigate it and see what we can do about making this work out of the box.

when trying to toggle play, the video always rewinds. Desired: after pause and play again, continues from where it stoped;

Shapespark currently doesn’t support adding the #t=0.5 at the end of the url. This causes this rewinding behavior. You can easily avoid this by setting the currentTime on the play event to 0.5 on the video directly. (Alternatively you could try cutting the video) We will also look into it and try to make it more robust, so that you can supply the #t=0.5 at the end of the url.

As for the last one

this one could be due to the video size, but it’s keep freezing while playing. Wish I could preload it and just then display the material, but if not possible will compact the video in the future.

This is probably because of the video’s size, your connection and the connection to the server serving the video. You can try to use canplaythrough instead of canplay, see the description of canplaythrough from mozilla dev docs

The canplaythrough event is fired when the user agent can play the media, and estimates that enough data has been loaded to play the media up to its end without having to stop for further buffering of content.

I hope this helps!

5

The next version of Shapespark should have the first two issues that you have found fixed.

That is, for videos with autoplay=false the thumbnail should be visible instead of the black image as well as the rewind should work properly on your video with a timestamped url.

6

Hello! Thanks for your answer. Really helped here.
I’ll leave my current solution while the next version isn’t released yet, in case anyone face this.

function updateHumanMaterial(material, file) {
  if (file.thumb?.url) {
    updateImageMaterial(material, file.thumb.url);
  }

  const video = document.createElement("video");
  video.crossOrigin = "anonymous";
  video.autoplay = true; // NEW: set autoplay to get the first frame of video later
  video.muted = true; // NEW: I dont want this, but need to autoplay
  video.src = file.url; // NEW: removed t=0.5

  video.addEventListener(
    "canplay",
    () => {
      const texture = viewer.createTextureFromHtmlVideo(video);
      material.baseColorTexture = texture;

      viewer.requestFrame();
      HUMAN = texture;

      setFirstFrameOnHuman(); // NEW: function to generate first frame
      video.remove();
    },
    { once: true }
  );
}

function setFirstFrameOnHuman() {
  function pauseForFirstFrame() {
    if (HUMAN.video.currentTime > 0) {
      pauseHuman();
      HUMAN.video.muted = false; // return the audio

      setHumanOpacity(1);
      showHumans();

      HUMAN.video.removeEventListener("timeupdate", pauseForFirstFrame); // just first event will fire
    }
  }

  if (HUMAN) {
    HUMAN.video.addEventListener("timeupdate", pauseForFirstFrame);
    setHumanOpacity(0); // show the material, but with no opacity
    showHumans();
    playHuman(); // force play
  }
}

function showHumans() {
  showObject(objKey?.human);
  showObject(objKey?.shadow);
  setPlayHumanButtonVisible(true);
}

function showObject(objName) {
  if (objName) {
    const objs = viewer.findNodesOfType(objName);
    objs?.forEach((obj) => obj.show());
  }
}

function setHumanOpacity(opacity) {
  setMaterialOpacity(materialKey?.human, opacity);
  setMaterialOpacity(materialKey?.shadow, opacity * 0.4);
}

function setMaterialOpacity(materialName, opacity) {
  if (materialName) {
    const material = viewer.findMaterial(materialName);
    if (material) {
      material.opacity = opacity;
    }
  }
}

function playHuman() {
  HUMAN?.play();
  setPlayHumanButtonVisible(false);
}

function pauseHuman() {
  if (HUMAN) {
    const time = HUMAN.image?.currentTime; // NEW: storage the current time to resume play
    HUMAN.pause();
    setTimeout(() => (HUMAN.image.currentTime = time), 1);
  }
}

function togglePlayHuman() {
  if (HUMAN) {
    if (HUMAN.isPlaying) {
      pauseHuman();
    } else {
      playHuman();
    }
  }
}

Thanks!

1 Like
7

Many thanks for sharing your solution!

I also am happy to inform you that because of today’s release (Shapespark releases - #280 by krzysztof) the first two dots of your original post should be working out of the box. That is the passing of timestamp in the url is accepted and won’t cause the play as rewind effect. The texture with autoplay=false should also have the first frame visible instead of black background.