Usage with React + TypeScript about agorawebsdk-ng HOT 5 CLOSED

OK, I will close this issue, if you have any other questions about this SDK's usage in the future, just post a new issue and related to this.

from agorawebsdk-ng.

I'm very glad you liked this SDK, and that's exactly why we refactor the older SDK.

Your solution looks elegant and simple. In your case (Video-only Live), it will work fine. But if you use Audio-Video Live in the future, this may cause some problems. user-published or user-unpublished is a Track-level event, not a Stream-level event. That's means if you receive user-published or user-unpublish, it does not mean that the Stream has been added or removed (expect Audio-only or Video-only).

const videoTrack = await AgoraRTC.createCameraVideoTrack();
const audioTrack = await AgoraRTC.createMicrophoneAudioTrack();

await client.publish([audioTrack]);
// at this point, the remote client will receive `user-published`(mediaType: "audio")

// ... do something

await client.publish([videoTrack]);
// the remote client will receive the 2nd `user-published`(mediaType: "video")

// ...do something
await client.unpublish([videoTrack])
// in this case, the remote client will receive `user-unpublished`
// But the Stream is not removed because the audio track is still alive

If you want to build an audio-video live application in the future, you should pay attention to these designs.

At last, you should use LocalVideoTrack.play to play the video tracks. Because the MediaStreamTrack of the LocalTrack may change due to the pre-processing of the SDK(such as beauty effect/switching camera). If you are not using these APIs, you can ignore it.

Thank you for sharing!

from agorawebsdk-ng.

Thanks very much @disoul.

I am going to want to use both audio and video in my integration. Are there any changes you might recommend to my design to ensure I handle audio/video tracks correctly?

I'm not using the Beauty Effects features at the moment, but I may choose to use camera switching. My concern with using LocalVideoTrack.play is that I'm handling video element creation through the React Components, and I wanted to allow the stream state to control their creation/being destroyed.

So, that StreamContainer component above maps over the streams, like this:

export interface IStreamContainerProps {
    mediaStreams: MediaStream[],
    containSourceOnly: boolean;
}

const StreamContainer: React.FC<IStreamContainerProps> = props => {
    return (
        <div className="stream-container">
            {
                props.mediaStreams.map((stream, index) => (
                        <IndividualVideoView 
                            key={stream.id}
                            mediaStream={stream} 
                            containSourceOnly={props.containSourceOnly}
                            location={index}
                            outOf={props.mediaStreams.length}
                        />
                    )
                )
            }
        </div>
    );
}

And IndividualVideoView handles the styling of the video elements:

interface IVideoViewProps {
    mediaStream: MediaStream;
    containSourceOnly: boolean;
    location: number;
    outOf: number;
}

const IndividualVideoView: React.FC<IVideoViewProps> = (props) => {
    const videoRef = useRef<HTMLVideoElement>(null);

    useEffect(() => {
        if (videoRef.current) {
            videoRef.current.srcObject = props.mediaStream;
        }
    }, []);

    const baseName = 'individual-video-view';
    const classes = classNames({
        [baseName]: true,
        [`${baseName}--contain`]: props.containSourceOnly,
        [`${baseName}--cover`]: !props.containSourceOnly,
        [`${baseName}--full`]: props.outOf === 1,
        [`${baseName}--half-left`]: props.location === 0 && props.outOf === 2,
        [`${baseName}--half-right`]: props.location === 1 && props.outOf === 2,
        [`${baseName}--quad-top-left`]: props.location === 0 && props.outOf === 4,
        [`${baseName}--quad-top-right`]: props.location === 1 && props.outOf === 4,
        [`${baseName}--quad-bottom-left`]: props.location === 2 && props.outOf === 4,
        [`${baseName}--quad-bottom-right`]: props.location === 3 && props.outOf === 4,
    });

    return (
        <video 
            autoPlay
            muted
            className={classes}
            ref={videoRef}
        />
    );
};

from agorawebsdk-ng.

First, I understand you can not control the video element creation because LocalVideoTrack.play will create a video element automatically by SDK. But you can control the video element container, you can change the video playback view by adding some css styles to the container element.

This is my example:

interface VideoPlayerProps {
  track: ILocalVideoTrack | IRemoteVideoTrack;
}

const VideoPlayer = (props: VideoPlayerProps) => {
  const container = useRef<HTMLDivElement>(null);
  // use React's useEffect to control the video element‘s creation and destruction
  useEffect(() => {
    if (!container.current) return;
    props.track.play(container.current, { fit: "contain" });
    return () => props.track.stop();
  }, [container, props.track]);
  return (
    { /** you can set your css here **/ }
    <div ref={container} style={{ width: "100%", height: "100%"}}></div>
  );
}

Second, if you are going to use both audio and video in the future, IStream maybe not suitable. I think it would be better to use IAgoraRTCRemoteUser directly or wrap it. In this case, you should pay attention to how to trigger the React's re-renders. My solution is to use mobx to create a RemoteUserWrapper with some observable members.

class AgoraRTCRemoteUserWrapper {
  @observable
  audioTrack?: IRemoteAudioTrack;
  @observable
  videoTrack?: IRemoteVideoTrack;
  @observable
  hasAudio: boolean = false;
  @observable
  hasVideo: boolean = false;
  @observable
  audioMuted: boolean = false;
  @observable
  videoMuted: boolean = false;

  user: IAgoraRTCRemoteUser;

  constructor(user: IAgoraRTCRemoteUser) {
    this.user = user;
    this.updateFromUser(user);
  }

  @action
  public updateFromUser(user: IAgoraRTCRemoteUser): void {
    this.audioTrack = user.audioTrack;
    this.videoTrack = user.videoTrack;
    this.hasAudio = user.hasAudio;
    this.hasVideo = user.hasVideo;
    this.audioMuted = user.audioMuted;
    this.videoMuted = user.videoMuted;
  }
}

// ...

client.on("user-unpulbished", (user, mediaType) => {
  // dispatch a mobx action to triggle react's re-render
  remoteUserWrapper.updateFromeUser(user);
});

Hope this helped, thanks.

from agorawebsdk-ng.

Thanks very much for your response @disoul. I have read it and I'll respond in a few hours.

In the meantime, if you'd like to close this, I'm perfectly fine with that - but would I be able to ask future questions here related to this topic?

Thanks.

from agorawebsdk-ng.