mrdysbv / webar Goto Github PK

1. Chinese README

2. Chinese Source Code Analysis

This is a WeChat Web AR. On July 5, 2019, WeChat miniprogram supported AR. It was added a new API named "CameraFrameListener".

CameraFrameListener API

We can create AR effects with the new API. This demo demonstrates a AR tracker effect using "tracking.js" and "jsfeat" library.

The "tracking.js" brings computer vision algorithms and techniques into browser environment. The "jsfeat" is also a JavaScript computer vision library. We can do real-time image and face detection.

tracking.js and JSFeat

Index Page of the WeChat Mini-program

Use the demo to scan a pattern image below.

A cat beard is on the pattern image.

A effect of translating and scaling.

A effect of rotating.

This is the same as above. Supports image rotation.

The image is rotated by 30 degrees.

The image is rotated by 90 degrees.

Use the demo to scan a rotating image below.

A video player is on and aligned with the image.

Use the demo to scan a yellow color. Expect a effect below.

A effect of hiddening.

Custom the color of the highlighted area.

Use the demo to scan a face. Expect a effect below.

A effect of translating and scaling.

Because the landmarks of the demo are simple and basic, only a effect of translating and scaling is on a rotating image.

You may replace the default url of a image for 2D mask.

File: /package_image_tracker/pages/photo/photo.js and package_face_tracker/pages/photo/photo.js

// a url of sprite image
const modelUrl = '../../utils/cat_beard.png';

The width and height of the "modelurl" image should be 256 x 256, 512 x 512 and 1024 x 1024 etc.

File: /package_face_tracker/utils/imageBusiness.js

const patternImageUrl = '../../../face_pattern.jpg';

a pattern image

Select a track point on a pattern image, the point is used to set the "cat beard" image.

File: /package_image_tracker/utils/modelBusiness.js

// a index of a track point on a pattern image
const trackPoint = { 
    x: 185, // the width of the pattern image is 375
    y: 224, // the height of the pattern image is 375
};

This is a map of the 31 keypoints of a face.

landmarks

For example, a number 27 and number 29 are the sides of the mouth.

File: /package_face_tracker/utils/modelBusiness.js

// index of the track points of the face
const trackPointA = {
    // index of a landmark
    id: 27,
    // X coordinate
    x: 155.69898111309, // the width of the face image is 375
};
const trackPointB = {
    // index of a landmark
    id: 29,
    // X coordinate
    x: 216.53075265284997, // the width of the face image is 375
};

The AR demo is very slow on iOS WeChat.