Recently, the "Number One Player" that has been raging on various social platforms must have been seen. Aside from the large number of colorful eggs in the film, the immersive VR technology that is displayed in the film is also very attractive. However, in addition to VR technology, 3D body scan technology may also achieve similar results in an alternative way.
3D human body scanning technology is the use of optical measurement technology, computer technology, image processing technology, digital signal processing technology, etc. to perform three-dimensional simulation of the human body surface contour non-contact automatic measurement. The application of this technology is not uncommon in real life. Some time ago, both Apple's Animoji and Samsung's AR emoji were derived from this technology. This technology can also be applied to clothing trials. Even gyms rely on this technique to measure various health-related key figures (body mass index, body weight). Fat ingredients, etc.) to attract customers.
However, the existing solution has a drawback. Some scanning devices require special camera equipment to detect the depth of field, while others require a camera matrix to detect the human body, and more recently, the Braunschweig University of Technology and the Max Planck Institute for Informatics. The researchers published a paper and proposed a new algorithm that can use a single-angle standard video material to create a 3D model for the human body in just a few seconds.
This paper describes how to do accurate 3D modeling for moving humans in monocular video. Based on a parametric human model, the paper proposes a robust process flow, and a 3D model with a resolution of approximately 5mm can be obtained for the wearer. In addition, the paper shows the results of the evaluation of a large number of objects and analyzes the overall performance. It can be said that only one smart phone or camera is needed. This method will allow everyone to create their own fully animated digital twins and apply it to virtual VR in social VR or online fashion shopping.
Figure 1: The technology presented in this paper is the first to extract accurate 3D human models from a single video sequence of a person, including hair and clothing. These people move in front of the camera to ensure that we can see them from all angles (picture from thesis).
Specifically, the method presented in the paper consists of three steps: (1) posture reconstruction; (2) uniform morphology assessment; (3) frame refinement and texture map generation. The first step is basically the use of previous research. The details of texture acquisition and time-varying in the third step are optional.
In the first step, the paper uses standard visual shell methods to capture static shapes from multiple perspectives. The camera light passing through the contour points of the image can constrain a 3D human shape. In this paper, the researchers first fit the SMPL model and 2D detection to estimate the initial human shape and 3D pose for each frame. Based on the fitting results, the researcher combines the contour points in each frame with the 3D points in the human model, transforms the projection ray according to the inverse deformation model of the corresponding 3D model point for each projection ray, and handles all the frames. After the result, a "visual shell" is obtained, that is, the shape of the human body displayed in a standard posture.
Research results on multiple 3D datasets show that the proposed method can reconstruct 3D human shapes, and the accuracy is still as high as 4.5 mm even when the monocular video is very blurred.
In order to evaluate the consistent shape of the object, the researchers first calculated the 3D pose of each frame. They extend the method of evaluation to make it more robust to maximize the outline in each frame instance. During the evaluation process, due to changes in the time, the clothes may have slight deformation and deformation. The uniform shape of these postures may have a slight mismatch with the frame contour.
When estimating geometric shapes in a video sequence, researchers need to optimize multiple poses at one time. More importantly, multiple models need to be stored in memory during the optimization process, making the calculation costly and not feasible.
Researchers use SMPL to efficiently optimize a single displacement surface, fitting all frames at once, requiring only one model to be stored in memory. Their method for the first time enabled accurate 3D human models (including hair and clothing) to be extracted from a single moving human video (so we can see this person from all sides).
Finally, in order to calculate the details of the evolution of the texture and the capture time, the deviation of the uniform pattern is optimized for each frame in the third step, thereby giving a refined frame-by-frame pattern to calculate the texture map. The method in this article depends on the foreground segmentation of the image. Therefore, the researchers used CNN-based video segmentation methods and trained each sequence with 3-4 manual segmentation images. In addition, in order to overcome the problem of ambiguity in 3D human morphological reconstruction when monocular, the researchers use the SMPL body model as a starting point.
Figure 7: Qualitative Test Results: Because the reconstructed template has the same topology as the SMPL manikin, we can use SMPL to change the pose and shape of the reconstructed body. Although the SMPL does not model the deformation of the garment, the deformed template still looks credible, and it is sufficient for a variety of applications in terms of quality.
Figure 9: Comparison of input image (vertical line left) and build image (vertical line right). As shown on the right side of the image, the shape constructed using this method accurately covers the input image. The side view of the building model is also presented in the lower right corner.
We have already mentioned a technique that can identify the whole body (Facebook new research lays the foundation for future full-body AR). It is different from the technology discussed this time. Facebook's new research identified the human body as 2D. Not 3D, but there are also connections between them, for example both use CNN-based video segmentation methods to identify human contours and textures. In fact, the paper also shows that the method of identifying textures is not innovative.
The new human body 3D scanning modeling technique proposed in this paper can use only one camera, so it can be applied to a large number of smart phones. In addition, it can not only be applied to the trial of clothing, but also through the scanning of the human body, so that the established model interacts with other virtual objects, and it can be regarded as an alternative to the "Number One Player" standard. Of course, the latter will have high computing power for the processor and may not have a better sense of immersion. Students interested in the paper can also poke and read the original download.
Pharmaceuticals,2-Methyl- Propanoic Acid Monohydrate Price,2-Methyl- Propanoic Acid Monohydrate Free Sample,Pure 2-Methyl- Propanoic Acid Monohydrate
Zhejiang Wild Wind Pharmaceutical Co., Ltd. , https://www.wild-windchem.com