Transforming three-dimensional model by using correlation to template model with template skeleton

12106,436

17/819242

August 11, 2022

A method and apparatus for transforming an input model according to an example embodiment are disclosed. The method includes receiving an input model including a first mesh with a first topology, generating a template model including a second mesh with a second topology, aligning the template model with the input model, generating correlation information between the second mesh of the aligned template model and the first mesh of the input model, generating, based on a skeleton of the aligned template model, a skeleton of the input model, generating, based on the correlation information, first skinning information indicating a connection relation between the skeleton of the input model and the first mesh, and transforming, based on at least one of the correlation information and the first skinning information, the input model.

CLO Virtual Fashion Inc. (Seoul, KR)

CLO Virtual Fashion Inc. (Seoul, KR)

1. A method for transforming an input model, the method comprising: receiving an input model including a first mesh with a first topology, wherein the first mesh includes first vertices of the first polygon constituting the first mesh; receiving a template model including a second mesh with a second topology, the template model including a template skeleton; aligning the template model with the input model to generated an aligned version of the second mesh and an aligned version of the template skeleton; determining correlation between the aligned version of the second mesh and the first mesh; determining a model skeleton of the input model based on the aligned version of the template skeleton; generating first skinning information indicating relationships between the model skeleton and the first mesh according to the correlation; and generating a transformed version of the input model using the correlation and the first skinning information, wherein determining the correlation comprises: generating first barycentric coordinates by projecting second vertices of a second polygon included in the second mesh of the template model to the first polygon included in the first mesh of the input model; extracting, from the first barycentric coordinates, a plurality of parameters representing three-dimensional (3D) positions of an index of the first polygon included in the first mesh and the second vertices of the second polygon matching the first polygon; and changing the second mesh of the template model to the first mesh with the first topology of the input model using the parameters extracted from the first barycentric coordinates.

2. The method of claim 1 , wherein the aligning of the template model with the input model comprises adjusting sizes and positions of polygons in the second mesh such that an outer shape of the template model better matches an outer shape of the input model.

3. The method of claim 1 , wherein the aligning of the template model with the input model comprises: globally aligning the template model with the input model by modifying at least one of a position, a length, and an angle of each body part of the template model with at least one of a position, a length, and an angle of each body part of the input model; and locally aligning the template model with the input model by finely adjusting each body part of the template model with each body part of the input model.

4. The method of claim 3 , further comprising displaying at least one the global aligning of the template or the local aligning of the template.

5. The method of claim 1 , wherein determining the correlation comprises generating second barycentric coordinates by projecting vertices of the first in the first mesh to the second polygon in the second mesh.

6. The method of claim 1 , wherein rigging information of the template model is used to determine the model skeleton.

7. The method of claim 1 , wherein the generating of the first skinning information comprises combining the determined correlation and second skinning information indicating relations between the aligned version of the template skeleton and the second mesh.

8. The method of claim 7 , wherein the first skinning information indicates weights assigned to bones of the skeleton of the input model in deforming or transforming polygons of the input model.

9. The method of claim 8 , wherein generating the transformed version of the input model comprises: receiving an instruction for controlling at least one bone of the aligned version of the template skeleton; and deforming or transforming the polygons of the input model by applying the weights to at least a bone of the model skeleton corresponding to the at least one bone of the aligned version of the template skeleton that are controlled by the instruction.

10. The method of claim 1 , further comprising rendering texture of the transformed version of input model.

11. The method of claim 1 , wherein the input model comprises at least one of a three-dimensional (3D) avatar, a 3D virtual object, and a 3D character.

12. A non-transitory computer-readable storage medium storing instructions thereon, the instructions when executed by a processor cause the processor to: receive an input model including a first mesh with a first topology, wherein the first mesh includes first vertices of the first polygon constituting the first mesh; receive a template model including a second mesh with a second topology, the template model including a template skeleton; align the template model with the input model to generated an aligned version of the second mesh and an aligned version of the template skeleton; determine correlation between the aligned version of the second mesh and the first mesh; determine a model skeleton of the input model based on the aligned version of the template skeleton; generate first skinning information indicating relationships between the model skeleton and the first mesh according to the correlation; and generate a transformed version of the input model using the correlation and the first skinning information, wherein the instructions to determine the correlation comprises instruction to: generate first barycentric coordinates by projecting second vertices of a second polygon included in the second mesh of the template model to the first polygon included in the first mesh of the input model; extract, from the first barycentric coordinates, a plurality of parameters representing three-dimensional (3D) positions of an index of the first polygon included in the first mesh and the second vertices of the second polygon matching the first polygon; and change the second mesh of the template model to the first mesh with the first topology of the input model using the parameters extracted from the first barycentric coordinates.

13. The non-transitory computer-readable storage medium of claim 12 , wherein instructions to align the template model with the input model comprises instructions to adjust sizes and positions of polygons in the second mesh such that an outer shape of the template model better matches an outer shape of the input model.

14. The non-transitory computer-readable storage medium of claim 12 , wherein the instructions to align the template model with the input model comprises instructions to: globally align the template model with the input model by modifying at least one of a position, a length, and an angle of each body part of the template model with at least one of a position, a length, and an angle of each body part of the input model; and locally align the template model with the input model by finely adjusting each body part of the template model with each body part of the input model.

15. The non-transitory computer-readable storage medium of claim 14 , further storing instructions to display at least one the global aligning of the template or the local aligning of the template.

16. The non-transitory computer-readable storage medium of claim 12 , wherein instruction to determine the correlation comprises instruction to generate second barycentric coordinates by projecting vertices of the first in the first mesh to the second polygon in the second mesh.

17. The non-transitory computer-readable storage medium of claim 12 , wherein rigging information of the template model is used to determine the model skeleton.

18. The non-transitory computer-readable storage medium of claim 12 , wherein instructions to generate the first skinning information comprises instructions to combine the determined correlation and second skinning information indicating relations between the aligned version of the template skeleton and the second mesh.

19. The non-transitory computer-readable storage medium of claim 12 , wherein the first skinning information indicates weights assigned to bones of the skeleton of the input model in deforming or transforming polygons of the input model.

20. The non-transitory computer-readable storage medium of claim 19 , wherein instruction to generate the transformed version of the input model comprises instruction to: receive an instruction for controlling at least one bone of the aligned version of the template skeleton; and deform or transforming the polygons of the input model by applying the weights to at least a bone of the model skeleton corresponding to the at least one bone of the aligned version of the template skeleton that are controlled by the instruction.

21. A computing device, comprising: a processor; and memory coupled to the processor, the memory storing instructions to: receive an input model including a first mesh with a first topology, wherein the first mesh includes first vertices of the first polygon constituting the first mesh; receive a template model including a second mesh with a second topology, the template model including a template skeleton; align the template model with the input model to generated an aligned version of the second mesh and an aligned version of the template skeleton; determine correlation between the aligned version of the second mesh and the first mesh; determine a model skeleton of the input model based on the aligned version of the template skeleton; generate first skinning information indicating relationships between the model skeleton and the first mesh according to the correlation; and generate a transformed version of the input model using the correlation and the first skinning information, wherein the instructions to determine the correlation comprises instruction to: generate first barycentric coordinates by projecting second vertices of a second polygon included in the second mesh of the template model to the first polygon included in the first mesh of the input model; extract, from the first barycentric coordinates, a plurality of parameters representing three-dimensional (3D) positions of an index of the first polygon included in the first mesh and the second vertices of the second polygon matching the first polygon; and change the second mesh of the template model to the first mesh with the first topology of the input model using the parameters extracted from the first barycentric coordinates.

10628666 April 2020 Sareen
20150178988 June 2015 Montserrat Mora

Feng, Andrew, Dan Casas, and Ari Shapiro. “Avatar reshaping and automatic rigging using a deformable model.” Proceedings of the 8th ACM SIGGRAPH Conference on Motion in Games. 2015. (Year: 2015). cited by examiner
Anguelov, Dragomir, et al. “Scape: shape completion and animation of people.” ACM SIGGRAPH 2005 Papers. 2005. 408-416. (Year: 2005). cited by examiner

edspgr.12106436