+86-13420957505
jennyren@joneytech.com
Augmented reality (AR) technology is to integrate the computer-generated virtual environment with the display environment around users by means of photoelectric display technology, interactive technology, various sensor technologies and computer graphics and multimedia technologies, so that users can be sure that the virtual environment is an integral part of the real environment around them from the sensory effect. Augmented reality has the new characteristics of combination of reality and reality, real-time interaction and three-dimensional registration.
Augmented reality technology has three basic elements, namely, the fusion of virtual and real space, real-time online interaction and three-dimensional registration of virtual and real space. The fusion of virtual and real space emphasizes the coexistence of virtual elements and real elements, which is the key to enhance the user's perception of the real environment; Real-time online interaction emphasizes the real-time calculation of interactive response between users and virtual and real objects to meet the response needs of users' senses to the time dimension; The three-dimensional registration of virtual and real space emphasizes the accuracy and intelligence of users' perception of space, which embodies the temporal and spatial consistency of virtual and real integration. These three elements are the key to realize the enhanced perception of real environment. Because this enhanced perception is dependent on spatial orientation, augmented reality systems usually present the effect of virtual and real integration with the help of special equipment such as helmets.
In 1974, Myrib Krueger invented the Videoplace system, which can interact with the projection screen through the user's own silhouette.
In 1990, engineers Tom Caudell and David Mizell of Boeing Company developed a penetrating head-mounted display device, which allowed aircraft assemblers to check the assembly drawings of cables in real time, and formally put forward the term Augmentation Reality.
In 1993, Louis Rosenberg of the US Air Force Research Laboratory developed the first immersive remote augmented reality system Virtual Fixture. Feiner proposed the famous knowledge-driven augmented reality system KARMA.
In 1994, JulieMartin performed dancers and virtual scenes on the same stage in the stage play Dancing in Cyberspace.
In 1998, Sportsvision applied video enhancement technology in the live broadcast of NFL League in the United States.
In 2000, Hirokazu Kato developed ARToolkit, which is called the first published computer augmented reality library in the world. The emergence of ARToolkit has enabled many programmers to have simple and easy-to-use augmented reality development tools, which has effectively promoted the application and popularization of augmented reality technology.
In 2017, Apple and Google successively embedded the augmented reality system into their respective mobile terminals, and launched the augmented reality software development platforms ARKit and ARCore respectively. China's Shangtang Technology, Netease and other companies also subsequently launched the independent augmented reality software development platform SenseAR and Insight AR. Augmented reality has gradually become a new technology to serve the society and the public.
Wearable devices generally include head-mounted displays, projectors, AR glasses, contact lenses, etc. The most widely used is head-mounted displays. According to the display model of real environment, augmented reality helmet display can be divided into optical penetrating helmet display and video penetrating helmet display. Through the transparent lens at the front end of the user's glasses, the optical penetrating helmet reflects the light from the real world at the same time, and reflects the virtual scene into the human eye, thus forming a scene of virtual and real integration. The video penetrating helmet captures the influence of the scene in real time through the binocular camera at the front end of the eyes, and superimposes the virtual scene on the video screen and presents it in the binocular display configured at the front end of the eyes of the user.
Mobile handheld display devices, such as smart phones, tablet computers and other mobile terminals with cameras and certain computing and drawing capabilities, use the built-in camera to capture the real world, and then merge with the virtual world drawn by themselves and present it to users.
Spatial augmented reality technology directly projects the virtual world into the real world by using optical principles or special equipment. Generally, it projects the virtual image into a fixed space, such as projecting the image on an object or forming a virtual image into a three-dimensional space.
In addition to some visual display devices, auditory devices have also gradually become one of the research contents of augmented reality. Through audio signal processing technology, the Virtual Auditory Space studied by teacher Chen Fei of south university of science and technology of china will be simulated by acoustic correlation algorithm based on some special acoustic effects of human ears, and the sound source will be reconstructed in any position in 3D space, so as to realize the reproduction of sound source orientation and produce realistic virtual auditory perception.
At the same time, some augmented reality is equipped with tactile devices, such as the vibration feeling generated by the mechanical actuator of an eccentric rotating mass (ERM) vibration motor. Eccentric rotating mass motors have different sizes and vibration force specifications, thus producing different tactile sensations. At the same time, mechanical stimulation can make the human sensory receptors transmit small electrical pulses to the central nervous system along the afferent nerve. Such as TeslaSuit DK1 system developed by Tesla Studios (St Andrew, Scotland).
Three-dimensional registration is the key technology to solve how to connect virtual scenes with real scenes. Three-dimensional registration technology binds the virtual scene to the coordinate system of the real scene. With the movement of users and the change of visual angle, the projection information of the virtual scene at this visual angle is calculated and fused to the image of the real scene, ensuring that the virtual scene and the real scene share the same space. When they are relatively static, the positional relationship and scale relationship between the virtual scene and the real scene are consistent with the change of camera pose; When they move relatively, it is necessary to accurately solve the geometric information of the three-dimensional moving scene and the motion trajectory of the camera with the help of three-dimensional registration technology. Because human eyes are very sensitive to the perception of images, if the 3D registration results are not accurate enough, the images presented to users will shake and the scene will drift, which will seriously affect the user's immersion experience.
(2) Realistic rendering
On the basis of three-dimensional registration technology, it is also necessary to visually present and integrate virtual scenes or information. In augmented reality system, there are generally two kinds of visual fusion presentation methods. One is to superimpose characters, numbers or other information on the real scene in an appropriate visual way to realize the active push and efficient presentation of knowledge; The other emphasizes the real existence of virtual scenes in the real environment, and draws realistic virtual scenes under uniform lighting conditions. Generally, the global lighting model is adopted to make the high-fidelity simulated light transmit in the virtual environment, which is relatively time-consuming. With the continuous improvement of CPU and GPU performance, with some sampling approximation and sum calculation methods, real-time global lighting rendering of virtual scenes or scenes is gradually possible.
(3) Human-computer interaction
The goal of human-computer interaction is to take users as the center, use various devices or interfaces, interact with virtual or real targets, and complete certain interaction tasks through information exchange. Augmented reality system allows users to interact in the three-dimensional space of virtual and real integration, and complete certain interaction tasks through information exchange. Augmented reality system allows users to interact in the three-dimensional space of virtual and real integration, and operate virtual objects by connecting other devices. Therefore, the involvement of interactive system should avoid confusion in the process of information registration or presentation, so as to enhance the interactive experience of users.
Because there is no single and fixed input and output mode in the interaction of augmented reality, and it is not the output of two-dimensional images on the display, it is difficult to have a fixed and universal interaction design. Augmented reality mainly adopts three-dimensional interaction, which has higher freedom, more interaction modes, bigger interaction tasks and more complicated three-dimensional user interface definitions. At present, the interaction mainly includes (1) touch interaction (2) gesture interaction (3) voice interaction (4) physical interaction.
Summary
Augmented reality is a technology that combines virtual scenery or information with the real physical environment and presents it to users interactively, thus creating a technology that virtual and reality share the same space. In essence, augmented reality is a new interface technology that integrates positioning, presentation, interaction and other software and hardware technologies. Its purpose is to make users feel the temporal and spatial correlation and integration of virtual and real space in their senses, so as to enhance their perception and cognition of the real environment.Augmented reality technology has three basic elements, namely, the fusion of virtual and real space, real-time online interaction and three-dimensional registration of virtual and real space. The fusion of virtual and real space emphasizes the coexistence of virtual elements and real elements, which is the key to enhance the user's perception of the real environment; Real-time online interaction emphasizes the real-time calculation of interactive response between users and virtual and real objects to meet the response needs of users' senses to the time dimension; The three-dimensional registration of virtual and real space emphasizes the accuracy and intelligence of users' perception of space, which embodies the temporal and spatial consistency of virtual and real integration. These three elements are the key to realize the enhanced perception of real environment. Because this enhanced perception is dependent on spatial orientation, augmented reality systems usually present the effect of virtual and real integration with the help of special equipment such as helmets.
Historical development
The development of augmented reality technology can not be separated from the research and development of virtual and real fusion presentation device, and its history can be traced back to the 1960s. In 1968, Ivan Sutherland, the father of computer graphics, put forward the idea of The UltimateDisplay, and designed and implemented the first head-mounted augmented reality display, Sword of Damocles. The transmission display realized by this facility can only be superimposed with a simple wireframe model in front of the field of view.In 1974, Myrib Krueger invented the Videoplace system, which can interact with the projection screen through the user's own silhouette.
In 1990, engineers Tom Caudell and David Mizell of Boeing Company developed a penetrating head-mounted display device, which allowed aircraft assemblers to check the assembly drawings of cables in real time, and formally put forward the term Augmentation Reality.
In 1993, Louis Rosenberg of the US Air Force Research Laboratory developed the first immersive remote augmented reality system Virtual Fixture. Feiner proposed the famous knowledge-driven augmented reality system KARMA.
In 1994, JulieMartin performed dancers and virtual scenes on the same stage in the stage play Dancing in Cyberspace.
In 1998, Sportsvision applied video enhancement technology in the live broadcast of NFL League in the United States.
In 2000, Hirokazu Kato developed ARToolkit, which is called the first published computer augmented reality library in the world. The emergence of ARToolkit has enabled many programmers to have simple and easy-to-use augmented reality development tools, which has effectively promoted the application and popularization of augmented reality technology.
In 2017, Apple and Google successively embedded the augmented reality system into their respective mobile terminals, and launched the augmented reality software development platforms ARKit and ARCore respectively. China's Shangtang Technology, Netease and other companies also subsequently launched the independent augmented reality software development platform SenseAR and Insight AR. Augmented reality has gradually become a new technology to serve the society and the public.
Fundamental principle
Augmented reality needs to complete the fusion of virtual and real vision in three-dimensional space, so it needs special display devices to meet the needs of visual fusion perception. At present, the display devices of augmented reality can be divided into three categories according to their usage methods: wearable devices, mobile handheld display devices and space enhancement devices.Wearable devices generally include head-mounted displays, projectors, AR glasses, contact lenses, etc. The most widely used is head-mounted displays. According to the display model of real environment, augmented reality helmet display can be divided into optical penetrating helmet display and video penetrating helmet display. Through the transparent lens at the front end of the user's glasses, the optical penetrating helmet reflects the light from the real world at the same time, and reflects the virtual scene into the human eye, thus forming a scene of virtual and real integration. The video penetrating helmet captures the influence of the scene in real time through the binocular camera at the front end of the eyes, and superimposes the virtual scene on the video screen and presents it in the binocular display configured at the front end of the eyes of the user.
Mobile handheld display devices, such as smart phones, tablet computers and other mobile terminals with cameras and certain computing and drawing capabilities, use the built-in camera to capture the real world, and then merge with the virtual world drawn by themselves and present it to users.
Spatial augmented reality technology directly projects the virtual world into the real world by using optical principles or special equipment. Generally, it projects the virtual image into a fixed space, such as projecting the image on an object or forming a virtual image into a three-dimensional space.
In addition to some visual display devices, auditory devices have also gradually become one of the research contents of augmented reality. Through audio signal processing technology, the Virtual Auditory Space studied by teacher Chen Fei of south university of science and technology of china will be simulated by acoustic correlation algorithm based on some special acoustic effects of human ears, and the sound source will be reconstructed in any position in 3D space, so as to realize the reproduction of sound source orientation and produce realistic virtual auditory perception.
At the same time, some augmented reality is equipped with tactile devices, such as the vibration feeling generated by the mechanical actuator of an eccentric rotating mass (ERM) vibration motor. Eccentric rotating mass motors have different sizes and vibration force specifications, thus producing different tactile sensations. At the same time, mechanical stimulation can make the human sensory receptors transmit small electrical pulses to the central nervous system along the afferent nerve. Such as TeslaSuit DK1 system developed by Tesla Studios (St Andrew, Scotland).
Technical characteristics
(1) 3D registration technologyThree-dimensional registration is the key technology to solve how to connect virtual scenes with real scenes. Three-dimensional registration technology binds the virtual scene to the coordinate system of the real scene. With the movement of users and the change of visual angle, the projection information of the virtual scene at this visual angle is calculated and fused to the image of the real scene, ensuring that the virtual scene and the real scene share the same space. When they are relatively static, the positional relationship and scale relationship between the virtual scene and the real scene are consistent with the change of camera pose; When they move relatively, it is necessary to accurately solve the geometric information of the three-dimensional moving scene and the motion trajectory of the camera with the help of three-dimensional registration technology. Because human eyes are very sensitive to the perception of images, if the 3D registration results are not accurate enough, the images presented to users will shake and the scene will drift, which will seriously affect the user's immersion experience.
(2) Realistic rendering
On the basis of three-dimensional registration technology, it is also necessary to visually present and integrate virtual scenes or information. In augmented reality system, there are generally two kinds of visual fusion presentation methods. One is to superimpose characters, numbers or other information on the real scene in an appropriate visual way to realize the active push and efficient presentation of knowledge; The other emphasizes the real existence of virtual scenes in the real environment, and draws realistic virtual scenes under uniform lighting conditions. Generally, the global lighting model is adopted to make the high-fidelity simulated light transmit in the virtual environment, which is relatively time-consuming. With the continuous improvement of CPU and GPU performance, with some sampling approximation and sum calculation methods, real-time global lighting rendering of virtual scenes or scenes is gradually possible.
(3) Human-computer interaction
The goal of human-computer interaction is to take users as the center, use various devices or interfaces, interact with virtual or real targets, and complete certain interaction tasks through information exchange. Augmented reality system allows users to interact in the three-dimensional space of virtual and real integration, and complete certain interaction tasks through information exchange. Augmented reality system allows users to interact in the three-dimensional space of virtual and real integration, and operate virtual objects by connecting other devices. Therefore, the involvement of interactive system should avoid confusion in the process of information registration or presentation, so as to enhance the interactive experience of users.
Because there is no single and fixed input and output mode in the interaction of augmented reality, and it is not the output of two-dimensional images on the display, it is difficult to have a fixed and universal interaction design. Augmented reality mainly adopts three-dimensional interaction, which has higher freedom, more interaction modes, bigger interaction tasks and more complicated three-dimensional user interface definitions. At present, the interaction mainly includes (1) touch interaction (2) gesture interaction (3) voice interaction (4) physical interaction.
