Collaboration on Augmented Reality – Summer in the Labs
How can 5G improve the user experience of augmented reality by leveraging technologies for audio and video delivery, recommender systems and 360 video? That has been in focus this summer for Bo, a University of Colorado Computer Science student doing an internship at Ericsson in Silicon Valley.
Students from many different fields join Ericsson Research for internships or thesis work. You could be next. Follow our blog to learn about the students at Ericsson Research this summer.
Hi Bo! What do you do at Ericsson Research?
Ericsson Research Media Technologies work on improving the quality of the media experience. This includes improving how audio and video is delivered and consumed. In Silicon Valley, work has been done in content discovery, recommender systems, and even 360 videos. Our team thinks up new research ideas and designs novel applications related to Human-Computer Interaction (HCI) and how 5G can leverage these technologies to improve user experiences.
Name: Bo Cao
Unit: Media Technology
Education: Computer Science at
University of Colorado Boulder
Location: Silicon Valley, USA
My teammate Jeeeun and I decided to work on something that has never been done before so we chose Collaboration on Augmented Reality (AR), in which learners can get help to finish tasks from a remote teacher. In this particular project, the user wears a HoloLens device and the teacher communicates via a modern web browser. Although the core idea is not new, as we dug into the literature review we were surprised at how many of the techniques behind it had not been attempted previously. Our task was to fulfill the gap between the implementation and the research idea. These challenging tasks involve HCI research into which factors matter to the learners, which computer graphics should be displayed, and how the learner and teacher perspectives should be transformed. We also explored web technology and system architecture, in terms of how the server and client should be implemented, how each component should be designed, and how they should communicate. In addition, we need to communicate with the HoloJS community to debug. After the development of this project, we will recruit participants for a user study.
What have you learned from Ericsson?
One of the most important things I have learned is how to prioritize tasks. We encountered some challenges during our development, from a higher level of team work to a lower level of technical details. Since we have twelve weeks to develop our application and run a user study, solving all these would take more than that. During the discussion, our mentors helped us to figure out what’s the most important and made a clear path for us to go.
As well as the technical skills I gained during the internship, such as web and HoloJS development, I have also learned how to think with a research perspective and how to solve problems in a deeper technical perspective with support from my two mentors, Alvin and Per-Erik. The issues in merging code in Github taught me a better way to collaborate in a group. The result is a web-based system, HoloLens application and user study. The report includes the current issues, the future work of this application, and how 5G can help solve these issues to enhance user experience.
What has been challenging during your internship?
One of the main challenges was to develop a HoloLens application on HoloJS, an open source framework that can be used to build on web technology. We encountered issues during the development since HoloJS had only been developed for six months since we began our own development. There were some issues when we developed our app on this framework, including communication between each component, where we implemented these functions ourselves. Another challenge in this framework was that the rendered objects in the HoloLens application were not displayed in the video stream on a web browser. I also faced the challenge of how to collaborate efficiently with other developers. Since we developed our application from different perspectives in various branches, there were issues when merging our code due to the different interfaces between each component. To address this, we had to communicate and figure out what needed to be changed before merging.
What education do you have?
I am a second-year master’s student in Computer Science at University of Colorado Boulder. Prior to that, I obtained my bachelor’s in Computer Science and Technology in China and worked for a while in Shenzhen, Guangdong. At the University of Colorado Boulder, I did research into Human-Robot Interaction (HRI) – mainly in the topic of Intuitive Gesture Interaction between humans and robots and Human Behavior Recognition from Image.
What do you do when you are not working or studying?
My main hobby is photography and exploring. You can see my pictures on 500px and my own website. I love exploring different places and expanding my horizons – not only in terms of natural landscapes but also different cultures. Part time I also participate in competitive programming contests as a hobby and follow up the development of AI technologies.
Facts about Silicon Valley
- Known as the capital of tech.
- In 2015 there were patent applications for 655 different products.
- The area was first dubbed ‘Silicon Valley’ in 1971 by American journalist, Donald C. Hoefler.
- The name “Silicon Valley” is derived from the large amount of silicon semiconductor manufacturing companies located in the region.