Shivaditya Shivganesh

This week was all about designing the static template for the project in VR. As displayed in the mockup of the environment, I have created 4 panes namely: surgery, vitals, radiography and chats. Each of this pane has a custom component designed in blender the actual processing will be handled

Video Server OpenCV pipeline for both segmentation and object detection was completed. The Mock HMD VR designed using Unity XR refers to a remote location on the server for getting video output. Apart from the server design. documentation was completed for WebXR approach. For UnityXR based method few sections of

Prototype for immersive was designed in Unity using inbuilt XR plugins. For HMD emulation MockHMD was used. The material was rendered on the plane. The material was linked to 2d texture. The VideoPlayer asset in Unity helped in converting the frames to textures. The camera was bounded to the camera

The video object detection pipeline is designed using openCV. The videos are loaded and a frame rate of 25FPS is assumed. Using the frames the tf.session is initiated and TF Object detection API model is run. The resulting bounding boxes are drawn using openCV. Resulting images with bounding boxes

The main focus for this week was correcting the errors in the bounding box drawn. The bounding boxes differed from the ground truth. That was because of issue regarding the canvas context drawing setting and not related to the actual model. After the correcting the drawing steps in the canvas

The main target for this week was POC development. I was successful able to develop a POC with help of HTML5 Canvas and WebXR. The total inference time is about 25 seconds on a Snapdragon 870. The previous week's model had inherent Frozen-Graph Operations issue because of which is did

This week was an eventful one. I started the week with TF Object Detection model development. I trained two models one was Faster RCNN and another one was MobileNet SSD. I plan to train YOLO models too and convert them into ONNX format for JS inference. For VR the bounding

Basic POC (Proof of Concept) design was completed and successfully implemented using WebXR and react. A JSON file was created which stores the links and names of all image file present in the directory. The BiT model was converted using TensorflowJS model converter. The model then generated was used in

The previous model trained was not performing well on the the testing dataset. So, I had to try a new approach for increasing the model accuracy as well as other parameters. When I tried training with other pre trained networks, after one epoch model started to overfit. So, I had

This week was all about model building and model validation. Initially, I was stuck a bit on the metric portion, but was later able to overcome it by using special keras libraries. The labyrinthine keras and tensorflow documentation proved to be invaluable. Model Development Dataset The dataset has labelled-images, unlabelled-images,