Google Summer of Code: Week 8

Prototype for immersive was designed in Unity using inbuilt XR plugins. For HMD emulation MockHMD was used. The material was rendered on the plane. The material was linked to 2d texture. The VideoPlayer asset in Unity helped in converting the frames to textures. The camera was bounded to the camera…

Google Summer of Code: Week 7

The video object detection pipeline is designed using openCV. The videos are loaded and a frame rate of 25FPS is assumed. Using the frames the tf.session is initiated and TF Object detection API model is run. The resulting bounding boxes are drawn using openCV. Resulting images with bounding boxes…

Google Summer of Code: Week 6

The main focus for this week was correcting the errors in the bounding box drawn. The bounding boxes differed from the ground truth. That was because of issue regarding the canvas context drawing setting and not related to the actual model. After the correcting the drawing steps in the canvas…

Google Summer of Code: Week 5

The main target for this week was POC development. I was successful able to develop a POC with help of HTML5 Canvas and WebXR. The total inference time is about 25 seconds on a Snapdragon 870. The previous week's model had inherent Frozen-Graph Operations issue because of which is did…

Google Summer of Code: Week 4

This week was an eventful one. I started the week with TF Object Detection model development. I trained two models one was Faster RCNN and another one was MobileNet SSD. I plan to train YOLO models too and convert them into ONNX format for JS inference. For VR the bounding…

Google Summer of Code: Week 3

Basic POC (Proof of Concept) design was completed and successfully implemented using WebXR and react. A JSON file was created which stores the links and names of all image file present in the directory. The BiT model was converted using TensorflowJS model converter. The model then generated was used in…

Google Summer of Code: Week 2

The previous model trained was not performing well on the the testing dataset. So, I had to try a new approach for increasing the model accuracy as well as other parameters. When I tried training with other pre trained networks, after one epoch model started to overfit. So, I had…

Google Summer of Code: Week 1

This week was all about model building and model validation. Initially, I was stuck a bit on the metric portion, but was later able to overcome it by using special keras libraries. The labyrinthine keras and tensorflow documentation proved to be invaluable. Model Development Dataset The dataset has labelled-images, unlabelled-images,…