Magic Leap
State-of-the-Art Augmented Reality Devices
At Magic Leap, I work as a Senior Computer Vision Software Engineer/Researcher developing state-of-the-art algorithms for online calibration of the multitude of sensors on device. Skillsets used predominantly involve optimization algorithms, bundle adjustment, Visual SLAM and a strong grasp of geometrical computer vision and C++. Project details can be found on my resume.
Company
Magna | Lyft Level 5
L5 Autonomous Vehicles
At Magna/Lyft, I worked as a SLAM/Deep Learning Software Engineer working on multi-layered HD maps, deep learning algorithms for highly resource constrained systems as well as sensor fusion. Skillsets used predominantly involved deep learning, quantization and pruning, sensor fusion, Visual SLAM and a strong grasp of 3D geometry, C++ and Python. Project details can be found on my resume.
Press Release
NASA Astronet
A Human-Centric Network of Co-Robots
The AstroNet Simulator is developed as an extension to NASA's open-source Astrobee simulator. Coverage and navigation algorithms for each of the robots are developed in a grid-based environment. ROS Kinetic is used for simulating the algorithms, and to interface with the quadcopters (representing astrobee robots in our lab environment). An immersive VR-based environment of the whole International Space Station, with the robots in it, has been implemented and can be visualized on the Oculus Rift.
Link Demo
AUVSI RobotX
Autonomous Surface Vessel Competition
AUVSI's Robot-X competition is an attempt at automating surface vessels to revolutionize the marine industry. The robot should be capable of performing several tasks such as SLAM, Trajectory Planning, Obstacle Avoidance, etc. My contributions to the project are in SLAM(Velodyne HDL32, Ladybug 3 and GPS/IMU module used as sensors), Trajectory Planning, 3D Object Detection, and Network and Communication Setup. The whole framework is being developed on ROS Kinetic.
Link
Path Planning | Exploration
Multi-Agent exploration in Turtlebots
This project involves comparative analysis of state-of-the-art path planning and multi-agent exploration algorithms via simulation and experimentation. Simmulations are performed on 50+ fabricated grid maps of size 50 x 50 on a python based simulator. To validate the results, experiments with Turtlebots are also performed. Nodes for the studied path planning and exploration algorithms are written and 10 different laboratory environments are explored.
Link Demo
PoseNet++
Deep Learning in SLAM
An online-deep learning based-data labeling paradigm derived from structural motion is implemented. We directly label (6-DOF measurements) the input data (camera video feed) streamed from a mobile camera using this paradigm. This labeled data is then used to train PoseNet, which acts as the sensor model for Incremental Smoothing and Mapping (iSAM). GPS data from the used dataset's is used to build the action model. Georgia Tech's GTSAM is then used for simulation. The algorithm is also implemented on a differential drive mobile robot.
Link Paper
Visual-Radio-Inertial Localization
an Indoor Positioning System based on WiFi
This project involved the development of a localization system for truly mobile devices, such as smartphones. The main aim of the project is to provide a means of localization in indoor/GPS-denied environments. A deep neural network capable of differentiating between LOS/NLOS measurements has been architected. Simulations in MATLAB and Python have been developed for the 2D and 3D case. Experiments with the Fetch robot were performed with an RMSE of around 2m. The project was done under the guidance of Dr. Maani Ghaffari.
Link Paper
Formula SAE Italy
Formula Prototype Competition
SAE holds it's annual international competiton at different locations to pit the best teams from around the world against each other. Our team from BITS Pilani participated in FSAE Italy '14, and was ranked 14th out of 46 teams in vehicle design. My contributions to the team were focussed on the design and fabrication of a double wishbone push-rod suspension system for the Formula SAE prototype. The work I did included the design and optimization of the bell-crank geometry, design and fabrication of the push-rod and a-arm geometry, and the design of the wheel hub. I also worked on calculating the roll center migration, spring rates, roll rates, anti-squat/anti-dive and the suspension frequency for the car.
Link
Sense
Android app serving all its sensor data via web
The Sense app was developed with the sole purpose of serving a developer with his needs. While working on robots, I often faced the issue where I had to buy multiple expensive sensors in order to make it perceptive of it's environment. This app alleviates this need by streaming all the sensor information(GPS, IMU, Camera, etc) from your android smartphone to a web-server. A supporting ROS Package is also developed to make it easy for developers to directly subscribe to useful topics and use them without the headache of installing device drivers, etc.
Simultaneous Localization & Mapping
Particle Filter based SLAM on a differential drive robot
In this project, we develop a simple particle filter based SLAM algorithm on a hobby differential drive robot using a single channel Scanse Lidar. We implemented an occupancy gird based mapping mechanism, particle filter based localization algorithm and Yamauchi's autonomous exploration algorithm on a differential mobile robot running on a raspberry pi3 and beaglebone black. My contributions to the team included implementations of the action model, sensor model, the particle filter and the exploration algorithm. I also worked on the low-level control structure of the robot to minimize latency in the receiving and sending of commands.
Link
Pedestrian Tracking
Mask R-CNN based Pedestrian Tracking
Matterport's implementation of Mask R-CNN is used in this application and slightly modified to detect only pedestrians (to keep computational requirements low). On top of this, we use Gunner Farneback's algorithm to compute the dense optical flow in each of the successive video frames. Since no ground truth is known here (as we do track pedestrians live using a mono camera), we use the instance segmentation given to us by Mask R-CNN and compute the net motion for each of the segments. We then use a particle filter for tracking one of the segments.
Link
Swarm Mapping and Control
Aerial and Ground Robot Swarm
This project is an effort by TARDEC towards using swarm robotics for recconnaisance and surveillance. In this project, I worked on multi-robot mapping and resilient control where the swarm as a whole operates without any issue, even if some agents constituting the swarm are malicious. We used a swarm of heterogeneous robots: CrazyFiles and Aion R1 Rovers to provide both aerial and ground functionality. Dr. Dimitra Panagou guided us through the project. Video demonstrations of the algorithm in action, and Gazebo Simulations will be uploaded soon.
3D Bounding Box Regression
3D Vehicle Detection and Bounding Box Regression
This project involves 2 parts: (i) Image Classification , (ii) Vehicle Detection and 3D Bounding Box regression. The GTA 10k dataset was used for this project and a total of 41 teams had participated in this private competition on Kaggle. A custom 20 layer SE-ResNet was implemented in PyTorch for classifying each of the images into one of the 23 pre-determined classes. Once classified, 3D bounding boxes were regressed using YOLO and geometry as outlined in this work by A.Mousavian et al. We secured the 5th position in the 1st competition and 7th in the 2nd competition. Our classification accuracy was ~73% and the MSE for the centroid was ~9, whereas the same for the winning team was ~79% and ~4.3 respectively.
Unsupervised Learning - Augmented Reality
Unsupervised Learning based on GMM
This project presents an unsupervised learning algorithm by which an aerial robot can stream assistive camera views, which are unknown a priori, to a human whose attention is split between an arbitrary number of complex tasks. This is accomplished by tracking the human’s head motions during multitasking and then fitting this data to a visual interest function, modeled as a mixture of Gaussians, via online expectation maximization. This function informs a dynamic coverage controller which then directs the robot to patrol those regions most visually interesting to the human.
Link Demo Paper
Robot Kinematics Simulator
Javascript based Kinematic Simulator for Robotic Manipulators
The simulator was built as part of one of my courses here at Michigan. It is capable of interfacing with ROS over the web, giving anyone access to control a robot through any operating system. Given the URDF of a robot, the simulator is capable of automatically parsing the structure of the robot and calculating the forward (using matrix stack) and inverse kinematics (using cyclic co-ordinate descent). Features such as object following and trajectory planning using RRT-Connect and other algorithms is built into the simulator.
Link Demo
Game Physics - Pendulum Simulations
Javascript based Simulator for Single, Double and Cart Pole Pendulums
One of the most trivial control problems is that of a pendulum. In this project, a simple PID control structure was written for controlling the pendulum. Some of the prevalent integrators were implemented for the physics of the simulation : Euler Integrator, Verlet Integrator, Velocity Verlet Integrator and the Runge Kutta (RK4) integrator.
Demo-1 Demo-2 Demo-3
Autonomous Balance Bot
Self Balancing two-wheeled bot
This project involved us building an autonomous balancebot from scratch. Tasks involved the mechanical design of the bot, the hardware design of the circuit board and finally the code base running the robot. The robot used a cascaded PID controller to maintain it's balance while navigating through obstacle courses with the help of wheel odometry and motion capture providing positional feedback.
Demo
Vision Based Autonomous Robotic Arm
Autonomous Robotic Arm controlled via visual cues
The aim of this project was to automate redundant tasks using a robotic arm. The equipment we were provided with were a Microsoft Kinect and a dynamixel arm with four degrees of freedom - RRR:R. Our task was to design an efficiemt gripping mechanism for the robot and detect different colored cubes using depth maps and images streamed from the overhead Kinect. Once done, we were to design a Spline -based trajectory algorithm for smooth motion of the arm and complete 5 assigned tasks whose details were predefined.
Gesture Controlled Robotic Arm
Gesture Controlled Robotic Arm with Arduino and XBee
A gesture controlled robotic arm was designed and fabricated under the guidance of Dr. R.K.Mittal. Arduino-UNOs (one on the hand and the other controlling the robot) as our micro-controllers, 2 Inertial Measurement Units- MPU 6050 (to sense hand gestures and provide feedback) and the XBee for wireless communication. An application in Processing was also developed so that the arm could be manually controlled via sliders, instead of the IMU, when desired. The arm was capable of augmenting human capabilities with a reachable spherical workspace of radius 2m and payload capacity of 500gm.
Portable White Board Cleaner
Portable White Board Cleaner Concept with Arduino
This project focussed on the conceptualization, design and fabrication of a state-of-the-art whiteboard cleaner which, unlike conventional ones, was portable and inexpensive. The design was based on a simple RR:PR manipulator structure and was capable of cleaning boards of sizes upto 4' x 6' autonomously on the click of a button. The prototype was powered by an Arduino UNO and 4 servo motors. The results of this research was presented in the IEEE UPCON'15 conference at IIIT, Allahabad.