machine learning, ai, cs

Notes on ICRA 2018

Here are my notes from my first ever visit to a robotics conference — ICRA 2018. Coming from a machine learning background, many things were very novel to me, but it was interesting to note the influence machine learning is having on the field. Neural networks were mentioned in every second talk, and while there is a general skepticism about DNNs being the ultimate solution, people like the thought of robots learning their behaviors.

Workshops

Machine Learning in Planning and Control of Robot Motion

keynote Robot Planning and Learning under Uncertainty: Data, Models, and Actions

Advocates for simultaneous learning of model and policy. QMDP-Net.

talk Integrating Algorithmic Planning and Deep Learning for Partially Observable Navigation

As AlphaGo combined planning and learning, the same idea should be applied to robotics to combine learning and planning. Visual navigation under partial observability.

keynote Machine-Learning Challenges in Planning for Hierarchically Organized Systems

Bremen harbor manages 6 million cars per year. Management of the port is a highly hierarchical task.

talk Deep Neural Networks for Swept Volume Prediction Between Configurations

The usual way of motion planning is to check all points along the trajectory. Another way is to compute the swept volume and then see if the volume intersects with objects. Swept volume is a volume that incorporates all points in space the robot occupies during motion. Swept volume is a better metric for motion planning than C-space Euclidean distance. Exact swept volume is intractable, so they learn an approximator (deep net: from start and end configuration to volume; 3 FC layers, 100k samples), which works well and is fast. To use for sampling-based motion planning.

talk Visual Robot Task Planning

Next state prediction from a high-level representation of the current state and the action the agent is about to take. Presented their DNN architecture to do it. PyBullet + UR5 model as the environment. Using prediction of future states in MCTS for planning.

talk Multi-Arm Self-Collision Avoidance: A Sparse Solution for a Big Data Problem

Sample poses in simulation to build a 5-million-point dataset and train an SVM to predict {collision, no collision}. Works in the real world (with KUKA arms) — if a collision-prone situation is near, the arms try to modify their paths. The SVM predicts every 2 ms.

keynote The Right Tool for the Job: About Swiss Army Knives, Hammers, Motion Planning, Control, and Machine Learning by Oliver Brock

Specific tools vs. general approaches: tools should be specific if they can. POMDP and DNN replicate solving problems that are already solved; the motivation is the promise of a general solution, but… maybe “what works well” should be the driver when choosing a tool. Motion planning: problem definition is different depending on the task, control, uncertainty level, perception. Idea of Expected Utility Sampling. Deep Learning with algorithmic priors: combine data-driven methods with algorithmic priors, no reason to get stuck on one particular side. Differentiable Particle Filter for localization.

talk Sample-efficient Reinforcement Learning via Difference Models

Small humanoid robot learning to walk via RL. Exploration via random noise breaks it in 5 minutes (with a pre-programmed controller it breaks in 8 hours). Damage is unavoidable. On real robots those failures prevent us from using RL in the real world. What to do? The first step is to learn in simulation; however, even for just walking, the differences are too big. Let’s learn the difference between simulation and the real world! They use DDPG. Learn a model in the simulator -> use the real robot and run the policy to collect training data -> use this data to train difference models -> learn a new policy that accounts for the difference in dynamics. The difference between sim and real is postulated as M:(statektrue,actionktrue)→(statek+1true−statek+1sim)M: (state_{k}^{true}, action_{k}^{true}) \rightarrow (state_{k+1}^{true} - state_{k+1}^{sim}).

Workshop on Informative Path Planning and Adaptive Sampling

talk Data-driven Planning via Imitation Learning by Sanjiban Choudhury

Real-world motion planning problems. He worked on helicopters and quadrotors, has lots and lots (700 h) of flying data. Usual planning algorithms spend time checking the paths that are clearly not suitable. An adaptive planner is needed. Adaptation can be inside or outside the planner. Outside: for example, an ensemble of planners — a quadrotor flew 16 km autonomously avoiding obstacles. Inside: learning a heuristic policy.

A quadrotor with the task to maximize exploration to map an indoor space. Applied to inspection of power lines and underground tunnels.

Part 1: Search based planning

BFS tree expansion to find the goal node. Tries out all suitable expansion nodes. Can this be replaced by a policy that is smarter about picking the next expansion node? Issues are: requires a lot of online effort, sample inefficient. Imitation learning in the context of MDPs, three paradigms:

  1. Open loop demonstrations. Usual behavior cloning. Compounding error is the problem: the agent ends up in an unknown state.
  2. Corrective feedback. Adapt to an ever-changing goal. DAgger is an example.
  3. Cost-to-go feedback. AggreVaTe (Ross and Bagnell, 2014). Similar to DAgger, but uses rollouts to change loss functions (?). They work in this paradigm. AggreVaTe →\rightarrow SaIL (CoRL’17). Hallucinating oracle computes posterior over states. Compared to the A* approach, SaIL is 100x faster by rolling far fewer nodes to expand.

Part 2: Informative path planning

Nodes are sensing locations where the robot receives measurements. Each node has a usefulness measure (how much info about the world was revealed). The goal is to maximize the utility given the exploration budget. They formulate this again as a POMDP: the action is the next node to visit, the state is the list of visited nodes and the world, the reward is the utility.

talk Marine Robotics: Planning, Decision Making, and Human-Robot Learning

Human input to improve path planning. The usual way of operating underwater vehicles is to manually define waypoints for exploration, receive data, come up with new waypoints, etc. Can we add more autonomy and a better mechanism of interaction with humans?

Given an initial trajectory provided by a scientist, optimize it under constraints of deviation and risk functions.

It seems that from the scientist’s perspective, deviation from the initial trajectory can make the whole run pointless. In my mind it is better to invest time in being able to autonomously avoid risks, but still get the measurements the scientist wanted.

Long-Term Autonomy and Deployment of Intelligent Robots in the Real-World

talk Place recognition through image sequence matching

Image sequence matching can be formulated as a graph search problem.

keynote One Map to Rule Them All? by Paul Newman

One algorithm for mapping without GPS, works in forest, field, underground mine, airport. Very strong autonomous driving group, read more: http://ori.ox.ac.uk/how-robotcar-works, http://ori.ox.ac.uk/theme/localisation, http://ori.ox.ac.uk/theme/mapping, http://ori.ox.ac.uk/media.

talk Learning Scan Context toward Long-term LiDAR Localization

Localization as classification with CNNs. Each place has an index — class (does not scale?).

talk Probabilistic Observation Maps for Use in Long-Term Human-Robot Interactions

Cluttered environments.

talk Long-term Large-scale Mapping and Localization Using maplab

A new framework for long-term mapping and localization. Open source.

talk Long-Term Deployment of Self-Driving Cars and Trucks (Uber)

Self-driving trucks decrease the haul cost, increasing demand, creating more (support) jobs. Uber uses all sensors: LiDAR, radar, cameras; no sensor is perfect. Uber has rich simulations and real-world test tracks to test all possible scenarios, reuse collected data to update intelligence, etc. Systems to find out “interesting” situations in the data they collect: operators mark those, but also automatically: when the vehicle braked hard, differences between human driving, failed predictions on offline data. An important direction to work on is to estimate and use uncertainty. Multisensor seems to be more important than limiting to one, constrained case.

talk DejavuGAN: Multi-temporal Image Translation toward Long-term Robot Autonomy

Predictions of appearance changes. One unified network (GAN) to transform from one weather condition to another.

talk Efficient Map Management Scheme for LiDAR-based Vehicle Localization

Convert heavy point clouds to images for building maps from LiDAR scans. Image patches are enough for localization.

talk Long-Term Autonomy for Self-Driving Cars: Challenges and Opportunities by John Leonard

Life-long work on SLAM. In terms of deep learning, the difference between robotics and pure computer vision is that achieving 90% is still bad and cannot be applied in the field. Levels of autonomy are defined for self-driving , what about other areas? “Guardian” system at Toyota — detects when a human falls asleep or is distracted and takes over.

talk A Trusted Goal Reasoning and Planning Framework for Long Term Autonomy

Underwater vehicles with 6-month deployment times. Avoiding environment changes and failure modes (low battery).

talk Deploying Mobile Robots and Keeping them Autonomous by Joydeep Biswas

Their research is aimed at providing methods for non-technical users to adapt and correct robot behavior to deploy robots for long-term autonomy. https://amrl.cs.utexas.edu/. Human-in-the-Loop SLAM: https://www.joydeepb.com/Publications/aaai2018_hitl-slam.pdf.

talk Designing for Long-Term Autonomy: Experiences With Collaborative Robots by Jonathan Kelly

STARS Laboratory in Toronto. Mobile manipulator with UR-10. Touch sensor to build surface point cloud.

talk One Year of Autonomy in Everyday Environments: The STRANDS Project by Tomas Krajnik

4D mapping, where the additional dimension is time: a probabilistic model to predict whether an object (or a voxel) is on the map at any specific time. Nicely models periodicity in the environment: people being present at their work places, people at the cafeteria, etc.

EXPO

Farming robot Two nuclear detection submarine bots and a nuclear coffee thermos. All Australian robots look like spiders. Of course they do. Flying car. Nope, does not fly yet. LiDAR survey drone A more intuitive control interface

Day 1

keynote Robots and People: the Research Challenge by Rodney Brooks

iRobot, Rethink Robotics. The number of elderly people will be much greater. Urbanization. Climate change. P-values have to stop.

talk Learning Modes of Within-Hand Manipulation

Another realization of the idea of having a classifier that predicts an event (in this case whether an object will fall from a grip) from the robot state.

talk Optimization Beyond the Convolution: Generalizing Spatial Relations with End-To-End Metric Learning

Train a supervised network to understand which scenes are similar in terms of how two objects are positioned relative to each other. The network takes 3 input images: one is a reference, the second has positioning similar to the reference and the third is different from the reference. The system is trained with a dual loss to output +1 for the pair of similarly positioned objects and -1 for differently positioned ones. Works to some extent.

keynote Machine Learning for Safe, High-Performance Control of Mobile Robots by Angela Schoellig

Trajectory tracking task with real robots. Enhance capabilities of mobile robots. Whenever we can solve a task without learning — we do it, it is much easier. But there are more difficult problems, for which it is hard to find a mathematical model. Examples: fly a robot to slalom between poles, high-speed driving. Without learning, it did not work. Safety through defining uncertainty envelopes, which allows driving faster once there is less uncertainty.

talk Talk Resource-Efficiently to Me: Optimal Communication Planning for Distributed Loop Closure Detection

See http://acl.mit.edu for a 2x more efficient information exchange in a multi-robot setting. In situations where we need to send point clouds, it could make a huge difference.

keynote Getting a Grip on Robot Grasping: Metrics and Protocols for Reproducibility by Ken Goldberg

In grasping for 35 years, actually… not so much progress.

talk LabelFusion: A Pipeline for Generating Ground Truth Labels for Real RGBD Data of Cluttered Scenes

How to make our own segmentation data for robotic manipulation training. Take an RGBD video of a scene. Map the object in the video. Now you can split the video into frames with objects labeled. They have generated a dataset with 350k images and 1M objects. Others can use the pipeline to create their own domain-specific datasets.

Robotics Startup Competition

Purple Robotics
Drones for work. Up to 90 minutes flight! 30+ minutes with 1.5kg payload.
HEBI Robotics
Modular hardware and software platform.
EyeSight
Tactile display for blind people.
aubot
Modular telepresence bot.
Anybotics
Legged robots for dangerous environments, sensors for gas and stuff, www.anybotics.com

Day 2

keynote Deep Reinforcement Learning for Navigation by Raia Hadsell

How mice navigate. Place cells and grid cells. Grid cell firing forms a hexagonal grid. Grid cells emerge from training an LSTM to do path navigation. First learns from velocities. Next they add a CNN, put it into an RL setting and train to reach a goal in a maze with A3C.

RNN state readout will look similar for similar states. How much is the spatial grid pattern that was formed a property of the pattern the maze has? The mazes it was trained on are squares with rooms positioned in a grid pattern. Would a grid-like structure emerge if the training environment consisted mostly of straight corridors or open planes?

MERLIN — Memory Enabled Reinforcement Learning uses a Differentiable Neural Computer for memory.

But all of these experiments run in a simulator. How are these results applicable to the real world? Example of learning city map graphs. Mention of progressive nets for transfer learning from one city LSTM to another city.

End-to-end Deep Learning for robots (Vincent Vanhoucke). Deep Learning transformed speech recognition (2010), vision (2012), translation (2014) from a modular approach into an end-to-end one. Is robotics next?

talk Where to Look? Predictive Perception with Applications to Planetary Exploration

Needs of the Mars Curiosity robot. Radiation-hardened computer on Curiosity is slower than Raspberry Pi, processing is slow, manual driving is faster even from Earth.

talk Design and Analysis of a Fixed-Wing Unmanned Aerial-Aquatic Vehicle

A vehicle that could swim, fly and transition between the mediums would have interesting applications. Works.

keynote Learning to Learn by Pieter Abbeel

Deep Reinforcement learning has achievements. How about the speed of getting there? Can fast reinforcement learning be achieved by meta learning?
Pieter’s belief is that RNNs, by combining data and compute, are the best way to learn how to learn in new environments. Or adapt to them.

Imitation learning. Has lots of successes in robotics. Model-Agnostic Meta Learning (MAML).

Use of simulation. Building realistic simulators is quite hard and compute-expensive. Idea of domain randomization: use a simple simulator, but with high variability of shapes, colors, textures; as it learns to generalize, we hope that the real world will just be yet another “weird” texture once presented to the agent. Extension of the idea to grasping: train on randomly generated meshes and then the real world is just yet another set of randomly generated objects.

Lifelong learning — agents keep learning after they are deployed. RoboSumo (Al-Shedivat, 2017): an adaptive robot learns to win against an initially stronger opponent.

Learning to learn allows discovering algorithms through compute and data rather than human ingenuity, which is a more limited resource, and even more so over time relative to data.

talk PRM-RL: Long-Range Robotic Navigation Tasks by Combining Reinforcement Learning and Sampling-Based Planning

Need to take a closer look.

talk Aggressive Flight with Suspended Payloads Using Vision-Based Control

Estimate the hanging payload trajectory and manipulate it via movements of the drone.

keynote From Building Robots to Bridging the Gap between Robotics and AI by Sami Haddadin

Franka Emika. FOPNet — combining fundamental laws of physics with ANN learning for robotic arms worked; a pure ANN was not able to capture the dynamics.

talk The UNAv, a Wind-Powered UAV for Ocean Monitoring: Performance, Control and Validation

The albatross (the bird) can fly amazing distances, 1000 km per day. How does it do that? It is a wind-powered system. Idea: make a sailboat with wings and minimal contact with water.

3D transformation while flying! Can change its shape while in the air.

talk Accurate and Adaptive in Situ Fabrication of an Undulated Wall Using an On-Board Visual Sensing System

Robots in construction at the construction site. Humans are heavily in the loop.

Day 3

keynote A New Perspective on the Birds and the Bees: Biologically Inspired Aerial by Mandyam Srinivasan

Biorobotics. They explore how bees fly with high-speed cameras and try to replicate that in a flying robot. Gaps, landing, etc. For distance estimation and navigation, optic flow plays a big role. Their plane that works on those principles flies and lands nicely. How to detect movement while you are moving yourself? Analyze optical flow and objects that are not following projected vectors must be moving.

talk Overcoming Exploration in Reinforcement Learning with Demonstrations

keynote Localizing in Tricky Spots by Paul Newman

Works on autonomous driving. Local navigation over a global coordinate frame. A way of de-shadowing an image when converting it to grayscale from RGB. Learning from experiences by learning transformations between different weather conditions using CNNs: day, night, rain, snow, etc. A separate GAN for every transition for now. The same approach to deal with rain on lenses, etc. “Ephemerality mask” to mask out moving objects. What if vision does not work at all? mmWave Radar.

talk Put-in-Box Task Generated from Multiple Discrete Tasks by a Humanoid Robot Using Deep Learning

talk OptLayer – Practical Constrained Optimization for Deep RL in the Real World

talk Uncertainty-Aware Learning from Demonstration Using Mixture Density Networks with Sampling-Free Variance Modeling

Robowars Brisbane

Satellite event “Robowars Brisbane 2018” provided a good explanation of why people actually do all of the stuff above: