Press event for FROG – 24 September 2014

Today FROG is the centre of attention at the Royal Alcázar in Seville!

FROG enjoying the limelight at the Royal Alcázar in Seville
FROG enjoying the limelight at the Royal Alcázar in Seville

All camera’s and microphones were turned on FROG…

photo 2-1
in the Vault Room … from behind FROG …

 

… and in the Tapestry Room.
… from behind the cameras.

The event was started with short presentations by Vicente C. Guzmán Fluja the Rector of Universidad Pablo de Olavide in Seville,  and FROG’s own Luis Merino Cabañas also from UPO, and Vanessa Evers, coordinator of this EU project representing the University of Amsterdam.

Rector of UPO opens the event
Rector of UPO opens the event

FROG then took everyone on a short tour of the Alcázar and demonstrated his capabilities. Journalists from all media were in the party, both local and national!

Back in the on-site lab, Luis Merino and Fernando Caballero were interviewed for TV and radio.

 

Interview in the lab
Luis and FROG were interviewed in the lab

 

This afternoon there will be a workshop for a small group of official tour guides from the Royal Alcázar and one from the Lisbon Zoo.

Coverage here 

Team from Imperial

 

Maja Pantic and Owen Holland
Maja Pantic and Owen Holland

Maja Pantic leads the Intelligent Behaviour Understanding Group (iBUG) at Imperial College in London.

Jie Shen and Ioannis Marras at work in the Royal Alcázar in Seville, September 2014
Jie Shen and Ioannis Marras in the Royal Alcázar in Seville, September 2014

 

Jie and Ioannis in Seville, June 2014
Jie and Ioannis in Seville, June 2014

 

running experiments in Seville, May 2013
running experiments in Seville, May 2013

 

Ioannis Marras
Ioannis Marras

Ioannis Marras is a researcher from the Intelligent Behaviour Understanding Group (iBUG) at Imperial College, London. His research interests include image/video processing, computer vision and pattern recognition. During his work, he has developed multiple computer vision techniques for 2D/3D face tracking and 2D/3D face recognition in the wild.

Jie Shen
Jie Shen

Jie Shen is a researcher from the Intelligent Behaviour Understanding Group (iBUG) at Imperial College, London. His research interests include software engineering, human-robot interaction, and affect-sensitive human-computer interfaces. During his work, he has developed the HCI^2 Framework (http://ibug.doc.ic.ac.uk/resources/hci2-framework/), which is a software integration framework for human-computer interaction systems, currently being used in multiple research projects.

Jie Shen defended his PhD thesis, titled ‘Realising Affect-Sensitive Multimodal Human-Computer Interface: Hardware and Software Infrastructure‘, in May, 2014.

Lisbon 2013
Lisbon 2013
IMG_7553
experiments in Seville, June 2014

The rain in Spain…

Unfortunately, the rain in Spain doesn’t always stay on the plain. Today we had plenty of it at the Royal Alcázar and here we have a FROG that doesn’t like getting too wet.

Outside the lab - 21 September 2014
Outside the lab – 21 September 2014
photo
Fine yellow sand in the gardens

In spite of all of the visitors the staff keep the halls and patios amazingly clean. There is a lot of loose yellow sand in the gardens and, of course, this gets washed and walked everywhere.

Wet but pretty, garden path
Wet but pretty, garden path
Blossom rained down from the trees
Blossom rained down from the trees

IMG_7901

 

Part of the tour route
Part of the tour route

 

(next day) Path close to the lab and close to the gardens - amazingly little sand
(next day) Path close to the lab and close to the gardens – amazingly little sand

Fernando Caballero Benitez

 

Fernando in Lisbon, November 2013
Fernando in Lisbon, November 2013

Fernando Caballero Benitez and Luis Merino have been working together since 2003. They work on robot localisation and navigation, respectively. A robot first needs to know where it is (localisation) in order to carry out its mission (navigation). Fernando’s expertise is a bit more towards aerial robots, Luis’ towards ground robots.

IMG_7801

PhD students who are fortunate enough to have these two as their supervisors are very lucky. It may be hard work but that goes with some solid training in the science that is the fundament of what they do, and sincere appreciation for what the PhD’s achieve. It’s a pleasure to see this close supervision in action – even if you don’t understand Spanish.

One or both
One or both of these supervisors is on hand nearly all of the time
Apart from lots of lecturing and work in projects, Fernando is taking part in one of the European Robotics Challenge in the GRVC–CATEC  team taking on Challenge 3. These Challenges are set up in stages, something like the levels in a game. Each stage has to be overcome in order to pass on to the next phase and to acquire more funding. The first stage is to sift out the serious participants. The GRVC – CATEC  team hopes to go on to the second phase after October 2014.

Fernando likes a good barbecue and he designed our polos.

Fernando also likes a good barbecue
A really good barbecue
An excellent barbecue
An excellent barbecue

One day Fernando will get around to making a serious, English language website and you will then find the link here. Till then we’ll have to make do with his publications page.

How are we going to tackle those curbs?
How are we going to tackle those curbs?

The UPO team

You will find some of UPO’s output for FROG and other research projects here: Luis, datasets and publications.

Luis Merino
Luis Merino hosting the consortium during an integration meeting

 

Luis with a film crew in February 2014
Luis with a film crew in February 2014
Fernando Caballero
Fernando Caballero and FROG

 

Luis and Fernando lead and supervise UPO’s team of  young researchers………..

 

Ignacio … without beard
Ignacio Pérez Hurtado de Mendoza … without beard

Ignacio Pérez Hurtado de Mendoza is a computer science postdoc interested in the application of machine learning techniques to social robotics. In the FROG project supports the development of software modules related to the navigation of the robot and assists with the deployment of experiments.

 

Noé Pérez Higueras
Noé Pérez Higueras

Noé Pérez Higueras is in charge of the safe and robust navigation of the FROG robot in (indoor and outdoor) environments with people. Noé is trying to add social capabilities to navigation algorithms so that the robot respects human social conventions and guarantees the comfort of surrounding persons.

His PhD Thesis is on: “Robot autonomous navigation and interaction in pedestrian environments“.

Noé’s PhD is directly related to his work in the FROG project. Mainly, he is studying the different robot navigation algorithms and trying to extend them by adding social skills. To do that he employs machine learning techniques in order to learn from real people how they navigate between each other in crowded scenarios.

Noé and the UPO person detection tool
Noé and the UPO person detection tool

In 2014 Noé spent 3 weeks in the Netherlands, mapping the Gallery at the University of Twente for the opening ceremony with the Dutch king.

Noé preparing to map the Gallery at the UT in February 2014
Noé preparing to map the Gallery at the UT in February 2014

Noé fitted in well with the UT PhDs and will continue to work with some of them in the TERESA project. When he makes a website we will link to it here.

Noé and some UT PhDs
Noé and some UT PhDs

 

 

Javier Pérez-Lara
Javier Pérez-Lara

In the FROG project, Javier Pérez-Lara is working on improving localisation algorithms, based not only in laser readings but also on appearance matching, to recover from erroneous convergences to wrong localisation or even recovering from complete lost situations.

Javier’s thesis topic is related to robust localisation for mobile robot navigation and mobile robot interaction in crowded environments. Where we try to take into account the variability of crowded environments when localising and relocalising mobile robots.
Javier’s website can be found here: http://grvc.us.es/staff(http://grvc.us.es/staff/jperezlara/) but as he himself says:  “it’s not updated!”

 

Javier in Lisbon, November 2013
Javier in Lisbon, November 2013

 

Javier in Seville, 2014
Javier, Carlos (IDM) and FROG in Seville, 2014

 

Rafael and FROG - both in white - Seville February 2014
Rafael and FROG – both in white – Seville February 2014

Rafael Ramón Vigo’s knowledge of transversal competences like electronics, software and hardware are all put to good use in the FROG project. Rafael provides help with the general set up of the robotic platform and assists with the deployment of experiments.

Rafael’s PhD thesis work is on how to infer from data and its statistics the insights of human motion navigation, with the idea of transferring it to the robot’s navigation stack. The basis of this approach is to use machine learning algorithms.

Rafael and FROG, Seville 2014
Rafael and FROG, Seville 2014

Rafael also grows delicious mangoes. Recently his family planted nearly 2000 young trees that will come into production in 4 or 5 years time.

Beautiful fruit...
Beautiful fruit…
…from sun drenched fields
…from sun drenched fields
… in a stunning landscape.
… in a stunning landscape.

 

In 2013, Noé, Javier and Rafael had to spend weeks at a time in Lisbon. Though often cold or very tired, they did discover some good places to eat and were given Wi-Fi access at most of them.
Noé, Javier and Rafael nearly took up residence in Lisbon and it wasn't always this sunny!
Noé, Javier and Rafael nearly took up residence in Lisbon and it wasn’t always this sunny!
 

Hidden meaning or just great design?

How about this for an emblem? It looks great on the FROG polos.

FROG polo, modelled by Paulo
FROG polo, modelled by Paulo

The emblem printed on the polos is actually a reversed version of the docking target. It is a set of ArUco markers that just happen to say FROG – though it probably took Fernando Caballero quite some time to find them.  UPO uses this ArUco marker to align the FROG to its charger during the docking procedure.

 

FROG Daddy

Meet Carlos Marques, aka FROG Daddy.

 

Carlos Marques - Seville 2014
Carlos Marques – Seville 2014
He built the FROGs.
He built the FROGs…(with help from Paulo Carriço and Mário Inácio).
He spent hours labeling all of the cables.
He spent hours labeling all of the cables.

 

He has tested most of the terrains.
He has tested most of the terrains
...indoors...
…indoors in Lisbon…
...and in Seville...
…and in Seville…

 

...outdoors.
… and outdoors.

 

He has to watch while the researchers put his baby through its paces.
He has to watch while the researchers put his baby through its paces.

 

And what do they do when he does leave early?
And what do they do when he does leave early?

 

He’s always there to help:

After care ...
After care … Carlos on top of the bookcase.
Maybe...
Maybe…

 

Well...
Well…

He assembled baby-FROG and is wizard with the game pad during experiments…

... and is wizard with the game pad during experiments (when he's finished trying out the terrain).
… when he’s finished trying out the terrain. Unfortunately we can’t show the experiment films as they include images of passers-by.

He works all hours…

… an has to catch a power nap.
… and has to catch the occasional power nap.

 

And all too often he has to leave his wife and kids in Lisbon while he attends meetings… Thanks from the FROG consortium to Carlos’ family!

 

 

FROG’s Wizard-of-Oz web interface

Yesterday, while the FROG was powered-down for hardware enhancements, Randy Klaassen and Jan Kolkmeier from the University of Twente added some features to their FROG Wizard-of-Oz web interface.

They designed and implemented this web interface to control not only the output of specific AR content but also to trigger chosen steps from the State Machine.

This interface is not used for the FROG tour when the robot is running autonomously but is a splendid tool for experiments or testing as it can trigger specific states or abilities. It can also be used to keep the FROG in action in the case f really bad weather as it can be used to run just the indoor parts of the robot mission.

FROG's WOZ web interface runs on any device - here on a laptop, a notepad and a smartphone.
FROG’s WOZ web interface runs on any device – here on a laptop, a notepad and a smartphone.

 

Earlier that had made a quick round of the Alcázar and effectuated some quick fixes in their coding for “Ripples through Europe” and “The toilets are NOT over there!

Ripples through Europe

Another thing that has changed since are last visit is tapestry with the map of Europe.  The AR overlay was achieved by defining markers (the position of a number of specific features) on a photo of the original to  build a model. These markers then had to be recognised by the FROG using incoming information from the antenna’s camera so that these could used to aim the overlay. The same set of markers was defined in the overlay and these had to match up to the features in the tapestry (the target) so that the overlay would be correctly positioned in the resulting overlay projection. All of this is necessary as, due to its autonomous social navigation, the FROG may stop at a different location in the hall on each separate tour.

Ripples in a tapestry that is five and a half by nearly four metres can be quite large – and they are not always in the same place.
Ripples in a tapestry that is five and a half by nearly four metres can be quite large – and they are not always in the same place.

IMG_7715

IMG_7714

Too many ripples in the tapestry at any time could mean that the feature markers cannot be recognised. Of course, there are several solutions to compensate for differences in the folds in the tapestry between tours. This would take some time to implement and as we have some more pressing matters for this session a quick solution was chosen. This involves using the whole tapestry area as a target instead of specific points on the target. Not as elegant as it could be but certainly a neat solution for a proof of concept.