Maffy, 2002/2003

Robotic Soccer - Overview

Robotic soccer is an AI research project. It's goal is:
'By the year 2050, develop a team of fully autonomous robots that can win against the human world soccer champion team.' (http://www.robocup.org)

Simulator league

RoboSoccer is played in many different leagues, which mostly differs by technical parameters of the robots (e.g.. size, number of legs or wheels, ...). There is also one software league called Simulator league.The simulator environment consists of Soccer Server (also known as SServer), Soccer Monitor (SMonitor), LogPlayer and two teams of players (10 + goalie + coach). The first versions of Simulator were made by Itsuki Noda. The server takes control of objects movement and interactions (turning, kicking, fouls, free kicks, ..) and player's perception. SMonitor shows the current state of the game. LogPlayer replays saved game logs. The team consist of ten players, goalie and coach, each of them served by its own program. All programs communicate via UDP sockets. Inter-team communication is allowed only by SServer commands (say, hear). The preferred platform is Linux, but SServer is fully portable. There also exists SMonitor and LogPlayer for Windows ( http://www.lits.tsinghua.edu.cn/RoboCup).

Maffy

Soccer simulator team developed as University project in 2002/2003.

Goal of the project

• Soccer team fully capable to play the Simulated Soccer Internet league ( http://sserver.sourceforge.net/league).
• Soccer team able too beat some team(s) currently taking part on SSIL (2002).
• Multiagent learning research. The players are supposed to learn playing with the help of the teacher (UvA Trilearn - http://carol.wins.uva.nl/~jellekok/robocup) in the first stage. In the second stage it should improve itself during the game.

Methods

• The player has only limited set of percepting and interacting possibilities. It doesn't get all informations about the field (visible_angle).
• The decision making will be based on neural network, which inputs is the game situation and output the reaction.
• The modified multilayer neural network will be used during the game. Self Organizing Maps will be responsible for kick off positioning.

Learning

Maffies are supposed to learn.

That's nice. But: How? What? When?

The middle question is the easiest one: To play soccer.

The last one is just a little bit more complicated: During the learning stage. Later also during the game.

The first question locked my attention to robosoccer topic. Many people take it for the most complex situations in AI. The environment is 'noisy' (agents - reacting entities - do not get all the informations correctly), the agents have to compete (there are two teams in soccer, remember?) and they have to cooperate as well. Who misses the cognition, I can just recommend another RoboCup league.

There are two basic styles of learning:

Supervised (with teacher)

The learning entity gets inputs and known right outputs. It's trying to adapt itself to give these correct outputs.

The agent is "thrown into the water of real life" and as it reacts to real situations, the teacher tells, how good each particular reaction was.

Unsupervised learning (without teacher)

This one is often used in pattern recognition. Well known example is OCR (automatic character recognition). You don't need the teacher to remember how something looks like!

Is any of these approaches good for robosoccer? Not at all! Can you imagine the database of play situations and it's right solutions? RIGHT solutions? In soccer? All situations? Good luck!

Right, unsupervised learning procedures show us, how to find a known situation similar to the current one. OK, we know it. And know? Where should I kick the ball?

Q-learning

When you want to learn without the supervisor, you need at least to know, how good your current situation is. Too bad for soccer.

Q-learning basic idea is:

• Did I get a goal? That's bad.
• Did I score? That's good.
• The current situation is roughly as good as the following one. This allows the good/bad evaluation in scoring situations 'leak' to other.

In fact, you know it from your life. The child just touched the hot oven. What a pain! The last moves were wrong decision. And that's it.

Implementation

As I mentioned before, the robosoccer is a really complex task. You can't use only one method and hope, that it's powerful enough.

The plan is to use Kohonen's maps (unsupervised learning) for basic situation recognition. Their outputs will be used as inputs for multi layered neural network. This one will make the move/kick/... decisions.

And how is it gonna learn?

As mentioned above, the easiest way is to have a teacher. And there is a lot of teams already playing the robosoccer. And they are good.

And later? After Maffies will beat their teacher? Did you hear about Q-learning?

Progress

November 25th 2003

Project successfully delivered, however, plans were not fullfilled (from my point of view).
Project postponed for its difficulty.

November 20th 2003

In september, some kind of prerelease was finished. All parts are debugged, everything is working except it doesn't play. The problem is, the neural networks are not learned yet.

May 5th 2003

• Array overflow found and solved
• Position triangulation logical error found and solved (the average of -175 degrees and +175 degrees is not 0)

January 28th 2003

The beta version is completed. Well, I'd rather call it demo. The situation Maffies can handle is just 'one player against one goalie'. And the goalie is blind or what.
It's nice to compare Windows and Linux doing this task. It takes about 99.9 percent of processor time, when running on Windows (R), and about 0.7 percent, when on Linux. And guess, where does it run faster?

January 6th 2003

The player is able to triangulate his position from dummy informations from the sserver (try to draw the triangle with edge sizes 67.5, 4, 73.5:)
This means we know where we are and where are we going!!!
But not where we SHOULD go :-(

December 11th 2002

Objects representation classes implemented.

December 2nd 2002

The wall was built. The previous program had to be rewritten from scratch, because of some features of SServer and UDP. It handles now SServer bad manners such as port changing during the game. It seems network interface to be complete. Hopefully.

November 25th 2002

The first simple client is running. Well, jumping. The client program receives commands from stdin and sends them to SServer. (move 10 20) is my favorite command. The player then jumps to specified position. Messages from SServer are printed to stdout.
I know, It's exactly, what is rcssclient in SServer distribution does. But I've decided not to use udpsocket.h. I'm using Mr. Donahoo's C++ socket library. It's much more clear.
The first server-client-in-one program is already working. It's supposed to stay in between the teacher and the server and re-send informations flying around. This allows the player to get the field state and teacher's reaction.