Machine Learning B
LV-Nr: 708.061 VO,
708.062 KU
Course Homepage: www.igi.tugraz.at/lehre/MLB
TUGonline link: Machine
Learning B
Design of Autonomously Learning and Evolving Systems
The
current
methods for the design of robustly functioning large computing systems
are
challenged by the need to design larger software systems with very
large
numbers of interacting components, and also by the need to design
hardware
devices that carry out computations with very large numbers of somewhat
unreliable components on the nanoscale. In addition one needs to
develop
computing systems (including robots) that can continuously and
autonomously
learn, i.e., learn without a supervisor which tells them during
training at
each instance what the ideal action would be. An important task for
computer
science is to develop tools that will be able to provide this new
generation of
system design principles and algorithms. Whereas one may doubt whether
this
task can be solved, nature provides a positive example: Biological
organisms
consist of extremely large number of communicating components that
evolve
without a supervisor, and carry out extremely complex computations not
only
within brains, but also within the gene regulation network of each
cell. In
addition, biological systems are highly adaptive, i.e., they can learn
to cope
with changing problems and environments.
The
goal of this course is to examine the most promising ideas for the
design
of autonomously learning and evolving systems, and to introduce the
students to
software that allows them to ealuate the potential and limitations of
these
ideas, and to develop better ideas. Whereas the so-called genetic or
evolutionary algorithms have dominated research on methods for evolving
well-functioning articifial organisms in previous decades (which has
lead to
some limited success), one has now realized that nature actually uses
quite
different principles for evolving, improving and maintaining biological
organisms
than those which were implemented in these algorithms: Since there is
no
master-agent that supervises the development of biological organisms,
nature
has invented powerful distributed methods for developing functioning
multi-cell
organisms, based on gene regulation networks within each cell that
follow a
complex set of rules (encoded in the DNA) that tell the cell how to
respond to
external signals (from other cells, or from the environment), and when
to build
and renew cell components.We will discuss in this course simple
mathematical
models that capture essential aspects of these design principles, and
which
provide new methods for the design of robustly functioning,
self-repairing, and
autonomously learning artificial organisms. In addition we present the
main
methods of reinforcement learning, which is apparently the primary type
of
learning that is used by autonomously learning biological organisms.
Covered topics:
* Simulated annealing as an alternative to genetic algorithms
* What is known about the way in which biological species have evolved, and how individual
organisms develop
* Simple mathematical models for gene regulation networks
* Principles of biological gene regulation networks (mainly from the book by Uri Alon)
* First examples for self-generating artificial circuits and organisms with self-repair capabilities
* Concepts and paradigms for self-organization
* Options for the design of autonomously learning systems
* Algorithms for Reinforcement Learning
* Open research problems
