The Planet Earth Simulator Project - A Case Study in Computational Semiotics

AFRICON2004-DOEBEN-HENISCH

Index Terms—Computational Science, Computational Semiotics, Epistemology, Knowledge Representation, Knowledge Engineering, Simulation of Knowledge

INTRODUCTION

The PLANET EARTH SIMULATOR (PES)-Project [1] is an open source project which has been started January 2003 by Gerd Doeben-Henisch and Jens Heise at the Institute for New Media [2] in collaboration with the University of Applied Sciences [3] in Frankfurt am Main (Germany). This project will be discussed here from a more theoretical point of view of COMPUTATIONAL SEMIOTICS and not primarily from the point of view of SOFTWARE ENGINEERING and PROGRAMMING.

To get a rough idea of the project we will have a short look to it from the point of usage.

From the point of USAGE (cf. Fig. 1) you can construct arbitrary intended models of parts of the world. For this you have a VISUAL LANGUAGE VisualFSL at your disposal to built VISUAL ARTIFACTS --diagrams, graphs-- as representations of your intended parts of the world. And, as far as other pre-built models are already available, you can SELECT MODELS for simulation and then you can RUN THE MODELS for to simulate some intended processes. During such a simulation you can also INTERACT with the simulated processes, individually or in groups. SYNCHRONOUSLY or ASYNCHRONOUSLY you can also activate some EVALUATION PROTOCOL for to collect and to show selected data of the process activities.

Fig. 1: The Use Case of the Planet Earth Simulator. The user can act either as a model builder or as a simulation user.

If you would take only this short description of the project and you would start to browse different scientific journals to get some feedback how this topic has already been discussed in the scientific community then you will find an overwhelmingly broad spectrum of disciplines and language games, which you could relate to this topic.

Let us have a short look to at least some examples: COMPUTER SUPPORTED COLLABORATIVE LEARNING (CSCL)(cf. e.g. the Proceedings of the conference in [4]) does investigate collaborative learning processes supported by computers; in the PES-Project we have collaboration, we have knowledge, we have learning, we have computers supporting the construction of models and their simulation. If you focus on knowledge as theories you could also mention THEORY REVISION (cf. e.g. the proceedings of the conference mentioned in [5]) as a paradigm which is touched; you will find further associations to CASE BASED REASONING (CBR) (cf. e.g. [6]) as well as to ACTIVITY THEORY (cf. e.g. [7]); in activity theory the learning is seen as a sequence of interactions of the learning agent with the environment. You can also relate this topic to parts of ARTIFICIAL INTELLIGENCE, to KNOWLEDGE REPRESENTATION or to COGNITIVE PSYCHOLOGY, even to EPISTEMOLOGY or COMPUTER BASED EPISTEMOLOGY. And this list is by far not exhaustive.

The View of Computational Semiotics

The plurality of language games mentioned above introduces a variety of terminologies, which is not only a blessing. Indeed, it can become confusing. If there would exist a language game, which could integrate most of the above mentioned individual perspectives without loosing any scientific strength and without missing a general flexibility to be able to cope with all the important problems with which we are faced today, then it would be tempting to use this new language game for the communication of important findings in the realm of COMPUTATIONAL SCIENCES.

It is the opinion of the author, that the paradigm of COMPUTATIONAL SEMIOTICS combines in an interesting way aspects of computation and of semiotics, thereby integrating the dimensions of engineering and cognitive science as well as many traditional topics of the so called soft sciences as well.

SEMIOTICS (cf. e.g. [23],[24]) is a discipline which investigates the sign processes primarily in the domain of human persons, but then, in a generalized way, also in other domains which have a sufficient resemblance to human based sign processes: animals, technical devices etc. This subject relates Semiotics directly to empirical disciplines like ethology, experimental psychology, phonetics and neurology, but also to linguistics, architecture, fashion, music etc.

To connect Semiotics to the powerful methodologies and tools of modern computational sciences and thereby to engineering it would be necessary to show, that one can map the concept of a SEMITOC AGENT in all important aspects completely into the concept of a COMPUTING AGENT.

As the author showed in [8],[9] it is straightforward to do this. One can indeed translate the concept of the SEMIOTIC AGENT as it is used by CHALES MORRIS [10] --one of the well accepted founders of Semiotics-- without any loss into the concept of a COMPUTING AGENT as it has been introduced by ALAN MATTHEW TURING [11]. This is important because it could be shown in the course of time that the Turing Machine concept is formally equivalent to all other kinds of formalizations of computation [12]. But also the concept of the Turing machine is the most powerful concept of computation, it is also the most simple and most convincing one (this not only, because Kurt Gödel, one of the mathematical giants of the 20th century, mentioned this [28]). The central argument for the formal equivalence of the two concepts COMPUTING AGENT --represented by a turing machine-- and SEMIOTIC AGENT has been the structural similarity between both kinds of agents (cf. Fig.2).

Fig. 2: Computing Agent vs. Semiotic Agent. The computing agent is realized as a Turing Machine and the Semiotic Agent is taken from the description of Charles Morris. Here is the structural similarity emphasized.

The environment E of the semiotic agent can be mapped into the tape of the turing machine and the internal states IS of the semiotic agent can as well be mapped into the states of the machine, arranged as a machine table. The perceivable stimuli S of the semiotic agent become the readable inscriptions of the tape and the responses R of the semiotic agent are becoming the symbols which writes the machine onto the tape. The movements of the semiotic agent in the environment are mapped into the movements of the turing machine on its tape.

With this argument we have a firm grounding of SEMIOTICS into COMPUTATION and therefore it is very sound to speak of COMPUTATIONAL SEMIOTICS as a KEY CONCEPT within the realm of COMPUTATIONAL SCIENCE.

If we now apply the language game of computational semiotics to the PES-project we can reveal a strong semiotic perspective of this project (cf. Fig. 3).

Fig. 3: Model Builder as Semiotic Agent. A circle represents a semiotic agent which can organize representations as well as meanings to model the external meanings and representations of the world

The user as a model builder will be interpreted within Semiotics as acting as an (semiotic) agent A which translates his internal (subjective) representations R_sub.A of some internal (subjective) meaning M_sub.A into an external representation R_ext.A. Another user B can do the same, i.e. translating his internal representations R_sub.B of some internal meaning structures M_sub.B into external representations R_ext.B assuming that R_ext.A and R_ext.Bare identical.

This view can be extended a bit more. If one partitions the perceivable world into representations R_{ex t}and non-representations called (potential) external meanings M_ext, then we can assume that a semiotic agent A is able to set up ('LEARN') a mapping between the external meanings M_ext and the internal (subjective) meanings M_sub.A. Connected with this learning of potential meanings a semiotic agent A can usually also set up a mapping ('learn') between the internal representations R_sub.A and the internal meanings M_sub.A. To learn a language would be some instance of this general case of learning a representation which is grounded in meaning (we assume here, that for a semiotic agent the internal representations R_sub are 'structurally identical' to the external representations R_ext.). If there is some grounding, then the representations can SIGNIFY the related meanings. Clearly one can also learn some external representations without an explicit grounding in meanings; then we have the case of purely formal structures, PURE 'SYNTAX' .

All the processes, which are involved in these sign processes are very demanding and far too complex to be able to described here in detail. We can only mention here that the learning of a semiotic agent always happens within a POPULATION of semiotic agents, that it is mediated by ACTIONS, that the agent itself is SELFDRIVEN and that such learning processes can only happen with a 'minimum' of COLLABORATIONS between all participating agents.

Although many details are missing at this point one can --presupposing the above mentioned assumptions-- introduce the following more complex terms:

The relation between the internal meanings of one semiotic agent and its external meanings can be said to be EMPIRICAL SOUND(M_sub.A, M_ext) if and only if one can address certain METHODS of MEASUREMENT MS for M_ext such that the DATA of MS are in 'sufficient agreement' with M_sub.A.

The relation between the internal representations of one semiotic agent and its external representations can be said to be ARTICULATORY SOUND(R_sub.A, R_ext) if and only if the STRUCTURES and the DYNAMICS of R_sub.Aare a subset of the structures of R_ext.

The relation between the internal representations of one semiotic agent and its internal meanings can be said to be REFERENTIAL SOUND(R_sub.A, M_sub.A) if and only if the 'necessary' STRUCTURES and the DYNAMICS of M_sub.A can 'sufficiently' be represented by R_sub.A.

A set of external representations R_ext is said to be WEAKLY COMPUTABLE(R_ext) if and only if there is a formal grammar G such that G is an element of the CHOMSKY HIERARCHY and R_ext is a subset of the language L_G, which is determined by G.

This last definition lets open the case that in the future there will be perhaps applications where the computation of some representations will be done by including the signified meanings of these representations. This case would perhaps be the case of STRONG COMPUTABLE representations.

At this point of our discourse about semiotic agents we will focus on the nature of the representations which are in some sense useful for the simulation of knowledge as we have it envisioned in the PES-Project. Because the external representations in case of the PES-Project are intended to represent some part of the PERCEIVED REAL WORLD it follows that these representations have to have some grounding in external meanings. Thus, in this case we can speak of a LANGUAGE. This language should be at least WEAKLY COMPUTABLE, ARTICULATORY SOUND, REFERENTIAL SOUND, and EMPIRICAL SOUND. The availability of such a language could allow semiotic agents to clarify the validity of their external representations R_ext.i with regard to the STRUCTURE and to the DYNAMICS of their internal representations R_SUB.i and through this they can furthermore try to correlate their own perceptions of their environments with the the perceptions of the other agents.

Letting aside lots of technical stuff for the analysis of the above mentioned terms the author will ask instead what kind of LANGUAGE FOR EXTERNAL REPRESENTATIONS would be rich enough to represent most of the intended processes as well as simple enough to be able to be handled by a computing agent?

A LANGUAGE FOR EXTERNAL REPRESENTATIONS

1. A Common Basis for All

Before we will discuss the subject in more detail we will first take a look to the general scenario (cf. Fig.4).

From the point of view of (Computational) Semiotics we are faced with semiotic agents which PERCEIVE possible meanings M_ext of the world, which PROCESS these perceived meanings and which INTRODUCE/ USE some external REPRESENTATIONS R_ext –here understood as languages-- to represent some of the processing contents for Communication (the processed CONTENT can also be seen as a subset of the semiotically relevant INTERNAL STATES of MORRIS).

Fig. 4: Basic Scenario for Semiotic Agents while transferring Perceptions of the World into some commonly shared Representations

From a more formal point of view one can show that this process only work if the perceptual and the processing apparatus is structurally and dynamically 'sufficiently alike'; only then will the changes in the realm of the external meanings be encodable as certain changes in a otherwise fixed framework which serves as a common reference between the different semiotic agents. This common framework serves as an 'implicit' structure of events.

It's worthwhile to notice that this idea was already worked out in the realm of transcendental philosophy, especially by Immanuel KANT in his “Kritik der reinen Vernunft” [13].

Charles S.PEIRCE, one of the other well known founders of modern Semiotics, notices this very clearly and in one of his early lectures [25] he gives a very lucid account of this issue.

These ideas have later on be reinforced by a pioneer of modern ETHOLOGY like Konrad LORENZ in his fames book “Die Rückseite des Spiegels”[14].

Recently --but probably not finally-- there is an additional support for these ideas by modern Neurobiology (cf. e.g. a handbook like [15]) which has revealed to us that the human brain despite its great plasticity is nevertheless to a high degree 'pre-programmed' in the way how an organism perceives and processes ideas, and this TYPICAL FORMAT OF BRAIN PROCESSING is the common basis for all human semiotic agents. Only by this fact are we able to set up commonly shared languages.

From the point of view of Computational Semiotics is SOFTWAREENGINEERING (SWE) an interesting activity. In SWE we have groups of semiotic agents which try to convert a subset of the possible meanings of the world into external representations which should cover nearly completely all intended meanings and at the same time should be written in a way that the final compilation into some computer readable machine code is unique and error-free.

To solve this problem the discipline of SWE has developed during the last 5 decades lots of generalized methods, formal models and standards.

Two main lines of thought I want to emphasize here: so-called object oriented SWE (OO-SWE) (cf. e.g. [16], [17], [26]) and systems oriented SWE (SO-SWE) (cf. [19]).

As the names of these two approaches already suggest is in the OO-SWE the main concept the OBJECT and in the SO-SWE the concept SYSTEM (but one has to keep in mind that in OO-SWE objects are understood as instances of more abstract concepts called CLASSES. Thus you have first to introduce a class S when you will use an object o1 as an instance of this class S.).

Fig. 5: The two main Concepts System and Class as used within SO-SWE and OO-SWE. As one can see is it straightforward to convert one concept into the other one

For our discourse here it is interesting that both concepts are part of very elaborated formal systems which allow the usage of these terms to express fairly complex structures and processes. Furthermore they have a graphical dimension as well as a purely symbolical one. Formally can both concepts directly be translated into the other one.

Whereas the OO-SWE approach seems today spreading more and more in those SW-projects which do not need special mathematical treatments and have not to meet special real time requirements, the SO-SWE approach seems to dominate the last two mentioned scenarios. Especially the REAL TIME DOMAINS where the dimension of time is crucial are dominated by the SO-SWE (cf. [18], [20-22]).

From the point of Computational Semiotics it is important that we have here two widely known representational systems which see the world of potential meanings either built up by objects or built up by systems.

I do not know of any empirical investigation in Psychology or Neuropsychology which did run systematic experiments to get empirical data to the question which of these both conceptual strategies is more akin to the neurobiological machinery which does the processing for us. Does it make a difference to think in objects or systems or does it not matter?

I personally have daily to work with both paradigms, but for theoretical discussions and for the realm of simulating Knowledge I prefer the systems approach.

In the PES-Project we have until now decided to take the well established terminology of systems and systems based programming languages as our computer readable language. We call this language FORMAL SYSTEMS LANGUAGE (FSL). As direct interface to a 'normal' user as model builder we will provide a graphical user interface with a VISUAL LANGUAGE VisualFSL based on SYSTEMS as smallest possible units (cf. Fig.6).

Fig.6: A user as model builder tries to translate a real world problem into some meaning structures which he also wants to represent with the aid of the visual Formal Systems Language FSL

Thus for the modeling of some PROBLEM of the world it is in the PES-Project presupposed that there are some internal meanings M_Sub.Problemwhich function as the perceptions of this problem in the model builder. These perceptions correspond to some internal representations R_Sub.Problem, which in turn have to be correlated with some external representations R_ext.Problem. The internal meanings have also the functions to be the INTENTIONS of the representations. In the case of the PES-project will the external representations R_ext.Problembe realized by graphical elements on the screen. These graphical elements are constituting what is called the VisualFSL-Language.

From this setting it becomes clear that the design of the visualFSL-Language can not be done as 'Design without Intention' [29].

Instead one sees that it is a special task to investigate how the visualFSL-Language is bound by the intended problems which have to be modeled.

Especially it has to be investigated whether the general structure of our problem perception can really be transformed without essential losses into a language like VisualFSL.

Leaving aside the intentions one can show that the representations of systems can be connected to ARBITRARILY COMPLEX NETWORKS OF SYSTEMS. Also the representation of time connected to every system is no problem.

But if one includes the dimension of intentions then the task is becoming much more demanding.

A WORLD OF SEMIOTIC AGENTS

What we can see here, finally, is, that within the PES-Project the world will be handled as if it consists only of networks of semiotic agents. Whether this is empirically and philosophically sound is perhaps an open question.

As we know from Edwina Taborski [27] and others [23] it is clearly possible to analyse the world in very different –but also semiotic-- ways.

Despite all these other semiotic pathways the author prefers to focus on the human sign usage as the primary source of inspiration for the understanding of signs, meaning, and communication. And we are really not at the end of this task but only in the very beginning.

INTRODUCTION

The View of Computational Semiotics

A LANGUAGE FOR EXTERNAL REPRESENTATIONS

1. A Common Basis for All

A WORLD OF SEMIOTIC AGENTS

References