SMART II TMR Network Coordinator's Web Site
[ Overview ] [ Partners ] [ Benefits ] [ Meetings ] [ Technical Annex ] [ Training ] [ Servers ]

SMART II PROJECT PROGRAMME

Network Title : Semi-autonomous Monitoring And Robotics Technology

Network Short Title : SMART II

Part A - The Participants

The Contractor will act jointly and severally towards the Commission with the following Associated Contractors to perform this Project Programme:

           ROBOSOFT SA,                                     France                        
           DIST, Univeristy of Genoa                        Italy                         
           AITEK Srl                                        Italy                         
           Scuola Superiore S. Anna / ARTS Lab              Italy                         
           CSATA-Novus Orta, Technopolis Bari               Italy                         
           University of Leeds                              Great Britain                 
           University of Reading                            Great Britain                 
           Trinity College, University of Dublin            Ireland                       
           Laboratory of Image Analysis, Aalborg            Denmark                       
           University                                                                     
           Institute of Systems and Robotics                Portugal                      
           EC Joint Research Center-ISPRA                   Italy                         
           University of Edinburgh                          Great Britain

The contractor and the Associated Contractors are hereinafter referred to jointly as "the participants".

Part B - The Joint Programme of Work

B.1 Research Objectives

The objective of the SMART network is to develop technologies in the areas of surveillance and monitoring. These include both fixed installations for tele-surveillance and monitoring systems and mobile surveillance and monitoring systems for hazardous situations such as underwater and nuclear environments. The primary goal is to create a research infrastructure for efficient mobility of human resources with the objective of technology transfer and dissemination of information. Consequently, the activities of the network are organized on several levels. In order of increasing visibility, these are:

1. Transfer and integration of technology for monitoring and surveillance, autonomous robotics, and man-machine interaction in each of the following application sectors: surveillance and monitoring of public areas, under-water robotics, construction robotics and nuclear material storage areas. This work requires long-term and short-term exchange of personnel and regular meetings.

2. Workshops to present research results and discuss research in progress.

3. Organization of an annual open scientific symposium (SIRS) to disseminate research work.

4. Preparation of "on-line" tutorials, course-ware, and educational curricula.

B.2 Methodological Approach and Work plan

The SMART II workplan has been designed to complement existing research activities with trans-national exchanges of doctoral and post-doctoral researchers. The workplan builds on common research projects between the partners. To construct the workplan, the consortium generated a list of possible joint tasks in the areas of mobility and computer vision for surveillance and monitoring. These research tasks were then clustered into workpackages. The first two workpackages concern topics which are central to the SMART network and thus represent the areas of most established collaborative activity. The third workpackages represent areas of growing interest. The fourth workpackage concerns scientific and industrial dissemination activities.

The workpackages which make up the SMART II workprogramme are listed as follows. Each of these workpackages are developed in the following paragraphs. Each task is accompanied by a table which indicates which partners will participate: "*" indicates current activity. ** indicates that a partner will be a consortium leader for this activity.

WP 1 Video Surveillance and Monitoring

This workpackage concerns semi-automated surveillance involving closed circuit TV. Three kinds of activity are addressed:

1. Critical Event Detection. The detection of critical events, typically involving a high security risk.

2. Process Monitoring. Monitoring of autonomous processes to ensure that pre-specified schedules of activities and events are followed.

3. Data Collection. The collection of statistics on the movements of objects for subsequent off-line analysis.

Workpackage 1 is composed of three tasks which represent critical technologies for surveillance applications. The objective of this workpackage is to develop and demonstrate component technologies which will enable the use of computer vision to automate surveillance tasks).

Task T1.1 Tracking Human Motion.

Observing humans presents a special challenge for computer vision. Techniques include the use of principal components analysis of contours from a sequence images of humans walking (the eigen-snakes), the use of principle components of the images of forms for recognition of actions and gestures, and real time tracking based on correlation, snakes, and other image processing techniques.

T1.1 Tracking Human Motion.                    
INPG    ROBO    DIST    AITEK   SSSA    CSATA   LEEDS   UR      TCD     AUC     ISR     JRC     EDIN    
**              **      *               *       **      *       *       *       *

Task T1.2 Recognising complex events from video surveillance

This task is concerned with methods which permit a surveillance operator to interactively specify sequences of detections which can have special significance. Such sequences, with alternate paths, can constitute behaviours to which the operator wishes certain reactions. Such reactions may range from simply notifying the surveillance operator, to automatic reactions such as instructing the steerable camera to obtain close up images suitable for identification

T1.2 Recognising complex events from video           
surveillance                                         
INPG    ROBO    DIST    AITEK   SSSA    CSATA   LEEDS   UR      TCD     AUC     ISR     JRC     EDIN    
*               *       *                       **      **              *

T1.3 Intelligent surveillance networks

Remote supervision of public places is now emerging as a major application area for broad-band networks - both for security and safety purposes. However, the continuous supervision of video images is not economically feasible in any but the most safety-critical circumstances. This task includes integration of visual and non-visual sensors, and issues related to communication and distributed computing.

T1.3 Intelligent surveillance networks               
INPG    ROBO    DIST    AITEK   SSSA    CSATA   LEEDS   UR      TCD     AUC     ISR     JRC     EDIN    
*               **      *               *       *       **              *               *       *

WP 2 Mobility for Surveillance and Monitoring

In the area of autonomous robotics, task level planning and execution have recently been demonstrated in prototype systems. More widespread application of these techniques requires real time sensing abilities similar to those developed for monitoring and surveillance. Technologies for teleoperation and tele-surveillance have continued to progress due to advances in available computing power and refinements in the electro-mechanical components. Adaptation of action level autonomy and binocular camera heads offer improvements in robustness and ease of use of tele-operated and semi-autonomous systems.

Task T2.1 Mission planning and trajectory control for surveillance robots.

Mission planning is typically formulated as the decomposition of a set of task statements into a sequence of actions. The tasks are treated as goals represented as states of the external world, and actions are treated as state transitions. In the real world, an autonomous robot must be able to use sensors to guide its actions and to monitor the execution of its actions to verify that the specified goal state has been achieved. When actions are not possible, or do not achieve the expected state, the system must be able to regenerate its plan. This plan regeneration often must involve managing constraints of energy and time. Research groups within the SMARTnet have been able to demonstrate systems for such plan execution dynamic replanning under constraints and using sensors.

Work in this area is guided by a set of reference applications. Underwater surveillance robots present an activity of growing importance for the SMART network. Other applications include a powered wheelchair for handicapped, and a semi-autonomous mobile surveillance robot.

T2.1 Planning and trajectory control for mobility    
INPG    ROBO    DIST    AITEK   SSSA    CSATA   LEEDS   UR      TCD     AUC     ISR     JRC     EDIN    
*       *       **      *       **                                      *       **      *       *

T2.2 Selective perception for autonomy

Recent research has indicated that there are significant computational advantages to purposive/task specific sensing over traditional reconstructive sensing, even where the diversity of purposes for real applications and potential concurrencies are taken into account. The formulation of purposive sensing strategies has so far focussed on signal level methods, which includes tuning of sensory parameters to 'filter' information prior to internal data processing. It is however anticipated that similar computational advantages may be obtained further into the processing chain. An obvious approach to such research could be formulation of virtual sensors (and actuators). Using a methodology similar to active sensing/perception it might now be possible to formulate control strategies that indicate controllability of the applied methods.

T2.2 Selective perception for autonomy               
INPG    ROBO    DIST    AITEK   SSSA    CSATA   LEEDS   UR      TCD     AUC     ISR     JRC     EDIN    
*               **              *                       +               **      **              **

T2.3 Learning navigation procedures for robust task achievement

Programming a robot requires specifying how the robot should react to all possible situations. For an industrial manipulator, the workspace can be controlled to restrict the situations to a manageable set. This is not possible for a mobile robot which must operate in a partially structured environment. Thus, robust task achievement requires the ability to automatically acquire operational knowledge from experience.

T2.3 Learning navigation             
procedures                           
INPG    ROBO    DIST    AITEK   SSSA    CSATA   LEEDS   UR      TCD     AUC     ISR     JRC     EDIN    
**              *               *       *                                       *       **

WP 3 Applications of Techniques for Surveillance and Monitoring

The construction of working systems has raised important issues in systems integration, control and performance evaluation. At the same time, the technologies for tracking people have raised new possibilities for man machine interaction. The tasks in work package three have been designed to address these areas.

Task T3.1 Vision as a man-machine interaction modality.

Vision techniques developed for video-surveillance can also be used to enhance the synergy between man and machine. In particular, computer vision can be used as a source of a number interaction modalities. Computer vision can track the position of the head of a user, thereby adapting the orientation of a remote camera, to create the illusion of looking through a computer screen as if it were a physical window. Keeping a face centred in the camera can also be useful for video communications and for video conferencing. Watching the lips can be used to trigger speech recognition. Computer vision can also be used to track the hands and objects manipulated by the user. Tracking fingertips allows any finger to serve as a pointing device or mouse.

T3.1 Vision as a man-machine interaction       
modality.                                      
INPG    ROBO    DIST    AITEK   SSSA    CSATA   LEEDS   UR      TCD     AUC     ISR     JRC     EDIN    
**                      *       *       *       **              *                       *

Task T3.2 Systems Integration, Cooperation and Control

Integration of techniques into systems requires methods for controlling perceptual processes for visual interpretation of scenes for surveillance applications. The workpackage will make use of the results in model based tracking so that specific application domain knowledge can be used both to improve the efficiency and effectiveness of scene interpretation and to generate descriptions of the scene relevant to event detection. This task will also adapt the work conducted in control of steerable camera heads for surveillance applications. The main technical development will be the integration of information from wide-angle cameras and steerable camera heads. The wide-angle cameras will provide the visual context within which the steerable camera heads will provide detailed close-up images. Also included in this task is work on cooperation between multiple surveillance robots, and between surveillance "agents".

T3.2 Systems Integration, Cooperation and Control.   
INPG    ROBO    DIST    AITEK   SSSA    CSATA   LEEDS   UR      TCD     AUC     ISR     JRC     EDIN    
*               *       *                               +       *               *       **

T3.3 Benchmarking and systems evaluation

The techniques used by the members of the network are based on different technologies and meet different application constraints. An important activity of the network will involve a systematic comparison of the techniques which have been developed by the partners in order to determine their relative strengths. An equally important activity will involve transferring this technology to the industrial partners through form of exchange of research and development personnel.

With the growth in available computing power it has become possible to build and experiment with systems architectures for mobile and fixed video surveillance systems. This, in turn, has led to experimental implementations using a number of novel architectural concepts. At the same time the growing demand for such surveillance systems has created a demand for information necessary for performance specifications. As leaders in this area, a number of partners have been called on as consultants to provide such information to industry. There is a growing interest among the partners to establish standard measures of system performance that can be used to specify commercial products and objectively compare performance. This task will provide a framework for exchanges in this area.

T3.3 Benchmarking systems       
INPG    ROBO    DIST    AITEK   SSSA    CSATA   LEEDS   UR      TCD     AUC     ISR     JRC     EDIN    
        *                               *               +               **      *       *

WP 4 Scientific and Industrial Dissemination

Scientific Workshops

Every six months, the SMART network organizes a scientific workshop at one of the partner sites. Transnational visitors who have been funded by the network in the previous six months are required to present their results at these workshops. In addition, network members make presentations of recent research results and research in progress. This has proven particularly useful for young researchers who are about to complete their thesis to present their results and make contact with sites which can offer them a post-doctoral internship. It is not unusual to invite participation by young scientists from outside the network who are interested in collaborative research with network partners. It has also occurred that established researchers have participated in these workshops in order to present recent results and learn from the network presentations. These workshops last 2 to 3 days and rotate between the partner sites. Such workshops have been held at Paris, Ispra, Bari and Lisbon .

Annual Symposium

In addition to the scientific exchange workshops, the network has created an annual symposium: The "Symposium on Intelligent Robotics Systems". This symposium lasts 5 days and is organised by partners in the SMART network within the European Union. An open call for papers is widely distributed each fall in major conferences as well as over the Internet. The program is composed by reviewing submitted extended abstracts. The most recent symposium have attracted sufficient submissions to become rather selective (50 % rejection). In 1994, the SMART network has invited the HEROS network to join with it in organising the the SIRS symposium. SIRS has also received a EuroConferences status, permitting it to award travel grants to young researchers to participate in the symposium. SIRS 93 was held in Zakopane Poland, SIRS 94 in Grenoble FR, and SIRS 95 in Pisa IT. SIRS '96 will be held in Lisbon PT. A special issue of the journal "Robotics and Autonomous Systems" is produced composed from the best papers from each of the SIRS symposia.

Technology Transfer

The SMART network includes two SME devoted to surveillance and monitoring technology. The partner AITEK is involved in a number of projects to equip public areas with surveillance and monitoring equipment. The partner ROBOTSOFT is one of the worlds leading manufacturer of mobile robotics platforms. Both partners participate actively in the network and exploit technology developed by the partners.

In addition, the network lists a number of corporate "Sponsors". Network sponsors are routinely invited to all network activity. The company ITMI, in Meylan France has been particularly active in participating in network activities. ITMI has recently built a corporate business plan based on producing automatic aids for video surveillance systems. ITMI has also demonstrated mobile robots for service applications. Other corporate sponsors include Ernitech in Denmark, a leading manufacturer of controllable optics for surveillance applications.

The Universities of Reading and Leeds have recently formed a Surveillance Systems Users group to accompany their national project funded by the EPSRC. This club includes the British home office (local police research agency) and Heathrow Airport. This users club will be invited to participate in the activities of the SMART II network.

B.3 Schedule and Milestones

The network will organise a review meeting at months 18 and 36. These review meetings will constitute the milestones at which the network progress may be assessed. Each review will be composed of overview presentations for each of the workpackages, as well as a review of the network activities. Information will be provided concerning transnational visitors, joint publications and joint projects which have been spawned by the network. Demonstrations of recent systems will presented using video-tape.

Network progress will be measured by

* Quality of Scientific and Technological Innovation

* Number of man-months of transnational visitors funded

* Number of joint publications

* Number of joint research projects launched

* Industrial applications resulting from the network activities.

B.4 Research Effort of the Participants

      Partners                                             Researchers     Researchers     
                                                           Financed from   Financed from   
                                                           SMART II        other sources   
                                                           (man-years)     (man-years)     
      ADR - GRAVIR, I.N.P. Grenoble                                  2.60  12              
      ROBOSOFT SA,                                                   2.66  9               
      DIST, Univeristy of Genoa                                      3.66  21              
      AITEK Srl                                                      3.83  12              
      Scuola Superiore S. Anna / ARTS Lab                            4.45  12              
      CSATA-Novus Orta, Technopolis Bari                             3.33  36              
      University of Leeds                                            3.00  12              
      University of Reading                                          3.00  18              
      Trinity College, University of Dublin                          2.60  9               
      Laboratory of Image Analysis, Aalborg University               2.50  9               
      Institute of Systems and Robotics                              4.00  45              
      EC Joint Research Center-ISPRA                                 3.00  9               
      University of Edinburgh                                        5.00  9

B.5 Network Organisation and Management

The administration of the SMART network is composed of a coordinator, a steering committee and a management board. The management board is composed of one member from each network partner. The management board meets at least once per year, and generally every six months, to approve the actions and budget of the network and to plan network activities and reports.

Day to day operations are carried out by the network coordinator, with the assistance of a steering committee. The steering committee is composed of one representatives from each country in the network, including the coordinator. The steering committee reports its activities to the management board for approval at least once per year. The steering committee members serve as national contact points for the network. The network coordinator serves as a communication channel between the EC program officer and network. The network coordinator is responsible administering the network "general fund" and will oversee collecting of financial and administrative reports for the commission in a timely manner.

The SmartNet organises an annual general meeting. This general meeting is different from the management board meetings, and is open to all network participants. General meetings occur in conjunction with scientific workshops and are designed to encourage interaction between participants and to communicate information about the network and its functions. The network will organises scientific workshops at six month intervals. Networks may be proposed and organised by any network partner. Funding for such workshops are provided by the partner with assistance from thee network general fund. All workshop funding will be approved by the steering committee.

The SMART management board serves as the management board for the annual Euro-Conference "SIRS"

B.6 Involvement of Industry

Industry is involved in the SMART network through participate of SME's as partners, through association of corporate sponsors, and through the organisation of the industrial users club.

Participation of SME's

The SMART network contains the SME's RobotSoft S.a.r.l and AITEKL S.r.l. as partners. More than half of the project partners use the RobotSoft Robuter as an experimental mobile platform. The network has made it possible to partners to send young researchers to RobotSoft for visits of 3 months to use the robotsoft facilities to test ideas for novel hardware and systems. At the same time this has permitted RobotSoft engineers to tighten their communications with their user community.

AITEK S.r.l. is involved in building surveillance and mobile systems for industry. SMART postdoctoral fellows have worked on applications of their research at AITEK.

Corporate Sponsors

Corporate sponsors are large established corporations who participate in SMART workshops and conferences at their own expense. Corporate sponsors are involved in SMART net because the network technology has potential uses in products targeted by their corporate business plans.

End-Users Club

The partners UReading, AUC and INPG have recently formed an industrial club of end-users of video surveillance technology in order to spread awareness of emerging technologies for video surveillance and monitoring.

Part C - Training

C.1 Employment of Young Researchers

The majority of the network budget is devoted to transnational exchanges. Each of the partners is allocated approximately 36 man-months of salary to be used on category 20 / 30 transnational visitors during the 36 month duration of the SMART II contract. This makes a total of 468 man-months of transnational exchanges during the 3 year contract period.

C.2 Training Programme

The network contributes to the training of researchers in the following ways:

1) By organising semi-formal workshops where doctoral students and young post-doctoral researchers can present and discuss recent research results and research in progress.

2) By organising an annual EuroConference where young researchers can present their research results in a formal setting.

3) By financing transnational exchanges of doctoral students and young post-doctoral researchers between the network partners (including industry).

The following table shows the distribution in man-months between young post-doctoral researchers and doctoral students for each partner.

      Partner                                              Doctorant       Post-doctoral   
                                                                           Researcher      
      INPG                                                             20  12              
      ROBOTSOFT SARL                                                   12  20              
      UNIVERSITA DI GENOVA                                             12  32              
      AITEK S.R.L.                                                     31  15              
      SCUOLO S.ANNA SUPER.                                             36  17.5            
      TECNOPOLIS CSATA NOVUS ORTUS                                     20  20              
      UNIVERSITY OF LEEDS                                               0  36              
      UNIVERSITY OF READING                                             0  36              
      DUBLIN UNIVERSITY                                                 0  32              
      UNIVERSITY OF AALBORG                                            15  15              
      INSTITUTO DE SISTEMAS E ROBOTICA                                 36  12              
      JOINT RESEARCH CENTRE                                            36  0               
      UNIVERSITY OF EDINBURGH                                          36  24              
      Total                                                           254  271.5

SMART II PROJECT PROGRAMME

Contents