A Kuriakidou, E Filtzantzidou, P Balaouras,I Roussakis, C. Mouzakis, I Stavrakakis (2001)

A video gateway between the next generation broadband Internet and
the current narrow band Internet for live and on-demand access
IST – 1999 – 10160
Title Academic Distance Learning Trial
Version 1.2
Deliverable # 15.4
Author
A Kuriakidou, E Filtzantzidou, P Balaouras,I
Roussakis, C. Mouzakis, I Stavrakakis
Company UoA
Date 30/01/2002
Filename UoAwp6-del15.4-v1.2.doc)
Security* Int#
* Int = Internal circulation within project (+ EC PO if requested)
Rest = Restricted circulation list (specify in footnote) and EC PO only
IST = Circulation within IST Programme participants
FP5 = Circulation within Framework Programme participants
Pub = Public document
#
FOOTNOTE: space for circulation list if required (delete this otherwise)

TITLE Academic Distance Learning Trial
VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 2 of 49
TABLE OF CONTENTS
INTRODUCTION 3
Description of the application 3
ROADMAP TO TRIALS 4
Trial design and pedagogical evaluation methology 4
Testing equipment 4
Setting the laboratory environment for Scenarios II-IV. 4
DESCRIPTION OF TRIALS 5
SCENARIO I 5
TRIAL IA 6
Technical Configuration 9
TRIAL IB 10
TRIAL IC 12
ScENARIOS II, III 15
Results of Scenario II & III 17
SCENARIO IV: PERFORMANCE EVALUATION 17
Performance of the MGW-2000 System 17
Performance of the MPEG-4 Transmission Server and ISO MPEG-4 Player 19
Results of Scenario IV 20
TRIALS THEORY AND RESULTS 21
A. – Learning and Instruction – Some Preliminary
Remarks 21
B. – Pedagogical Evaluation of Distance Education –
Forms and Criteria Applied in the Trials 22
C. – Evaluation Results 25
C.1 – Evaluation of Pedagogy / Instruction 25
C.2 – Pedagogical / Perceptual Evaluation of Technical Features 27
C.3 – Scenic Direction 31
C.4 - A statistical analysis of the factors affecting the satisfaction of the participants32
GENERAL SUGGESTIONS – CONCLUDING REMARKS 33
APPENDIX QUESTIONNAIRE 35

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 3 of 49
INTRODUCTION
This deliverable is the evaluation report of the UoA trials.
Description of the application
UoA have conducted a series of distance learning trials in which students participated as well
as testing trials executed in the laboratory. These trials have implemented four (4) different
scenarios. The objective of the distance learning trials was to evaluate VideoGateway’s ability
to provide live lecture material to geographically disparate users at rates ranging from as low as
500 kbps for desktop users up to 4 Mbps for classroom users. The testing trials aimed to:
 (a) evaluate the VideoGateway system performance bounds, the MPEG-4 transmitter
and the associated players
 (b) verify the transcoding and rate adaptation features which were part of the R&D
effort of the project.
Let us introduce the scenarios:
The first scenario – scenario I – was focused on the distance learning trials in which more than
120 undergraduate and graduate students, research and technical staff participated. This
scenario utilised the encoding features – PAL to MPEG-1, MPEG-2 and MPEG-4 streams of
the VideoGateway project products.
The trials have engaged campus based students who attended live academic lectures of both the
undergraduate and postgraduate programmes of the Department of Information and
Telecommunications. Technical staff of the UoA’s Network Operation Centre participated in
these trials as well. The population of users for each trial test is illustrated in the tables that
accompany each trial scenario.
The MGW-2000 and MPEG-4 transmitters were used to deriver the various streams in terms of
encoding formats and rates. Since it was hard to simultaneously broadcast a lecture and co-
ordinate the large number of involved users/students, it was decided to carry out three (3) trials
focused on scenario I, instead of one (1) as originally planned, to better manage the trials
themselves.
The evaluation of all the trials of scenario I was based on user input (perceptual) through
appropriate forms.
The second scenario – scenario II – was combined with the third scenario – scenario III – and
aimed to evaluate the transcoding and rate adaptation features of the Transcoding technology
developed in the VideoGateway project. This scenario was implemented in a laboratory
environment.
The fourth scenario – scenario IV – was carried out in the laboratory environment aiming to
stretch and investigate the system performance limitations of the encoding features
Deliverable 15.5 provides a common summary of all the VideoGateway project trials.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 4 of 49
ROADMAP TO TRIALS
The actions taken to prepare the trials are briefly reported below:
TRIAL DESIGN AND PEDAGOGICAL EVALUATION METHOLOGY
 Contact Prof. D. Mathaiou and his research team – Department of Primary Education,
UoA – for preparing the pedagogical evaluation methology and the questionnaire.
 Fine tuning of the trial settings.
 Determination of the required equipment and ordering it from Optibase.
 Contact and invite partners from GUnet to participate in the trial (Athens University
of Economics and Business, National Technical University of Athens, Aristotle
University of Thessaloniki).
 Redesign the trials according to the project progress, results and products.
 Select lectures for delivery.
 Select users.
TESTING EQUIPMENT
 Communication with Optibase for equipment shipment.
 Communication with Greek Customs to get the equipment.
 Using, managing and testing the equipment.
 Integrating the equipment with the UoA teleteaching classroom infrastructure.
SETTING THE LABORATORY ENVIRONMENT FOR SCENARIOS II-IV.
 Install a linux based routing system with 3 NICs.
 Install and test for enabling multicast routing.
 Install the NIST emulator.
 Test the rate adaptation algorithm before the integration with Optibase’s transcoder.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 5 of 49
DESCRIPTION OF TRIALS
In this section the trials are organised in Scenarios (I to IV). The Scenarios described are the
following:
I) Encoding Features
II) Transcoding Features
III) Rate Adaptation
IV) System Stretching.
For each Scenario, a number of different trials and tests were executed. The Scenarios (trials
and tests) are described in terms of equipment used, network, stream parameters, etc.
SCENARIO I
As mentioned in the Introduction, the first scenario – scenario I – was focused on the distance
learning trials. Instead of simultaneously broadcasting a lecture at different rates and due to the
complication of co-ordinating the large number of involved users/students, it was decided to
carry out three different (3) trials focused on scenario I (Figure 1). Let Ia, Ib and Ic refer to
these trials.
Scenario I –Scenario I – utilising encoding featuresutilising encoding features
UoA classr oom
VG
Video
Matrix
PA L
M PEG-1, MPEG-4
to Desktop users at 500 kbps
P ict ur eT e l
P ict ur eT e l
P ict ur eT el
MPEG-1, MPEG-4
to Desktop users at 1 Mbps
P ic tu re T el
P ic tu re T e l
P ic tu re T e l
MPEG-2, MPEG-1, MPEG-4
to Classroom
AUEB classr oom
Ia
Ib
Ic
Figure 1 Trials Ia, Ib and Ic of Scenario I
The aim of Scenario I was to investigate the potential of exploiting products that deliver
MPEG-2, MPEG-1 and MPEG-4 streams for the deployment of distance learning services. The

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 6 of 49
targeted users are students located at remote classrooms and students at remote PC laboratories
(desktop users). Thus, the technical part of scenario I utilised the encoding features – PAL to
MPEG-1, MPEG-2 and MPEG-4 streams of the Video Gateway project products. Let us
examine each distance learning trial separately:
 Trial Ia was focused on a lecture delivered to a remote classroom. The video and
audio streams of the instructor were transmitted as MPEG-2, MPEG-1 or H.323
compliant streams. The presentation slides were in electronic form and separately
transmitted to the remote classroom via the network as well. An H.323 audio-visual
channel was used as return channel for permitting interaction between the two
classrooms.
 Trial Ib was focused on a lecture delivered to a remote classroom, as well. The video
and audio streams of the instructor were transmitted as MPEG-1, and MPEG-4. The
presentation slides were in electronic form and separately transmitted to the remote
classroom via the network as well. No return channel for permitting interaction
between the two classrooms was available. Trial Ib, in comparison to trial Ia, engaged
in addition the MPEG-4 format. The MPEG-1 stream in trial Ib was transmitted at a
lower rate than the rate of the MPEG-1 transmission in trial Ib.
 Trial Ic was focused on a lecture delivered to desktop users. The video and audio
streams of the instructor were transmitted as MPEG-1 and MPEG-4 but in lower rates
than the trial Ib. The presentation slides were printed-out and provided to the students
before the beginning of the transmission. The presentation slides were not transmitted
separately in electronic form but they were part of the video view that was transmitted
No return channel was available.
The trials have engaged campus based students who attended live academic lectures of both the
undergraduate and postgraduate programmes of the Department of Information and
Telecommunications. Technical staff of the UoA Network Operation Centre participated in
these trials as well. The population of users for each trial test is illustrated in Table 1.
The evaluation of all the trials of scenario I was based on user input (perceptual) through
appropriate forms. The detailed description of each trial of scenario I is presented below.
Table 1: Participants
Trial Number of
participants
Ia 42
Ib 60
Ic 22
Total 124
TRIAL IA
Trial Ia was held on 12th July 2001. Dr. Xandrinos started his lecture at 1:30 pm and lasted for
1 hour and 30 minutes. The lecture took place in the UoA teleteaching room and was
transmitted to a similar room in the Athens University of Economics and Business (AUBE). 18

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 7 of 49
students were located in the instructor’s room and 24 students in the remote room. The lecture
was divided in three sections, each of duration of 30 minutes:
 Section A. The instructor’s video and audio were transmitted as multiplexed MPEG-2
stream at a rate of 4 Mbps. The presentation slides were shared via the MS
NetMeeting – a T.120 Application Sharing software.
 Section B. The instructor’s video and audio were transmitted as multiplexed MPEG-1
stream at a rate of 1.5 Mbps.
 Section C. The instructor’s video and audio were transmitted by utilising H.323
audio-visual systems at a rate of 1.5 Mbps.
The software and equipment used during Trial Ia was the following.
Equipment:
 Optibase – MGW-2000, VideoPlex MPEG-1 and MPEG-2 hardware decoding card.
 Other – VCON’s H.323 audio-visual terminal
Software:
 Optibase – MGW-2000 Director, ComMotion Receiver 4.0
 Other – MeetingPoint VCON, MS NetMeeting
In more detail, the MGW-2000 was used to transmit the MPEG-1 and MPEG-2 streams from
the UoA classroom to the AUEB classroom. The VideoPlex hardware decoding card and the
ComMotion Receiver 4.0 player were used to receive the aforementioned streams and decode
them into analogue form. VCON’s H.323 audio-visual terminal was used during the entire
session as a return channel to enable the communication between the two classrooms and as the
main channel as well during section C. The software of MS-NetMeeting (T.120 application
sharing session) was used in order for the instructor to share his presentation slides with the
remote classroom. For the application sharing session two Personal Computers were used.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 8 of 49
TrialTrial IaIa: 12/07/2001 Dr.: 12/07/2001 Dr. XandrinosXandrinos
UoA classr oom
Video
Matrix
PA L
M PEG-1 and M PEG-4
to Desktop users
P ic tu reT e l
P ic tur eT e l
P ic tur eT e l
MPEG-1, MPEG-4 stream
to Desktop users
P ic tu re T el
P ic tu re T e l
P ic tu re T e l
MPEG-2@ 4M bps,
MPEG-1@ 1.5 Mbps
H.263 @ 1.5 Mbps to Classroom
AUEB classr oom
24 Students
MGW2000
Figure 2: Trial Ia
155 ATM
UoA
C isco LS1010
C isco 7513
H ercule s
Fore A SX-1000
Tele -education class room
@ Dept. of Infor matics
Tele -education classroom
@ AU EB
AUEB
GRNet
Fore A SX-1000
12 M bps
A UEB
m ain router
8 M bps
Figure 3: Network Infrastructure

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 9 of 49
Technical Configuration
Section A.
The instructor’s video and audio were transmitted in MPEG-2 format. The MGW 2000 Director
configuration was set as follows:
Source
Video Input Composite
Hue
Saturation 128
Brightness 128
Contrast 128
Horizontal Offset 0
Video Horizontal Filter 6
Video Vertical Filter 7
Audio Input Unbalanced
Audio Gain 3
Stream
Colour System PAL
Encoding Mode Full D1
Stream Format MPEG Transport
Send System Header 0
Target Bit Rate 4 Mbps
Video Frame Sampling Full
Audio Encoding Mode Stereo
Audio Bit Rate 192 Kbps
Audio Sampling Rate 44100 Hz
Target
Current Target 1
Target Type UDP
Target Address Remote room’s address (unicast)
Port 11111
UDP Block Size 1024
TTL 10
The students’ video and audio were transmitted with the H.323 terminal at 384 Kbps (video)
and 128 kbps (audio). The instructor’s material was sent separately with a T.120 application.
(NetMeeting).
Section B
The instructor’s video and audio were transmitted in MPEG-1 format. The MGW-2000
Director configuration was set as before with the only difference in the Stream format – MPEG
System.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 10 of 49
Section C
The instructors’ video and audio were transmitted with the H.323 terminal at 1.5 Mbps for the
video stream and 128 kbps for the audio.
TRIAL IB
Trial Ib was held on 11th October 2001. Prof. Stavrakakis started his lecture at 10:30 pm and its
duration was 1 hour. The lecture took place in the UoA teleteaching room and was transmitted
– over a loaded network – to another UoA classroom equipped with audio-visual equipment. 10
students were located in the instructor’s room and more than 50 students in the remote room.
The lecture was divided in two sections of duration of 30 minutes:
 Section A. The instructor’s video and audio were transmitted as split MPEG-4
streams at a rate of 1 Mbps. The presentation slides were shared via the MS
NetMeeting, a T.120 Application Sharing software.
stream at a rate of 1 Mbps. The presentation slides were shared via the MS
NetMeeting, a T.120 Application Sharing software.
TrialTrial IbIb: 11/10/01: 11/10/01 ProfProf. I.. I. StavrakakisStavrakakis
UoA classr oom
Video
Matrix
PA L
MPEG-1, MPEG-4
to Desktop users
P ic tu re T el
P ic tu re T e l
P ic tu re T e l
MPEG -1@ 1 Mbps
MPEG -4 @ 1 Mbps to C lassroom
UoA classroom
50 StudentsMGW2000
MPEG-4 Transmission
Server
Figure 4: Trial Ib
The software and equipment used during Trial Ib was the following.
Equipment:
 Optibase – MGW-2000, MPEG-4 Transmission Server.
 Other – Audio-visual equipment

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 11 of 49
Software:
 Optibase – MGW-2000 Director, ComMotion Receiver 4.0, ISO-MPEG-4 player
 Other – MS NetMeeting
In more detail, the MGW-2000 was used to transmit the MPEG-1 stream from UoA’s
teleteaching classroom to the other classroom. The ComMotion Receiver 4.0 player was used to
receive the MPEG-1 stream and decode it. The stream was rendered in its digital form and not
in analogue form as in Trial Ia. The MPEG-4 Transmission Server was used to transmit the
MPEG-4 stream from the UoA teleteaching classroom to the other classroom. The ISO-MPEG-
4 player was used to receive the MPEG-4 stream and decode it. The stream was rendered in its
digital form. The software of MS-NetMeeting (T.120 application sharing session) was used in
order for the instructor to share his presentation slides with the remote classroom. For the
application sharing session two Personal Computers were used.
Section A.
Director configuration was set as follows.
Source
Hue
Saturation 128
Brightness 128
Contrast 128
Horizontal Offset 0
Audio Gain 3
Stream
Color System PAL
Encoding Mode SIF
Stream Format MPEG System
Target Bit Rate 1 Mbps
Target
Current Target 1
Target Type Multicast
Target Address 224.1.1.1
Port 11111
UDP Block Size 1024

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 12 of 49
TTL 10
Section B
The instructor’s video and audio were transmitted in MPEG-4 format. The configuration was
set as follows:
Description Teleteaching
RTP Address 224.1.1.77
Port 11200
Preview File Video Optibase4.mp4
Preview Track 1
Video Live Encoding Parameters:
Video Capture Device Video Device 1
Bitrate (bps) 1000000
Intra Period 30
Picture Description SIF
Frame per Second 25
Audio Live Encoding Parameters:
Audio Capture Device Audio Device 1
Bitrate (bps) 64000
Sample per Second 44100
Channel Mode Stereo
TRIAL IC
Trial Ic was held on the 2nd
November 2001. A pre-recorded lecture (the lecture of Prof.
Stavrakakis in trial Ib) was transmitted in two labs of the Department of Informatics and
Telecommunications and to the Network Operation Centre. The transmission started at 15:00
and lasted about 1 hour. There were 20 participants, postgraduate students and research and
technical staff. The lecture was divided into two sections of duration 30 minutes:
 Section A. The instructor’s video and audio were transmitted as split MPEG-4
streams at a rate of 500 kbps. The participants received the lecture with the ISO
MPEG-4 Player.
stream at a rate of 500 kbps. The participants received the lecture with the
ComMotion Receiver.
The presentation slides were not shared via the MS NetMeeting as in the previous trials but
instead the users downloaded and printed the slides before the transmission.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 13 of 49
TrialTrial IcIc: 2/11/01: 2/11/01 ProfProf. I.. I. StavrakakisStavrakakis
UoA classr oom
Video
Matrix
PA L
20 Particip ants
MGW2000
MPEG-4 Transmission
Server
MPEG-1, MPEG-4
to Desktop users at 500 kbps
P ict ur eT el
P ict ur eT el
Pict ur eT el
Figure 5: Trial Ic
The software and equipment used during Trial Ic was the following.
Equipment:
 Optibase – MGW-2000, MPEG-4 Transmission Server.
 Other – Audio-visual equipment
Software:
 Optibase – MGW-2000 Director, ComMotion Receiver 4.0, ISO-MPEG-4 player
In more detail, the MGW-2000 was used to transmit the MPEG-1 stream from the UoA
teleteaching classroom to the desktop PCs. The ComMotion Receiver 4.0 player was used to
receive the MPEG-1 stream and decode it. The MPEG-4 Transmission Server was used to
transmit the MPEG-4 stream from the UoA teleteaching classroom to the other classroom. The
ISO-MPEG-4 player was used to receive the MPEG-4 stream and decode it.
Section A.
The instructor’s video and audio were transmitted in MPEG-4 format. The configuration was
set as follows:
Description Teleteaching
RTP Address 224.1.1.77
Port 11200
Preview File Video Optibase4.mp4
Preview Track 1
Video Live Encoding Parameters:

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 14 of 49
Video Capture Device Video Device 1
Intra Period 30
Picture Description SIF
Frame per Second 25
Audio Live Encoding Parameters:
Audio Capture Device Audio Device 1
Sample per Second 44100
Channel Mode Stereo
Section B
Director configuration was set as follows.
Source
Hue
Saturation 128
Brightness 128
Contrast 128
Horizontal Offset 0
Audio Gain 3
Stream
Colour System PAL
Encoding Mode SIF
Target Bit Rate 500 kbps
Target
Current Target 1
Target Type Multicast
Target Address 224.1.1.1
Port 11111
UDP Block Size 1024
TTL 10

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 15 of 49
SCENARIOS II, III
The scenario II, as defined in WP2, aimed to utilise the transcoding features of the Video
Gateway, that were the “on the fly” transcoding of MPEG-1 streams to MPEG-4 streams.
During the project it became clear to the partners that the stream transcoding feature would not
give any additional advantages from the user/student point of view in comparison to the trials of
Scenario I, where MPEG-4 was evaluated. Thus it was decided that the scenario II to
demonstrate the transcoding technology developed in the context of the VideoGateway project
by the real time transcoding of MPEG-1 files to MPEG-4 stream.
Pictu reTel
1. Uo A Dum p clie nt,
2. Bit R ate Verifie r
M P EG -1 file
M P EG -1 to M PEG-4
rate adaptable transcoder
PictureT el
M PEG -4
pla yer
linux system
NIST emulator
mrouted (for mu lticast)
IP m ulticast add res s: 2 24.1.1.77
Vid eo p ort: 1 1200
O D port: 11130
Dem o targets:
- Transcoding MPEG-1 to MPEG-4
- Transcoding & R ate Adaptation of MPEG-4 output streams
PC1
PC2
PC3
PC4
Figure 6: Transcoding and rate adaptation trial
Scenario III aimed to demonstrate the rate adaptation features of the MPEG-4 sources by
incorporating the R&D results of Optibase and UoA, concerning the rate adaptation algorithms
both at the encoding/transcoding and the network side.
Scenarios II and III were combined and a single trial was conducted using an MPEG-1 to
MPEG-4 transcoder capable of producing rate adaptable MPEG-4 video streams.
The architecture of the trial is that presented in Figure 6.
Optibase has provided the MPEG-1 to MPEG-4 transcoder - based on a transcoding engine
which is capable of transcoding MPEG-1 files to an MPEG-4 split audio and video stream. The
transcoder broadcasts the two streams over multicast IP. The streams are transported via
RTP/UDP/IP packets.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 16 of 49
The ISO MPEG-4 player listens to a specific multicast IP address and port, receives the MPEG-
4 RTP/UDP/IP packets and decodes the video stream, which is displayed on the monitor. This
player does not have the capability of reporting the packet losses due to network congestion by
sending back Real Time Control Packets (RTCP) to the transcoder.
That is why UoA provided the UoA-dump client which is capable of receiving the video stream
(but not decoding) and reporting back to the transcoder the network status by RTCP Receiver
Reports (RR). The transcoder receives the RTCP RR from the UoA-dump client and adapts its
transmission rate based on the rate adaptation algorithm provided by UoA. This algorithm
implements a multi-state congestion control based on the fractional packet loss reported by
RTCP RR and the history. A detailed description of the multi-state congestion control
algorithm may be found in a paper which is part. The trials have been carried-out in live
academic lectures from both the undergraduate and the postgraduate programmes of the
Department of Information and Telecommunications to campus based students. Technical staff
of the Network Operation Centre of UoA also participated in these trials. The population of
users for its trial test is illustrated in Deliverable 9 (R&D results).
The adaptation of the transmission rate is verified by using the Bit Rate Verifier, which reports
the mean rate of the stream every 3 seconds.
During the trial the transcoding and rate adaptation features were tested in the UoA laboratory.
The transcoder, ISO MPEG-4 player, UoA-dump and Bit Rate Verifier client were installed in
three Personal Computers (PC1, PC2, PC3) running Win2k OS, that were connected to a linux
router system equipped with three NICs. The network emulator software NIST was installed.
This software was used to introduce packet losses that triggered the rate adaptation.
Pictu reTel
1. UoA Dum p clie nt,
2. Bit R ate Verifie r
M P EG -1 file
M P EG -1 to M PEG-4
rate adaptable transcoder
PictureT el
M PEG -4
pla yer
linux system
NIST emulator
mrouted (for mu lticast)PC1
PC2
PC3
PC4
RTP
RTCP R R
packet
losses
Figure 7: RTP/RTCP Packets

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 17 of 49
Results of Scenario II & III
The conclusions drawn from the execution of scenarios II & III were that both the transcoding
and rate adaptation features of the Transcoder are working properly. Another preliminary
conclusion is that indeed the MPEG-4 encoding scheme tolerates a greater fractional packet
loss in comparison to the MPEG-1 encoding scheme.
SCENARIO IV: PERFORMANCE EVALUATION
The objective of scenario IV was the stretching and performance evaluation of the MGW-2000
system, MPEG-4 Transmission Server and the ISO-MPEG-4 player components. Since, the
MPEG-1 player (ComMotion Receiver) is a well established product in the market, no
evaluation was carried out for it. It was important to stretch and evaluate the three
aforementioned systems because they may constitute basic components to an architecture that
would deploy live distance learning scenarios.
Performance of the MGW-2000 System
IP
Pic tureTel
P ictureTe l
Video
M atrix
PAL
PictureT el
MGW2000
Figure 8: Stretching of MWG-2000
Test 1: MPEG-1 transmission over a 10 Mbps port connection
The MGW 2000 was set to transmit three live MPEG-1 streams. Only one NIC was used for the
transmission and it had access to a 10 Mbps switch. The following table shows the achieved
frame rates for different values of the target bit rate.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 18 of 49
TARGET BIT RATE FRAME RATE DELAY (IN SECONDS)
3 sources at 2 Mbps 24.7-24.4 1
3 sources at 3.5 Mbps 14.5-15.5 2
1 source at 4 Mbps 24.6 1
2 sources at 4 Mbps 22.5 1.5
3 sources at 4 Mbps 10-12 4
When all sources were transmitted at 4 Mbps the system failed to respond after five minutes of
use and had to be reset. The problem was due to the fact that the aggregate output traffic rate of
the MGW-2000 was greater than that enabled by the port of 10 Mbps.
Test 2: MPEG-1 transmission over a 100 Mbps port connection
After the first test we used a 100 Mbps switch and repeated the same tests. The table below
shows these results.
TARGET BIT RATE FRAME RATE DELAY (IN SECONDS)
3 sources at 3.5 Mbps 24.4-24 1
3 sources at 4 Mbps 24.7-24 1
Thus when a proper network connection (100 Mbps) was used, no problem appeared in the
performance of the MGW-2000 system.
The MGW-2000 configuration of tests 1 and 2 is described in the tables below.
Source
Hue
Saturation 128
Brightness 128
Contrast 128
Horizontal Offset 0
Audio Gain 3
Stream
Colour System PAL
Encoding Mode SIF
Target Bit Rate 2, 3, 3.5 and 4 Mbps

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 19 of 49
Test 3: MPEG-2 transmission over 10 and 100 Mbps port connections
The MGW-2000 was set to transmit three live MPEG-2 streams. At first the NIC had access to
a 10 Mbps switch and the results were similar with the first test (MPEG-1 transmission). When
the MGW-2000 had access to a 100 Mbps switch we were able to transmit up to 5 Mbps
successfully.
Performance of the MPEG-4 Transmission Server and ISO MPEG-4 Player
Due to its nature this trial was performed in the laboratory. Figure 9 illustrates the trial
architecture. The major components used were:
 The MPEG-4 transmitter server run on a Personal Computer equipped with dual
processors at 800 MHz, 128 MB RAM and the OSPREY 200 video grabbing card.
The Win2k server OS was installed.
 The receiver was a PC with a single Pentium III at 400 Mhz, 128MB memory and the
Windows Professional 2000 operating system. The ISO-MPEG-4 player was installed
in the receiver PC. The duration of each test was fifteen minutes.
The Transmitter server and receiver were connected over a 10 Mbps connection.
IP
PictureT el
PAL
MPEG-4 Transmission Server
ISO MPEG-4 Player
Receiver
Figure 9: Stretching of MPEG-4 Transmission
Server and MPEG-4 ISO Player
The performance evaluation test aimed to:
 evaluate the perceived quality and delay of the MPEG-4 video at different bit rates at
the receiver
 record resource usage such as CPU time and memory usage at both the transmitter
(MPEG-4 encoding process) and the receiver (MPEG-4 decoding process).
It was noted that the decoding process proceeded without any problem for the bit rates of 0.5,
1.0, 1.5 and 2.0 Mbps. Beyond the bit rate of 2 Mbps the receiver (MPEG-4 player) processing
power was not adequate to successfully decode the MPEG-4 stream (see also Table 3).
The perceived video quality at the receiver was excellent. The overall delay due to the
encoding, transmission, decoding and rendering processes when movement occurred was about

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 20 of 49
1.8 seconds for bit rates of 0.5, 1.0 and 1.5 Mbps. At 2.0 Mbps the delay was about 2.1 seconds.
At higher bit rates the MPEG-4 decoder and encoder were not capable of performing the
equivalent processes. Table 3 presents the measurements for the perceived quality, the delay of
the MPEG-4 video and resource usage at the receiver. The perceptual quality is measured with
a scale from 1 to 5 ( Table 2).
Table 2:Quality measures
Excellent Quality 5
Very Good Quality 4
Good Quality 3
Bad Quality 1,2
Table 3: Results at the Receiver
BIT RATE (MBPS) 0.5 1.0 1.5 2.0
Perc. Quality 5 5 5 5
Delay (second) 1.8 1.8 1.8 2.1
CPU Time (%) 65 75-83 84-89 95-99
Memory Usage (K) 43.448K 41.9K 38.748K 44.1K
At the receiver the CPU time was increasing gradually with the bit rate. At 2.0 Mbps the
decoding process was using 99% of the CPU time. The receiver was not able to handle bit rates
greater than 2.0 Mbps. The memory usage varied between 38.7K and 44.1K.
At the transmitter the above parameters hardly changed with the bit rate increase. The CPU
time of the decoding process varied between 35 and 37 percent and only at the bit rate of 2.0
Mbps was there a slight increase and the CPU time was between 39 and 43 percent. The
memory usage parameter varied between the 19.4K and 19.6K. Table 4 presents the CPU time
and memory usage at the transmitter side for bit rates of 0.5, 1.0, 1.5 and 2.0 Mbps.
Table 4: CPU time and Memory Usage of
Transmitter
BIT RATE (MBPS) 0.5 1 1.5 2
CPU Time (%) 35-37 34-38 35-37 39-43
Memory Usage (K) 19.4K 19.4K 19.5K 19.6K
Results of Scenario IV
The conclusion of this trial is that the three major components, the MGW-2000 system, the
MPEG-4 Transmission Server and the ISO MPEG-4 player do not present any performance
limitations that could prohibit the deployment of live distance learning scenarios.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 21 of 49
TRIALS THEORY AND RESULTS
A. – LEARNING AND INSTRUCTION – SOME PRELIMINARY REMARKS
When it comes to instruction we must bear in mind that:
 Teaching and learning are different processes and the former does not necessarily
imply the latter.
 There is a vast range of factors involved in both processes, such as the different
individual characteristics of the participants, the dynamics of the peer groups, diverse
attitudes and approaches towards teaching and learning, but also the nature of
different subject contents to be taught and learned. This renders it impossible to come
up with an “ideal” instructional approach or a “universally applicable” instructional
methodology.
 The type of learning that is demanded from different disciplines / subjects also varies.
As Entwistle and Ramsden1
noted: “In arts, students should be encouraged to search
for personal meaning, which seems to depend on empathy and openness from staff,
informal teaching (discussion) methods, freedom for students to explore their
interests, and yet, because of that freedom, the setting of clear goals and standards. In
science and social science, good teaching depends more on pitching the information at
the right level and being alert to student difficulties. A deep approach in science
depends more on operational learning, on relating evidence and conclusion, and on
the appropriate use of certain amounts of initial rote learning to master the
terminology. But this versatility in learning will emerge readily only when the
workload is reasonable, and where freedom in learning is allowed. The forms of
assessment, the types of questions, will also need to be consistent with lecturers'
attempts to develop critical thinking”.
 In order to achieve learning, any instructional setting, regardless its being a traditional
classroom lecture or a computer-mediated videoconference, must respond to the
learners’ needs. Otherwise the “transactional distance”2 among the key players of the
instructional process (i.e. the teacher, the learner and the knowledge / subject content)
can result in cognitive misunderstandings / misconceptions, communication gaps or
psychological pitfalls (loss of interest, withdrawal, isolation etc). In distance learning,
physical separation of teacher and learner can reinforce these kinds of problems or
can delay the proper responsive actions.
1
Entwistle, N.J. and Ramsden, P. (1983) Understanding Student Learning. Croom Helm
2
Moore, M. G. (1980) Independent Study. In Boyd R D and Apps, J. W. and Associates, Redefining the
Discipline of Adult Education. Jossey-Bass

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 22 of 49
B. – PEDAGOGICAL EVALUATION OF DISTANCE EDUCATION – FORMS AND
CRITERIA APPLIED IN THE TRIALS
These unique features of learning and instruction render educational evaluation a very difficult
and subtle task. Educational researchers have identified different models of educational
evaluation. Following Alexander and Hedberg3
, educational evaluation approaches fall into
four main paradigms4
:
“Objective-based: Evaluation as a process of determining the degree to which educational
objectives are being achieved. This follows the scientific tradition and is straightforward to
apply, but does not take account of unintended outcomes, and takes no account of students as
individuals with all their differences.
Decision-based: Focuses on the decisions made during development and improvements that
could be made. It is useful for programmes with a large scope or multiple levels, but needs the
co-operation of decision makers. It has proved difficult to put into practice and expensive to
maintain.
Value-based: Evaluation is not only concerned with goals, but also whether the goals are worth
achieving. Formative and summative evaluation is used, and the evaluator considers major
effects, achievements and consequences of the programme. This acknowledges the importance
of unintended outcomes, and learners' perceptions of the learning experience, and evaluation
can be made without the need to know about the objectives. Its perceived disadvantages are that
it may leave important questions unanswered.
Naturalistic approach: organises evaluations around the participants' key concerns and issues.
Uses qualitative data collection such as journals, observations and interview. The advantages
are that it acknowledges context and can be used to benefit those being studied, but participants
may identify criteria with little educational worth.”
In the approach we adapted for the streaming video trials, we have tried to evaluate (a) the
parameters of the technical setting / context that we thought had a significant impact on the
educational interaction and, (b) the instruction and the learning process itself.
We conducted the evaluation using:
 Questionnaires for the students who attended streaming video classes
 Semi-structured interviews with the instructors
 Observation grills filled during the trials by the evaluation team.
3
Alexander, A. and Hedberg, J. G. (1994) Evaluating technology-based learning: Which model? Interactive
Multimedia in University Education, In Beattie, K., McNaught, C. and Wills, S. (eds) Designing for Change in
Teaching and Learning, Proceeding of the IFIP Working Conference, Melbourne, 6-8 July 1994, Elsevier
4
cf : (1999) Desktop Videoconferencing for Tutorial Support. PhD Thesis University College London

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 23 of 49
The questions we sought concerning the evaluation of the technical context refer mainly to the
factors that influenced the attention of the learners, their willingness to sit the streaming video
courses, their perceptions about the process. The parameters we investigated were:
 Parameters concerning the quality of the transmitted picture:
 the size of the projected image in the remote classrooms and the desktop
computers
 the size of the projection of the instructional aids (i.e. PowerPoint
presentations)
 the fidelity of the projected picture (clearness, depth, colours etc.)
 the transmission of the movement of the educator or the participants
(interrupted, awkward, natural etc.)
 the visualisation of the communicative expressions of the participants (i.e. the
expressions on the face of the instructor) and the synchronisation of sound and
picture
 the quality of the transmitted instructional material (colours, fonts, sound,
movement etc.)
 Parameters concerning the quality of the transmitted sound:
 the fidelity of the sound
 the intensity of the sound
 the hue / tone of the sound
 the noise due to networking problems
 the echo effect
 the interruptions or the delays in the sound received in the classrooms /
desktops
 Parameters concerning the effectiveness of the direction / mastering choices
 the use of different camera views of the teacher or the host classroom audience
(when there is one)
 the wideness of the camera view
 the focus, the zoom and the interchange of views during the transmission
 the use of a variety of instructional techniques and instructional materials
 the movement of the instructor
 Parameters concerning the classroom set up
 organisational features of the host classroom (aesthetic approach, setup of the
projection stand, etc.)
 functionality of used lights, microphone locations etc.
 parameters concerning the technological / network infrastructure
 usability / adequacy of the technological and communicational media used by

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 24 of 49
the learners
 readiness to incorporate different instructional aids into the transmitted
instruction
 reliability of the used electronic devices (PCs, projectors, storage systems,
control equipment etc.)
 reliability of the network communication (interruptions, delays, other problems
concerning the normal transmission of sound and picture)
The questions we seek concerning the instructional process are similar to those used in the
evaluation of conventional instruction. Emphasis is on the active learning involvement of the
students and the reinforcement of their motivation, since these two elements need to be clearly
addressed in a distance learning environment5
. The parameters used to evaluate the
instructional aspect of the trials were:
 Parameters concerning instructional activities:
 clear statement of the pursued learning objectives
 organisation of the instructional materials
 consideration of the students' experience
 adequate organisation and use of instructional time
 clear use of learning strategies
 Parameters concerning communication / interaction:
 communication style of the instructor
 non verbal communication
 use of questions
 Parameters concerning the instructional material / aids
 genres of instructional material / aids used
 amount of information included
 attractiveness
 stimulation of deep learning
 stimulation of higher cognitive skills
 usability / adequacy
 Parameters concerning the learners
 individual features of the learners
 interest towards taught subject
5
see ADEC, 1999) [ADEC Guiding Principles for Distance Learning and Teaching, American Distance
Education Consortium

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 25 of 49
 familiarity with new technologies / streaming video
 Parameters concerning the instructor
 appearance of the instructor
 communicative skills
 instructional skills
 use of technical media
C. – EVALUATION RESULTS
As mentioned above the evaluation plan involved (a) the use of questionnaires, which were
filled by the participants including both closed and open-ended questions, (b) the use of
observation grilles, which were filled by the research and evaluation team during the trials, and
(c) interviews with the tutors about their experience from the lectures. In this section we are
going to present the results from the analysis of the questionnaires and the observation grills.
The evaluation team has gathered a total of 105 questionnaires that can be used for our
purposes. This sample can not be used to draw generalisations about the pedagogical /
instructional effectiveness of the medium, but can give us a good idea of the features that affect
the process. About 80% of the respondents were male (20% female), and 74% were between
19-23 years of age (19% were from 24-28 and 7% were older than 29 years). The majority for
trials Ia and Ib were undergraduate and students (67% for trial Ia, 100% for trial Ib), while in
trial Ic the majority were graduate students (50%) and technical / academic staff (28%). All
respondents shared familiarity with popular computer programmes (such as MS Widows, MS
Office) and the Internet (all knew how to use a web browser and an e-mail client).
Again, in the first two trials, the majority of the respondents had little or no experience with
streaming video (56% for trial Ia and 72% for trial Ib), while for the third trial (Ic) 67%
admitted they were familiar with streaming video. This feature clearly identifies two different
groups of respondents (a "novice" group and an "expert" group).
The evaluation team analysed the frequencies of the responses to the questionnaires and used
the observation grills to clarify certain features of the trials that were not obvious from the
statistical analysis.
C.1 – Evaluation of Pedagogy / Instruction
Most people in the trial groups were moderately or very familiar with the thematic of the
delivered lecture (75% in Ia, 54% in Ib, 67% in Ic). But most of them were not familiar with the
tutor (this had been the first chance to attend a lecture by the tutor for 60% of the participants).
To evaluate the pedagogical / instructional adequacy of the delivered lecture, the participants
were asked to answer 17 questions about different features of the lesson. Since the views
expressed in all three cases (Ia, Ib and Ic as mentioned above) are similar, we are going to
discuss them all together, distinguishing between the different trials only when necessary.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 26 of 49
In each case the tutor was reported by the respondents to have clear-cut educational objectives
(Ia: 100%, Ib: 75%, Ic: 84.3%). The participants stated that the tutor took under consideration
the background knowledge of the students (Ia: 74.4%, Ib: 67.5%, Ic: 58.8%), delivered the
lecture in a structured manner (Ia: 100%, Ib: 73.1%, Ic: 79%) and established rapport with the
students (Ia: 70%, Ib: 56.6%, Ic: 56.3% -note that the during the first trial the tutor could
interact with the remote students). The differences are attributed to the instructional
characteristics of each tutor and the nature of the content of the lecture. These facts can lead us
to assume that for the purposes of the present evaluation all three trials can be considered as
pedagogically successful.
According to the responses of the participants:
In each trial the educational material (mainly presentation slides) that was used facilitated the
development of the topic (Ia: 92.1%, Ib: 63.4%, Ic: 45%) The differences are attributed to the
design of the material, which in some instances did not show well over the network. In most
cases "non-verbal" communication from the tutor, such as his movement or his "body language"
was helpful for the participants (Ia: 81%, Ib: 48%, Ic: 70%). The only occasion where
participants expressed their reservations about the use of instructional material or about the
movements / expressions of the tutor were when these features were disturbed due to technical
reasons (mostly scenario Ib) or when the camera could not follow these movements (mostly
scenario Ia and Ic). This is also the case with gestures and facial expressions, since in some
occasions it was difficult to understand their meaning due to the transmitted view angle
(scenarios Ia and Ic).
The duration of each trial (about 60 min overall, featuring a small intermediary break to change
stream settings) was considered by most of the participants as adequate for the delivery of a
lecture through streaming video (Ia: 58.2%, Ib: 65.8%, Ic: 72.2%). But most participants (over
60%) agree that the information / lecture content that is delivered through streaming video must
be moderately less than the amount presented in conventional classes, more carefully structured
and adequately supported by custom-made instructional aids (such as slide presentations).
As expected, the lack of the physical presence of the tutor affected the attendance of remote
students (Ia: 20%, Ib: 41.4%, Ic: 38.1%), and deteriorated the rapport that could be established
between the two parts (Ia: 43.8%, Ib: 56.1% Ic: 65%). Please note that the interactive
communication that was established during trial Ia deteriorated the negative feelings of the
participants. What is surprising, however, is that a large part of the participants (about 40%)
feel that they were marginally influenced by these factors while only about 10% were very
sceptical about them. The evaluation team attributes this feature (a) to the successful
instructional experience, as it was described above and, (b) to the experience of the students
with television, where we feel familiar with individuals who we have never met in person. In all
three trials more than 70% of the students felt that the tutor addressed equally all students
regardless their location (host or remote classroom), so that they did not feel neglected or
deprived.
In the open-ended questions, the students called for some form of interactivity between them
and the tutor especially when the lecture is demanding and there are many questions to be
addressed (mostly in trial Ib). They also called for more applied knowledge -not only
theoretical considerations- (Ib) and they emphasised the ability of the tutor to deliver the lecture
in structured, comprehensive manner (Ia and Ib).

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 27 of 49
We must note that while in trial Ia the host and the remote classroom were able to interact, the
questions which the evaluation team used to assess the pedagogical effectiveness were not
markedly difference between this trial and the other two. But from this particular trial the
participants did not call in their open-question responses for interactivity or probable
unanswered questions. On the contrary these students enjoyed their chance to participate
actively during the lecture (80%) and to collaborate with peers located remotely (32%).
C.2 – Pedagogical / Perceptual Evaluation of Technical Features
1st Trial – Scenario Ia
1st
Part – MPEG-2 (4 Mbps)
The participants’ views about:
POSITIVE (%) NEUTRAL (%) NEGATIVE (%)
The size of the screen 73.7 10.5 15.8
The quality / fidelity of the
picture
65.8 18.4 15.7
sound
62.9 25.7 21.4
Picture Interruptions: 16.7 30.6 52.7
Transmission of movements
of the tutor
63.6 24.2 12.1
Echo 50 32.4 17.6
Sound Interruptions 28.2 25 46.9
Network noise 21.3 42.4 36.4
Lip synchronisation 51.4 28.1 12.5
2nd Part - MPEG 1 (1,5 Mbps)
picture
64.7 23.5 11.8
sound
56.3 34.4 9.4
of the tutor
50 37.5 12.5
Acoustics 35.5 51.6 12.9
Sound Interruptions 25.8 41.9 32.3
3rd Part – H.323 (1,5 Mbps)
The size of the screen 58.2 25.8 16
The quality / fidelity of the 49.8 27.3 22.9

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 28 of 49
picture
sound
56.4 29.4 14.2
of the tutor
59.6 28.5 11.9
Acoustics 36.5 48.5 15
During the first part they felt positive about the size of the screen (73.6%), the fidelity of the
picture (65.8%) and the fidelity of the sound (62.9). They also experienced positively the
movement of the tutor (63.6%) and lip synchronisation (59.4%). Picture (52.7%) and sound
(46.9%) interruptions have been the most annoying aspects of this session.
During the second part of the trial (MPEG-1) the participants felt positive about the size of the
screen (64.7%), the fidelity of the picture (64.7%) and the sound (56.3%). They also expressed
positive feelings about the movement of the tutor (50%) and lip synchronisation (51.6%).
Picture and sound interruptions were not so much of a problem as they were during the 1st
part
(22.6% and 32.3% negative views respectively). During the third part of the trial (H.323) again
the size of the screen and the fidelity of the sound and picture was reported positive (58.2%,
49.8% and 56.4% respectively). As in the previous parts the transmission of the movements of
the tutor and the lip synchronisation were positive (less lip synchronisation this time 47.6%). In
this part there have also been picture interruptions (24.1%) but the sound was reported as the
major problem (31.2%).
Diagrams No. 1 and No.2 present the overall opinion of the participants as far as the technical
part is concerned based on the average of their answers to the technical features questions. The
results are contradictory to the straightforward answer of the participants to the open-ended
question calling for their favourite stream.
In the open-ended questions the participants expressed their preference for the second part of
the trial (MPEG-1 – 65%) over the third (H323 – 27.5%) and the first part (MPEG-2 – 7.5%).
We have tried to explain this contradiction using other statistical techniques, like chi-square.
The results of the further analysis are presented in paragraph C4.
13,6
35,4
26,3
20,0
4,5
0,0
10,0
20,0
30,0
40,0
V.P. P. A. N. V.N.
Diagram 1 - Scenario Ia, MPEG-2
9,6
35,9
39,0
11,6
3,9
0,0
10,0
20,0
30,0
40,0
V.P. P. A. N. V.N.
Diagram 2 - Scenario Ia, MPEG-1

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 29 of 49
2nd Trial – Scenario Ib
1st Part – MPEG-4 (1000 Kbps)
The size of the screen 32.5 15 52.5
picture
22 31.7 46.3
sound
61 19.5 19.5
of the tutor
67.5 22.5 10
Acoustics 55 42.5 20
Sound Interruptions 32.5 42.5 25
2nd Part – MPEG-1 (1000 Kbps)
picture
42.8 16.7 30.6
sound
38.9 16.7 44.4
of the tutor
77.8 13.9 8.3
Acoustics 44.4 33.3 22.2
The quality of the sound (61%), the transmission of the movement of the tutor (67.5%) and lip
synchronisation (68.1%) were the most positive aspects of the first part of the trial according to
the participants' views. The quality of the picture and the size of the screen were considered as
the most negative aspects of this part (46.3% and 52.5% respectively).
During the second part we encountered positive views from the participants concerning most
features (size of the screen 55.5%, transmission of the movements of the tutor 77.8%, lip
synchronisation 48.2% were the most positive. The fidelity of the sound has been reported poor
in this part (negative 44.4%)
Diagrams No. 3 and No 4 refer to the average of the opinions of the participants concerning the
technical features of the second trial.
The open-ended questions give us a clearer picture about the preference of the participants. In
this trial the second trial the participants clearly showed their preference for the 2nd
stream
14,7
35,1
24,7
21,1
4,4
0,0
10,0
20,0
30,0
40,0
V.P. P. A. N. V.N.
Diagram 3 - Scenario Ib, MPEG-4
8,3
36,6
25,2 24,9
4,9
0,0
10,0
20,0
30,0
40,0
V.P. P. A. N. V.N.
Diagram 4 - Scenario Ib, MPEG-1

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 30 of 49
(MPEG-1 – 81%) over the first (MPEG-4 – 19%), which offered them more positive
experiences as far as the quality of the picture is concerned. More than 60% of those who
preferred the first stream, justified their view referring to the better quality of the sound during
the 1st
session.
Once again the transmission of the movements of the tutor and the lip synchronisation was very
positive in both parts of the trial, a fact that has certainly contributed to the instructional
success of the trial.
3rd Trial – Scenario Ic – Desktop Users
1st Part - MPEG 4 (500 Kbps)
picture
52.4 14.3 33.3
sound
58.2 19 23.8
Picture Interruptions: 15 75 10
of the tutor
71.4 28.6 0
Echo 16.7 55.6 27.8
Sound Interruptions 35 25 40
Lip synchronisation 19 42.9 38.1
2nd Part - MPEG 1 (500 Kbps)
picture
57.1 19 23.8
sound
76.2 19 4.8
of the tutor
71.4 23.8 4.8
Echo 38.9 44.4 16.7
Sound Interruptions 55 35 10

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 31 of 49
During the first part of the trial (MPEG-4) the participants expressed positive feelings about the
size of the screen and the quality of the picture (52.4%), the transmission of movement (71.4%)
and the fidelity of the sound (58.2%). On the other hand they criticised the sound interruptions
(40%) and the lack of lip synchronisation (38.1%).
During the second part they were positive about the quality of the picture (57.1%), the quality
of the sound (76.2%), the transmission of movement (71.4%). Picture and sound interruptions
were not a problem in this part (9.6% and 10% negative aspects respectively).
Diagrams No 5 and No 6, show the average of the views of the participants concerning the
technical features.
This time the results are compatible to the opinions expressed at the open-ended questions,
where we can see that the 2nd
session (MPEG-1) satisfied about 85% of the participants over the
first session (15%). Many participants from this group justified their preference in technical
terms, referring to the less demanding computer power of the second stream.
C.3 – Scenic Direction
For the first two trials, the camera of the UoA teleteaching room was used to capture the
movement of the tutor. In the 3rd
trial a camcorder was used to pre-record the lecture. This
difference clearly affected the views of the participants as to the scenic aspect of the trials. The
respondents clearly favoured the use of the classroom camera (Ia: 65.7%, Ib: 48.8%, while Ic:
36.4% were positive about the position and the angles of the used cameras), which was able to
follow the movement of the tutor more accurately, without losing its focus in most cases.
The participants have identified several features that must be taken under consideration. As
most significant features / aspects of the scenic setting and direction have been mentioned:
 The use of movement during the lecture as opposed to steady shots (overall about
54%). At the same time they prefer smooth movement, which does not take the
camera out of focus (34%).
 The interchange of views between the tutor and the students or between the tutor and
the instructional materials (41%). Parallel projection of the tutor and the instructional
material was also welcomed (29%).
5,3
32,8
36,3
21,2
4,4
0,0
10,0
20,0
30,0
40,0
V.P. P. A. N. V.N.
Diagram 5 - Scenario Ic, MPEG-4
15,7
38,1
32,1
9,7
4,4
0,0
10,0
20,0
30,0
40,0
V.P. P. A. N. V.N.
Diagram 6 - Scenario Ic, MPEG-1

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 32 of 49
 Avoidance of a long close-up of the tutor (57%). The respondents opted for more
open plans, which carry the feeling of the classroom spatial arrangements (43%).
 The tutor is asked to look at the camera (29%). This is reported to help the
establishment of rapport between the host and the remote classrooms.
 The lights of the host classroom must be carefully prepared, so that the tutor is shown
without shadows and his / her expressions can be seen when the camera focuses on
him / her (36%).
 The instructional material must be designed in a way that facilitates its transmission
(e.g. well-designed slide shows) (37%).
Bad choices concerning these features can affect negatively the attention of the individuals and
trigger repulsive reactions. In the open-ended questions many respondents justify their negative
feelings against streaming video instruction in terms of "false" (according to their perception)
scenic direction choices (28%).
The participants have identified several features concerning the site where they have been, the
remote classroom. The note that everyone must be able to have a clear view of the used screens,
and the lights must not disturb their attendance (26%).
C.4 - A statistical analysis of the factors affecting the satisfaction of the participants
We have analysed the data of all three trials using chi-square test, to reveal the factors affecting
the satisfaction the participants gained from their experience with streaming video.
We have found that the degree of satisfaction was independent from their personal
characteristics, such as their age (p=0.714), their sex (p=0.072) or their study level (p=0.428). It
was also independent from their degree of familiarity with computers (p=0.667). However we
have found that the degree of satisfaction to be significant for those students who claimed to be
very familiar with the Internet (p=0.002), and those who used it regularly to support their
scientific work (p=0.005) or to communicate via e-mail (p=0.036).
Carrying on the analysis on the mutual influence between the pedagogical and the technical
features of the trials, we have found that there is statistically significant relation between the
degree in which the content of the instruction corresponded to the expectations and the interests
of the participants, on the one hand and several organisational - instructional aspects (p<0.005)
and several technological features (p<0.005) on the other, have affected the degree of
satisfaction of the participants.
As far as the instructional factors are concerned, the statistical analysis showed that the
participants gained more satisfaction when (a) the instructional goals were clearly defined
(p=0.000), (b) the information was presented in a structured manner (p=0.000), and (c) when
the tutors took under consideration the prior knowledge of the participants for the subject
(p=0.000).
When the participants had sat in lectures of the tutor before the streaming video trials this also

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 33 of 49
affected positively their degree of satisfaction from the trials (p=0.037). The gestures and the
facial expressions of the tutor (0.003) had also a positive effect on the degree of satisfaction of
the participants as is the case with their movement during the lecture (p=0.001).
Concerning the technical features, we have found that the influence of the quality of the
streaming video picture and sound was a statistically significant factor of the satisfaction of the
participants. During the first trial the satisfaction of students was negatively affected by sound
problems, such as echo (p=0.039) and noise due to the network (p=0.039), while during the
second trial their satisfaction was negatively affected by interruptions in the picture stream (p=
0.011).
GENERAL SUGGESTIONS – CONCLUDING REMARKS
The distance learning trials have shown that streaming video can be an effective educational
medium as far as the opinion of the involved groups is concerned. Instruction through
streaming video is reported to be a source of positive experience to the learners, although
markedly different than conventional instruction.
On the other hand, the participants have strongly stated their views of what they would expect
in order to improve the educational and communicational effectiveness of the medium:
 The technical part is the primary source of negative experience for the learners. They
expect the stream to be consistent, without delays or interruptions, obviously
influenced by the features of television. On the other hand, the participants are ready
to compromise to the quality of the streaming video, as long as it does not disturb the
instructional process. The bottom line from their remarks is that they would like the
technology to be a facilitator rather than a potential source of trouble. Consistent
picture, consistent sound and lip synchronisation play a leading part for the learners to
gain a positive experience.
 The scenic direction of the tutorial is also important. The learners expect that the view
they get from the host classroom reveals clearly “where the tutor points”, or “what is
written on the whiteboard”, or “who is talking” or “what is the expression on the
tutor’s face”. They call for interchange of views, avoidance of monotonous closed
plans, movement of the tutor. On the other hand they prefer smooth movements that
do not disturb the focus of the cameras and so make it difficult for them to catch up
with what the tutor is doing. If the scenario of the instruction provides for questions
coming from the attendants at the host classroom, they note that all students must
have access to microphones so they can be heard. Bottom line they opt for a carefully
planned scenic appearance of the instructional process, close to what somebody
would expect from a television show.
 The instructional aids should be carefully designed. The slides should be clearly seen
and have live colours. They must carry small and consistent pieces of information.
The learners prefer the slides to be always present on their screens. So they opt for the
parallel transmission of the classroom view and the instructional material.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 34 of 49
 The tutors should be adequately prepared for the lecture. They must have their course
carefully structured, preferably in a modular way, which allows them to convey
controlled pieces of information. They must also plan their time very carefully, so that
the lecture fits the time frame of the streaming video (approximately 30 to 40
minutes). Longer classes can be proved very tiring for the participants, affecting their
ability to attend effectively. They must also take care of their movements and
expressions so that the remote viewers can understand and follow them.

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 35 of 49
APPENDIX QUESTIONNAIRE
END USER EVALUATION PLAN
QUESTIONNAIRE / INTERVIEW PROTOCOL
“Dear Colleague,
You have been able to participate in an innovative project, investigating alternative ways of
delivery of lectures / lessons through streaming video, a promising technology which is being
developed for transmitting video and sound on the Internet. The aim of our investigation is to
identify the potential pedagogical advantages and / or the potential pedagogical disadvantages /
weaknesses of the use of streaming video in delivering courses / lectures on the Internet. It will
help the research team to improve the technical, instructional, logistical and pedagogical
aspects of the enterprise and to avoid potentially disturbing features of the attempted innovation
in the future.
We would be grateful if you could participate in this semi-structured interview, according to the
protocol issued by members of our research team. We wish to reassure you that anonymity is
guaranteed and that the research team will inform you on the scientific conclusions of the
ongoing project and the outcomes of this survey.”
Section 1 (Filled - out by the research team)
Scenario:
No
Type of Stream / Session
MPEG-2 Mcast-NTUA
MPEG-1 Desktop WMT-medium Desktop

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 36 of 49
MPEG-4 Desktop MPEG-4 Desktop
MPEG-4 Dial-up WMT-low Dial-up
Thematic of the Course:…………………………………………………………..
………………………………………………………………………………………
Tutor:……………………………………………………………………………….
Date: / /
Section 2 - Identification of the type of the End User and
Personal Data
 Sex:
Male Female
 Age:
19-22 23-26
27-30 31-33
34-37 38-41
 Status of the End User:

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 37 of 49
Status Department University
Undergraduate Student _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Postgraduate Student _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Faculty _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Technical Staff _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Administrative Staff _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Other_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
 Students:
Year / Semester of study _ __ _ _ _ _ _ __ _ _ _ _ _ __ _ _ _ _ _ __ _ _ _ _ _ _
 Staff - Faculty - Others:
Type of work _ _ _ _ _ _ __ _ _ _ _ _ __ _ _ _ _ _ __ _ _ _ _ _ __ _ _ _ _ _ _ _
Years of experience_ _ _ _ _ _ __ _ _ _ _ _ __ _ _ _ _ _ __ _ _ _ _ _ __ _ _ _ _
 Site of Attendance:
Remote Classroom
Laboratory Which one? _ _ _ _ _ _ _ _ _ _ _ _
Office
Household Connection Speed? _ _ _ _ _ _ _ _ _ _ _ _

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 38 of 49
Other_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
 Familiarity with PCs
Mention the 3 software programs you use the most:
1. _ _ _ _ _ _ _ _ _ _ _ _ _ 2. _ _ _ _ _ _ _ _ _ _ _ _ _ _ 3. _ _ _ _ _ _ _ _ _ _ _ _ _
Do you posses any certification in a computer related field?
Yes (Which?) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
No
 Familiarity with the Internet
I use of the web:
Daily
Weekly (Times?) _ _ _ _
Monthly (Times?) _ _ _ _
Type of use:
General information gathering

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 39 of 49
Scientific work
Communication via e-mail
Surfing for fun / hobbies
Other _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
 Familiarity with streaming video
None Little Some Extensive
Section 3 - Pedagogical / Instructional Evaluation
 Familiarity with the thematic / content of the lecture / lesson:
None Little Some Extensive
 Familiarity with the tutor:
This is his / her first lecture I attend

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 40 of 49
I have attended up to 5 lecture with the tutor
I have attended more than 5 lectures with the tutor
 Please tick the box which expresses your view on the following:
Components Affecting Instruction Strongly
Disagree
Disagree Neither
Disagree
Nor
Agree
Agree Strongly
Agree
1. The instructional aims have been
clearly defined
2. The information is presented in an
organized / structured manner
3. The speaker used a variety of
instructional methods and techniques
4. The speaker took into account the
degree of familiarity with streaming
video
5. The speaker took into account the
background knowledge of the audience
6. The speaker used feedback questions
during the lecture / lesson
7. I could interrupt the tutor during the

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 41 of 49
lecture, with the use of available
technology
8. The speaker took into account and took
advantage of the students' questions
9. The time devoted to Q+A and
discussion was adequate
10. The students develop collaboration
during the lesson
11. The speaker encourages the audience
to participate actively
12. Familiarity between the tutor and the
students was accomplished
13. The expressiveness of the tutor
(gestures, facial expressions, etc) has
helped me during class
14. The movement of the tutor (walking,
body movements) facilitated the
attendance of the lesson
15. The instructional material that was
used facilitated the attendance of the
lesson
16. The bulk of information presented was

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 42 of 49
excessive
17. The duration of the lesson was
satisfactory
 Tele-teaching through streaming video was up to my expectancies and satisfied my
interests
Not at All A Little A Lot Very Much
 Lacking the natural presence of the tutor has affected my attendance of the lesson
 The use of technology has overthrown the climate of personal contact which is
usually present in conventional tutorials

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 43 of 49
 During the lecture I had the feeling that the tutor:
Lectured only for those students present in the host classroom
Lectured only for those students who attended from remote classrooms
Lectured for everyone attending independent from his / her location
Section 3 - Pedagogical / Perceptual Evaluation of Technical
Features
A. Video and Sound
 How would you describe the contribution of the following technical features of picture
and sound of the streaming video you attended to the development of your interest for the
lecture/lesson?
 Size of the streaming video window / projection screen:
Very negative Negative Neutral Satisfactory Positive
 Fidelity of the streaming video picture (how close to reality was the streaming
video?)

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 44 of 49
 Sound fidelity (clearness, volume, etc)
 Video Stream (were there any interruptions / delays?)
 Movement of the streaming video picture of the tutor (from awkward movement to
near natural movement)
 Echo

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 45 of 49
 Sound Stream (were there any interruptions / delays?)
 Noise (from the network)
 Lip Synchronisation
(Is there lip synchronisation between what the speaker says and what is seen on the
video?)
B. Production / Direction / Logistics
 How would you describe the contribution of the following features of the production
setting of the streaming video you attended to the development of your interest for the
lecture / lesson?

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 46 of 49
 Position of camera / cameras
 Use of different angles of view
 Zoom / focus / view change
 Range used in the lecture shooting (range of the speaker’s movement, background
pictures, object setting)

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 47 of 49
Section 5 - Open questions calling for the opinion of the End User
 Are you satisfied from your attendance of the lecture / lesson through streaming video?
(Please explain)
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………
 Please specify your views referring to the technical part / to the instructional part / to the
logistical part?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 48 of 49
 Would you like to attend more lectures / lessons through streaming video? (Please specify
the advantages and / or disadvantages).
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………
 What are yours suggestions to the research team in order to improve the quality / the
effectiveness of the medium?
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………………………………………………………………
…………………………………………
 In case of use of different streams: Which part of the tutorial was (technologically) more
satisfactory (A, B or C)? Please explain.
…………………………………………………………………………………………
....……………………………………………………………………………………….

VERSION 2
DELIVERABLE # 15.4
AUTHOR
COMPANY UoA
DATE 30/01/200
page 49 of 49
………………………………………………………………………………………….
………………………………………………………………………………………….
………………………………………………………………………………………….
………………………………………………………………………………………….
—o0o—

A Kuriakidou, E Filtzantzidou, P Balaouras,I Roussakis, C. Mouzakis, I Stavrakakis (2001)

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (12)

Ähnlich wie A Kuriakidou, E Filtzantzidou, P Balaouras,I Roussakis, C. Mouzakis, I Stavrakakis (2001)

Ähnlich wie A Kuriakidou, E Filtzantzidou, P Balaouras,I Roussakis, C. Mouzakis, I Stavrakakis (2001) (20)

Mehr von hmouzak

Mehr von hmouzak (7)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

A Kuriakidou, E Filtzantzidou, P Balaouras,I Roussakis, C. Mouzakis, I Stavrakakis (2001)