SlideShare ist ein Scribd-Unternehmen logo
1 von 90
Downloaden Sie, um offline zu lesen
Web Information Systems
         prof. Athena Vakali




Geographical Data Management for
        Web Applications
        Symeon Papadopoulos




         Informatics Department, AUTh
                 13 Dec 2011
Overview


• Introduction
• Representation & storage of geographical data
• Geographical data management with Java (JDBC)
• Text Indexing for Geographical Data
• Google Maps API
• Use Cases: Geographical Data Clustering + Heat maps


                    Web Information Systems
The Era of Geo


• 350K websites make use of Google Maps API (2010)

• GPS-enabled mobile phones > 200M (2010)

• Foursquare users: 5M (2010)  15M (2011)

• The HotPotato check-in service was acquired by
  Facebook for $10Μ (2010)


                    Web Information Systems
Categories of Geo-Services

• Primary services (providers)
   – Google Maps, Bing Maps, Yahoo Maps, MapQuest,
     OpenStreetMap

• Derivative services (mash-ups)

• Location-based social networks
   – Foursquare, Gowalla (acquired by Facebook), Brightkite

• Intelligent geo-based services
   – Analytics, Route planning, Real estate

                        Web Information Systems
GIS


• GIS offer sophisticated software for geographical
  data management and processing.
• They offer many means of representation, search
  and rendering.
• They mostly target geo-centered applications:
   – Urban planning, environmental studies, etc.
• Popular tools: ArcGIS, MapInfo, GeoMedia, κλπ.
• GIS will not be discussed in this lecture.

                       Web Information Systems
Representation and Storage of
     Geographical Data
Lat, Lon


• Geo coordinates constitute the basic element of any
  geo-enabled service.

• A coordinate element is defined by the following:
                  Latitude                     Longitude




                     Web Information Systems
Lat, Lon (ΙΙ)


• Traditional writing:
                               40° 38′ 0″ N, 22° 57′ 0″ E

• Decimal representation: 40.633333, 22.95

• Important note for storage:
     Precision of 0.00001 in the decimal representation
     corresponds to approximately 1m distance (near the
     equator)

                         Web Information Systems
Altitude, Bearing


• Two additional variables related to geographical
  points (still not very common):
   – Altitude: vertical distance of point from the level of sea
   – Bearing: direction to which the user/device is pointed


• The increasing use of GPS-enabled devices is
  expected to increase the importance of these
  variables, especially in Augmented Reality (AR)
  applications.

                         Web Information Systems
Distance calculation                                      (lat1, lon1)

                                                                         d
• Haversine formula:
   –   Δlat = lat2 – lat1, Δlon = lon2 – lon1
   –   a = sin2(Δlat/2) + cos(lat1)  cos(lat2)  sin2(Δlon/2)               (lat2, lon2)
   –   c = 2  atan2(sqrt(a), sqrt(1-a))
   –   d=Rc                [R: earth radius (6 371 km)]

• In JavaScript:                            More accurate and computational expensive
   – var R = 6371; // km                    alternative: Vincenty formula
   – var dLat = (lat2-lat1).toRad();
   – var dLon = (lon2-lon1).toRad();
   – var a = Math.sin(dLat/2) * Math.sin(dLat/2) +
          Math.cos(lat1.toRad()) * Math.cos(lat2.toRad()) *
          Math.sin(dLon/2) * Math.sin(dLon/2);
   – var c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1-a));
   – var d = R * c;


                                Web Information Systems
Geo in DB

• Representation in table form
• e.g. List of Points of interest  Table of Pois




• double  slightly faster
• DECIMAL  higher accuracy                     Field type:
                                                double ή DECIMAL(13,9)

                      Web Information Systems
Use cases


• Query 1: What points of interest can I see near point
  of interest X?
   – Online city exploration in cities (e.g. recommend new Pois
     based on proximity)


• Query 2: What points of interest are there near me?
   – Valuable within location-based applications (e.g. I need
     recommendations for sightseeing, eating, etc. based on
     my current location)



                        Web Information Systems
Geo in DB – Proximity Queries (I)

• Implementation 1 – Compute distance by use of SQL and
  return ordered results:
• set @lat0 = 40.645466; set @lon0 = 22.858342;
  set @dist = 10;
• SELECT *, 6371  2  ASIN ( SQRT( POWER(
  SIN(@lat0 – abs(dest.lat))  pi()/180 / 2), 2) +
  COS (@lat0  pi()/180 )  COS(abs(dest.lat)  pi()/180)  POWER(
  SIN(@lon0 – dest.lon)  pi()/180 / 2), 2) )) as distance
  FROM pois dest
  HAVING distance < @dist                        Haversine formula

  ORDER BY distance LIMIT 10;
                         Web Information Systems
Geo in DB – Proximity Queries (II)


• Problems:
  – Painfully slow!
    For tables of even modest sizes (~1000), response might
    take as long as 8-10 seconds.

  – Distance computations are not reused in successive
    invocations of the same query, since caching computations
    is not supported

  – Not an option for web applications.


                      Web Information Systems
Geo in DB – Proximity Queries (IIΙ)

• Alternative implementation:
  – If, instead of nearby Pois, we ask for Pois that lie within a
    certain radius from a fixed point (e.g. current location)?




                            300m

    Aghia           300m
   Sophia




                        Web Information Systems
Geo in DB – Proximity Queries (IV)

• Bounding box query:
  – SELECT * FROM pois WHERE
    pois.latitude > @min_lat AND pois.latitude < @max_lat
    AND pois.longitude > @min_lon AND
     pois.longitude < @max_lon;
                                                (@max_lat, @max_lon)

  – The question now is how
    to determine
    min_lat, min_lon,
    max_lat, max_lon.

                              (@min_lat, @min_lon)

                      Web Information Systems
Geo in DB – Proximity Queries (Vα)

• Bounding box computation (top and bottom):
• 1°difference in latitude ~ 111km (110.567 near the
  equator, 111.699 near the poles) 
   – 300m  ~0.0027° latitude (since our fixed point is
     located at approximately 45°)
                                                @max_lat
   – @min_lat = @lat0 – 0.0027
   – @max_lat = @lat0 + 0.0027

   e.g. for lat0 = 40.632844 (Aghia Sophia)       @lat0
   @min_lat ~ 40.630144
   @max_lat ~ 40.635544
                                                      @min_lat

                        Web Information Systems
Geo in DB – Proximity Queries (Vβ)

• Bounding box computation (left and right):
• 1°difference in longitude  depends a lot on the latitude of
  the fixed point, e.g ~111.320 km near the equator and 0 near
  the poles.
                                                          Distance (km) for 1°
   – Table with distance correspondences                 difference in longitude
   (the more lines, the better the achieved
      approximation – for intermediate values
      one needs to use interpolation)
   – 300m  ~0.003805° latitude since the
        fixed point lies at latitude~45°
   – @min_lon = @lon0 – 0.003805
   – @max_lon = @lon0 + 0.003805
   – e.g. for lon0 = 22.947094 (Aghia Sophia)                   @max_lon
   @min_lon ~ 22.943289
   @max_lon ~ 22.950899                       @min_lon @lon0

                             Web Information Systems
Geo in DB – Proximity Queries (VI)


• In summary:
  – During the table creation, we make sure to index the
    latitude and longitude columns (conventional B-tree).

  – We convert the proximity query to a bounding box query
    by use of geographical approximations (that can be made
    sufficiently precise if needed).

  – In this form, the SQL query can be executed really fast
    thanks to the indexes.


                       Web Information Systems
Geo-enabled DBs

• PostGIS (Extension for PostgreSQL)
   – Support for “geographical objects”:
      •   POINT (X Y)
      •   LINESTRING (0 0, 1 1, 1 2)
      •   POLYGON (0 0, 4 0, 4 4, 0 4, 0 0)
      •   MULTIPOINT, MULTIPOLYGON, GEOMETRYCOLLECTION
   – Support for tables and calculations (e.g. proximity queries)
     directly through the query engine.

• GeoServer (GIS)
   – Complete Java framework for search of geographical
     objects and map rendering based on a server-client
     architecture.

                        Web Information Systems
Geographical data management in
          Java (JDBC)
Typical lifecycle of geographical data


                                                   Relational model
                                                     specification


                                             JDBC

   Geo data source    Transformation to
                                                    RDBMS storage
   (e.g. Wikimapia)        records
                                                         JDBC
                                          UPDATE                      READ


                                                      Processing

                                                        JAVA




                          Web Information Systems
Data model specification

• Very simple example: table with id + 3 columns
   – Name:      VARCHAR (100)
   – Latitude: DOUBLE
   – Longitude: DOUBLE

     Pois




                           Web Information Systems
Input data + transformation

                                            parsing
                   <name>Benaki museum</name>  “Benaki Museum”
                   <lon>23.70825</lon>  23.70825
                   <lat>37.974515</lat>  37.974515



                  Record
                  name                      latitude   longitude
                  Benaki museum             23.70825   37.974515




                  Web Information Systems
Record insertion with JDBC (1)

• Step 1         Connection Creation
   (A) Load the appropriate JDBC driver
   E.g. in the following example: JDBC_DRIVER = “com.mysql.jdbc.Driver”
   Class.forName(JDBC_DRIVER);

   (B) Connection initialization
   Τhe variable dbURL has the form: protocol + vendor + server + port nr + dbName
      + params
   e.g. jdbc:mysql//myserver.gr:3306/testdb/?
                             useUnicode=true&characterEncoding=UTF-8
   Connection con = DriverManager.getConnection(dbURL, username, password);

   throws exceptions (InstantiationException, IllegalAccessException,
      ClassNotFoundException, SQLException)

                ATTENTION: Connection objects should be reused!

                               Web Information Systems
Record insertion with JDBC (2)

• Step 2         Creation and execution of statement
   (A) Statement creation
   String sqlInsert = “INSERT INTO Pois VALUES (‘Benaki museum’,
                                                        23.70825, 37.974515)”;
   Statement insertStmt = con.createStatement();

   (B) Statement execution (throws SQLException)
   insertStmt.executeUpdate(sqlInsert);

   Alternatively
   PreparedStatement prepInsert = con.prepareStatement(sqlInsert,
                              Statement.RETURN_GENERATED_KEYS);
   prepInsertStmt.executeUpdate();
   ResultSet rs = prepInsertStmt.getGeneratedKeys();
   long recordId = -1;
                                                     Record database key  could
   if (rs.next()) recordId = rs.getLong(1);
                                                     be useful later (better keep it
                                                     than query for it)
                              Web Information Systems
Wrapper method creation


• Encapsulation of process for easy reuse
  public long insertPoi(Connection con, String poiName, double lat, double lon) {
       String sqlInsert = “INSERT INTO Pois VALUES (‘” + poiName + “’,”
                              String.valueof(lat) + “,” + String.valueof(lon) + “)”;
       PreparedStatement prepInsert = con.prepareStatement(sqlInsert,
                              Statement.RETURN_GENERATED_KEYS);
       prepInsertStmt.executeUpdate();
       ResultSet rs = prepInsertStmt.getGeneratedKeys();
       long recordId = -1;
       if (rs.next()) recordId = rs.getLong(1);
       return recordId;
   }




                                 Web Information Systems
Batch insertion

• In case of massive insertions, it is better to make use of the
  batch mode of insertion in order to significantly speed up
  the process.

  con.setAutoCommit(false);                                     Very important!!
  Statement batchInsert = con.createStatement();
  for (int i = 0; i < 10,000; i++){
         batchInsert.addBatch(“INSERT INTO Pois VALUES (‘“ +
          names*i+ + “’,” + String.valueof(lats*i+) + “,” + String.valueof(lons*i+) + “)”;
  }
  int[] updateCount = batchInsert.executeBatch();
  con.commit();
  con.setAutoCommit(true);

                                      Web Information Systems
ResultSet structure



  Initial cursor position    C1    C2    C3     C4 C5 C6 C7 ….
                     R1      22    ab    0.1
                     R2      27    cd    0.5
rs.next()            R3      28    gc    1.2
                     R4      40    ob    3.5
                     R5      51    gz    3.2

                                                                   rs.absolute(3)
                 rs.getInt(1)           rs.getDouble(3)
                 rs.getInt(“C1”)        rs.getDouble(“C3”)

                            rs.getString(2)    (faster)
                            rs.getString(“C2”)


                                         Web Information Systems
Record reading

• Use SELECT statement and iterate through ResultSet
   (A) SELECT statement creation
   Statement selectStmt = con.createStatement();

   (B) Form and execute SELECT statement
   List<Poi> pois = new ArrayList<Poi>();
   String sqlSelect = “SELECT * FROM Pois”;
   ResultSet rs = stmt.executeQuery(sqlSelect);
   while ( rs.next() ) {
       long poiId = rs.getLong(“id”);
       String poiName = rs.getString(“name”);
       double lat = rs.getDouble(“latitude”);       Candidate wrapper method
                                                    Poi extractPoiFromResultSet(ResultSet rs)
       double lon = rs.getDouble(“longitude”);
       Poi poi = new Poi(poiId, poiName, lat, lon);
       pois.add(poi);
   }


                                Web Information Systems
Record updates

• Suppose I have collected data about the popularity of a Poi
  (e.g. count the number of results from a search engine) and I
  have its id at my disposal:
  public void updatePoiPopularity(Connection con, long poiId, int popularity) {
        PreparedStatement updateStatement = con.prepareStatement(
                  “UPDATE Pois SET poi_popularity = ? WHERE id = ?”);
        updateStatement.setInt(1, popularity);
        updateStatement.setLong(2, poiId);
        int n = updateStatement.executeUpdate();
  }
                                                         NEW COLUMN
             name             latitude      longitude    poi_popularity
             ….               …             …            …


                               Web Information Systems
Example Design                                             WikimapiaWrapper      PopWrapper



                                                                     PoiDbWrapper
• Task separation
  List<Poi> pois = WikimapiaWrapper.collectPois(“Thessaloniki”);
  for (int i = 0; i < pois.size(); i++){
         long poiId = PoiDbWrapper.insertPoi(con, pois.get(i).getName(),
                                pois.get(i).getLatitude(), pois.get(i).getLongitude());
         pois.get(i).setId(poiId);
  }
  ….
  Map<Poi,Integer> poiPopMap = PopWrapper.collectPoiPopularities(pois);
  for (Entry<Poi,Integer> poiPopEntry : poiPopMap.entrySet()){
         PoiDbWrapper.updatePoiPopularity(con,
                                poiPopEntry.getKey().getId(), poiPopEntry.getValue());
  }

                                     Web Information Systems
Text Indexing for Geographical Data
Full-text Indexing

• Extremely valuable technology for applications requiring full-
  text search features.

• Classical DB systems do not offer a standard means of
  indexing and searching large amounts of text.

• Established technology: Inverted indices

• Popular implementations:
   –   Lucene, Solr (Java)
   –   Sphinx (C++)
   –   OpenFTS (PostgreSQL)
   –   Lemur (C++)


                              Web Information Systems
Text queries in RDBMS

• Operators LIKE and RLIKE
   – SELECT * FROM pois WHERE name LIKE ‘%museum%’
   – SELECT * FROM pois WHERE name RLIKE ‘museum’


• Conventional indexing structures in RDBMS are not
  particularly helpful in quickly responding to such queries
  (might require visiting every record  too slow for large
  tables)

• Each vendor offers own solution, e.g. FULLTEXT INDEX από
  MySQL.


                         Web Information Systems
Inverted index (I)


• Starting point: Document collection (corpus)

• Text processing 
  Dictionary: set of unique terms appearing in the
  documents of the collection + additional information
  (stats, stems, stop words)

• Index: Document retrieval mechanism based on the
  terms contained in the documents

                     Web Information Systems
Inverted index (II)


• Inverted index: Set of inverted lists.

• Inverted list: A list containing information about the
  occurrence of a term within the documents of the collection –
  represented as a list of postings.

• Posting: Occurrence of a term in a certain position of a
  document. Apart from the document, where the term occurs,
  the posting also contains its position in the text  support for
  phrase search (increases the size of the index and complexity
  of the query mechanism).

                         Web Information Systems
Inverted index (IIΙ)

Example
• Document collection:
   – Document 1: “White Tower”
   – Document 2: “Byzantine Museum”
   – Document 3: “Archeological Museum”

• Inverted lists of postings:
   –   “Archeological”: {(3,1)}
   –   “Byzantine”: {(2,1)}
   –   “Museum”: {(2,2), (3,2)}
   –   “White”: {(1,1)}
   –   “Tower”: {(1,2)}
                         Document id
                                 Term position within the document
                             Web Information Systems
Inverted index (IV)

Query example
• Query: “Museum”  documents 2 and 3

• Query: “Byzantine Museum” 
   – Result 1: document 2 (contains both terms)
   – Result 2: document 3 (contains only the term “museum”)


                                                    • “Archeological”: {(3,1)}
   • Document 1: “White Tower”                      • “Byzantine”: {(2,1)}
   • Document 2: “Byzantine Museum”                 • “Museum”: {(2,2), (3,2)}
   • Document 3: “Archeological Museum”             • “White”: {(1,1)}
                                                    • “Tower”: {(1,2)}


                          Web Information Systems
Solr (text search engine)


• Complete framework for indexing and searching massive
  document collections.

• Based on the Lucene library.

• Flexible document representation structure.

• Flexible query structure.

• Can be installed as a servlet in an appropriate container (e.g.
  Tomcat, Jetty)  Possibility for use over HTTP


                         Web Information Systems
Solr basics (Ι)


• Every document consists of a set of fields (similar to Lucene)

• Every field can be of different type (e.g. text, number, etc.)

• Every field can processed (e.g. tokenized, normalized) and
  indexed in a different way.

• Every field or combination of fields can be queried in a
  different way  flexible query system


                          Web Information Systems
Solr basics (ΙΙ)

Example:
    – id: 13424
    – name: White Tower of Thessaloniki
    – latitude: 40.626369
    – longitude: 22.948428
    – history: The tower, which once guarded the eastern end of the city's sea
      walls, was for many years attributed to Venice, to which the Byzantines ceded
      Thessaloniki in 1423.
    – wikipedia_link: http://en.wikipedia.org/wiki/White_Tower_of_Thessaloniki

Xml representation suitable fοr use by Solr:
    <add><doc>
       <field name=“id”>13424</field>
       <field name=“name”>White Tower of Thessaloniki</field>
       ...
    </doc></add>


                               Web Information Systems
Solr: index management

• Addition of documents:          java -jar post.jar lefkos_pirgos.xml
   – The xml file should follow the structure of the example in the previous
     example.
   – A list of files can be provided as input (whitespace separated).
   – If a document with the same id (unique key) is provided as input, then
     the new document will replace the existing.

• Deletion of documents:
      java -Ddata=args -Dcommit=no -jar post.jar
              "<delete><id>13424</id></delete>"

• Committing changes:             java –jar post.jar

• The above operations (and more) are available through a Java
  (not only) API as well.
                            Web Information Systems
Solr: Queries

HTTP API: Base URL: http://server:8983/solr/select/?

Return fields “name”, “id” of all entries containing ‘museum’.
• q=museum&fl=name,id

As above but also return relevance score for each entry.
• q=museum&fl=name,id,score

As above but return all stored field.
• q=museum&fl=*,score

Return entries in JSON format.
• q=museum&wt=json

Return all entries containing the keyword ‘pirgos’ in their field “name”.
• q=name:pirgos

                               Web Information Systems
Challenges

• Text indexing and search frequently encounters the following
  two challenges:

   – Synonyms, writing variations: users search for something using a form
     different than the “reference” form.
     e.g. White Tower, Λευκός Πύργος, λευκός πύργος, lefkos pyrgos, ... 
     Solution: Attempt to collect and index many different lexical forms of
     the same term.

   – Polysemy/Ambiguity: A single term refers to more than one concepts.
     e.g. the term museum may refer to a number of real-world museums
     Solution: Appropriate interfaces are necessary to make the user
     aware of the different senses/contexts of a term and ask for query
     reformulation/specification (could also come up with suggestions).

                            Web Information Systems
Synonyms


• Strategy 1 (simple but not always available):

   – Use dictionary/thesaurus/gazetteer (e.g. Geonames)
     Contains alternative names and variations for a large number of Pois
     and locations.

   – Addition of a “synonyms” field in the index

   – Include the additional field in the search
     …but give a higher weight to the “name” field.




                            Web Information Systems
Related terms

• Strategy 2 (more complex but generally applicable):
   – Issue query to search engine or other content application (e.g. Flickr)
       • Use Poi name as query.

   – Process search results in two ways:
       • Find terms or term sequences that frequently occur in the same context (e.g. web
         page, title of photo, etc.).
       • Find terms that are lexically similar to the query term (e.g. based on the
         Levenshtein distance) .

   – Addition of a “similars” field in the index.
   – Include the additional field in the search
       • but give higher weight to the “name” field




                                  Web Information Systems
Google Maps API
Licensing


• The Google Maps service is free for web pages that
  are freely available. Charges apply for commercial
  usage of the API.

• Since V3, the API does not require a key.




                     Web Information Systems
Google Maps API (I)

• The map service is based on the notion of map tiles,
  which, when stitched together, provide an
  impression of a continuous map.




                     Web Information Systems
Google Maps API (II)

• There are different types of map types, e.g. ROADMAP,
  SATELLITE, HYBRID, TERRAIN. Those are called base map tiles.
     ROADMAP                             HYBRID




      HYBRID                             TERRAIN




                        Web Information Systems
Google Maps API – Coordinates (I)

Coordinates:
• World: Point 0,0                        lat ~ 85°
                                           lon 180°

• Pixel: pixelCoord = worldCoord  2zoomLevel


The API first determines the visible area of the map on the client
  (viewport) based on the zoom level, the map center and the
  size (in pixels) of the DOM element in the client web page.


                         Web Information Systems
Google Maps API – Coordinates (II)

Coordinates:
• Tile coordinates:
   – With every scroll or zoom to a new viewport, the API needs to render
     a new area on the map. For this to happen, it needs to translate the
     pixel coordinates into appropriate tile coordinates in order to load the
     appropriate tiles. At each zoom level the tile coordinates are different.

   – e.g. at zoom level 2 (beside) 16 tiles
     need to be indexed. The tile containing
     Greece has coordinates (2,1).
     At zoom level 3, 16 x 4 = 64 tiles need to
     to be indexed, and so on.




                             Web Information Systems
The “hello world” of Google Maps
•   The map object is integrated to any web page as a DOM element.
    <html>
    <head>
    <meta name="viewport" content="initial-scale=1.0, user-scalable=no" />
    <style type="text/css"> html { height: 100% } body { height: 100%; margin: 0px; padding: 0px }
    #map_canvas { height: 100% }</style>                                             Library import
    <script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=false"></script>
    <script type="text/javascript">
    function initialize() {                                     Map initialization and centering
             var latlng = new google.maps.LatLng(40.66, 22.95);
             var myOptions = { zoom: 12, center: latlng,
                                      mapTypeId: google.maps.MapTypeId.ROADMAP };
             var map = new google.maps.Map(document.getElementById("map_canvas"), myOptions);
    }
    </script>
    </head>
    <body onload="initialize()">                                     Map canvas DOM element
    <div id="map_canvas" style="width:100%; height:100%"></div>
    </body>
    </html>
                                  Spans the whole page body

                                        Web Information Systems
Basic objects


• google.maps.LatLng
   – Initialization: var latlng = new google.maps.LatLng(lat_p, lon_p);
   – Very useful object, e.g. for marker placement.
   – In case we don’t know the geo coordinates, but know the street name,
     we could make use of a Geocoding service.


• google.maps.Map
   – Initialization: var mapx = new google.maps.Map(domElement,
     options);
   – Basic parameters: zoom level, center, mapType (ROADMAP, HYBRID,
     etc.)
   – It is possible to have more than one map objects on the same page.


                           Web Information Systems
Zoom level

• Very important to select the proper zoom level depending on
  the application needs.
     zoom = 4               zoom = 8              zoom = 12




     zoom = 16               zoom = 18             zoom = 20




                       Web Information Systems
More basic objects


• google.maps.Marker
   – Initialization:
       var latlng = new google.maps.Marker({
           position: latlng,
           map: mapx,
           title: “POI Title”
       });




• Very useful object!
• Use it for marking Pois.


                             Web Information Systems
Events on the map UI


• Interacting with the map, as in every UI, triggers a
  variety of events.

• There are two basic event categories:
   – User/UI events: Objects on the map respond to certain
     user actions.
   – MVC events: Triggered by changes in the state of the map,
     e.g. change of value for some parameter (zoom level).



                       Web Information Systems
Map events

• UI events
  Look like common DOM events but are separately triggered in
  order to prevent incompatibility with different browsers.

  e.g. the Marker object responds to the following events:
   –   “click”,
   –   “dblclick”,
   –   “mouseup”,
   –   “mousedown”,
   –   “mouseover”,
   –   “mouseout”.


                        Web Information Systems
Event listening (I)

• addListener()
    – Object that “listens”
    – event that is “listened”                                 MVC event
    – Method that is invoked
      as soon as the event occurs
• Example:
      google.maps.event.addListener(map, 'zoom_changed', function() {
          setTimeout(moveToDarwin, 3000); // fire moveToDarwin() after 3 secs
      });
      google.maps.event.addListener(marker, 'click', function() {
        map.setZoom(8);
      });                                                    UI event

      function moveToDarwin() {
        var darwin = new google.maps.LatLng(-12.461334, 130.841904);
        map.setCenter(darwin);
      }
                              Web Information Systems
Event listening (II)

•   Argument passing
     – Only for UI events
     – Access properties and their values as in any JavaScript object

•   Example:
     google.maps.event.addListener(map, 'click', function(event) {
          placeMarker(event.latLng);
         });
     function placeMarker(location) {
         var marker = new google.maps.Marker({
             position: location,                       Marker creation at the point clicked by
             map: map                                  the user.
         });
         map.setCenter(location);                      Map centering on the same point.
     }


                                    Web Information Systems
Event listening (IΙI)

• Read and write properties within MVC events
   – The value that we read may be different than the value at the time of
     event triggering (usually we care only for the current value).


• Example:
   google.maps.event.addListener(map, 'zoom_changed', function() {
       zoomLevel = map.getZoom();
       alert("Zoom: " + zoomLevel);
       if (zoomLevel == 0) {
         map.setZoom(10);
       }
   });

                            Web Information Systems
Event listening (IV)

• Listen to common DOM events
   – addDomListener()
                                                    window, document.body, etc.
                                 DOM element
• Example:                                          named elements
   <script>
                                               event name
      function initialize() {
          // map initialization                     method to be invoked
      }
      google.maps.event.addDomListener(window, 'load', initialize);
   </script>
   <body>
      <div id="map_canvas"></div>
   </body>

                          Web Information Systems
Control elements (I)

• UI elements that enable users to interact with the
  viewport on the map.

• Navigation control: pan & zoom



• Scale control

• MapType control

                     Web Information Systems
Control elements (II)

• Determine active controls by use of approprate arguments in the map
  object options (only at create time):
      {
          navigationControl: boolean,
          mapTypeControl: boolean,
          scaleControl: boolean
      }

• Example:
    function initialize() {
        var myOptions = {
          zoom: 4,
          center: new google.maps.LatLng(-33, 151),
          navigationControl: false,
          scaleControl: true,
          mapTypeId: google.maps.MapTypeId.ROADMAP
        }
        var map = new google.maps.Map(document.getElementById("map_canvas"),
            myOptions);
    }

                                 Web Information Systems
Control elements (IΙI)

•   Control elements offer additional options:
     – google.maps.NavigationControlStyle.SMALL  mini-zoom form
     – google.maps.NavigationControlStyle.ZOOM_PAN  standard form of zoom & pan
     – google.maps.MapTypeControlStyle.DROPDOWN_MENU  select map types from
       dropdown menu

•   Example:
     var myOptions = {
          ...
          mapTypeControl: true,
          mapTypeControlOptions: {
            style: google.maps.MapTypeControlStyle.DROPDOWN_MENU
          },
          navigationControl: true,
          navigationControlOptions: {
            style: google.maps.NavigationControlStyle.SMALL
          },
          ....
     }
     var map = new google.maps.Map(document.getElementById("map_canvas"),
            myOptions);


                                 Web Information Systems
Control elements (IV)

• Control elements can be placed in different positions on the
  map:




  navigationControl: true,
  navigationControlOptions: {
    style: google.maps.NavigationControlStyle.ZOOM_PAN,
    position: google.maps.ControlPosition.TOP_RIGHT
  }

                          Web Information Systems
Control elements (V)                              custom control




• It is possible to create custom
  control elements.
• Control elements are
  essentially div elements.
• In order to create them, you need:
   – Define their appearance by use of CSS.
   – Determine their interactions by use of event handlers.
   – Create div element and add it to controls property of the
     Map object.


                        Web Information Systems
Overlays

• Objects with specific coordinates.
                  Markers                             Icons




      Polylines                    Polygons                   InfoWindow




                            Web Information Systems
Overlays - Markers

• Markers are initialized with their position, the map,
  and a title as arguments (the title appears in a tooltip
  upon mouse hover).

• It is also possible to animate markers with two
  predefined animation schemes by use of the method
  marker.setAnimation(method):
   – google.maps.Animation.DROP: Drops and stops.
   – google.maps.Animation.BOUNCE: Bounces.
   – null: Static.


                      Web Information Systems
Overlays - Icons

• Markers with a different look.
• Set the icon property.
• Example:
   var image = `beachflag.png’;
   var myLatLng = new google.maps.LatLng(44.890, 22.27);
   var beachMarker = new google.maps.Marker({
         position: myLatLng,
         map: map,
         icon: image
   });
• Options for more complex appearance (e.g. by use of
  shadow and stacking).
                       Web Information Systems
Polylines

• Useful for presenting routes, directions, etc.
• Example:
   var flightPlanCoordinates = [
         new google.maps.LatLng(37.772323, -122.214897),
         new google.maps.LatLng(21.291982, -157.821856),
         new google.maps.LatLng(-18.142599, 178.431),
         new google.maps.LatLng(-27.46758, 153.027892)
   ];
   var flightPath = new google.maps.Polyline({
         path: flightPlanCoordinates,
         strokeColor: "#FF0000",
         strokeOpacity: 1.0,
         strokeWeight: 2
   });
   flightPath.setMap(map);

                           Web Information Systems
Polylines - Processing


• It is possible to dynamically modify a polyline.

• Read and modify the points in a line:
   – var path = poly.getPath();
   – var latlng_idx = path.getAt(idx)  read element on
     position idx
   – path.insertAt(idx)  insert element in position idx
   – path.removeAt(idx)  delete element on position idx
   – path.push(latlng)  add element at the end of the line


                       Web Information Systems
Polygons

• Polygons cover an area.
• Example:
    var triangleCoords = [
          new google.maps.LatLng(25.774252, -80.190262),   Same start and
          new google.maps.LatLng(18.466465, -66.118292),
          new google.maps.LatLng(32.321384, -64.75737),    end coordinate
          new google.maps.LatLng(25.774252, -80.190262)    (optional)
    ];
    bermudaTriangle = new google.maps.Polygon({
          paths: triangleCoords,
          strokeColor: "#FF0000",
          strokeOpacity: 0.8,
          strokeWeight: 2,
          fillColor: "#FF0000",
          fillOpacity: 0.3
    });
    bermudaTriangle.setMap(map);
• Same behaviour as Polylines with respect to processing their elements
  (e.g. getPath(), etc.).
                              Web Information Systems
Info Window (Ι)

• Very important for providing information for a Poi
  upon click.
• May contain: (a) simple text, (b) HTML code, or (c)
  DOM element.
• Initialized by use of the following elements:
   – content: String (simple text/HTML), DOM node
   – position: If the Info Window is to be opened on top of a
     marker, then the marker position is provided.
   – maxWidth: Maximum width in pixels (autowraps text).


                        Web Information Systems
Info Window (ΙΙ)

•   Example:
     var contentString = '<div id="content">'+
         '<div id="siteNotice">'+ '</div>'+
         '<h1 id="firstHeading" class="firstHeading">Uluru</h1>'+
           '<div id="bodyContent">'+
           '<p><b>Uluru</b>, also referred to as <b>Ayers Rock</b>, is a large ' +
           'sandstone rock formation in the southern part of the '+
           'Northern Territory, central Australia. It lies 335 km (208 mi) '+ .... + </p>'+
           '<p>Attribution: Uluru, <a
         href="http://en.wikipedia.org/w/index.php?title=Uluru&oldid=297882194">‘ +
           'http://en.wikipedia.org/w/index.php?title=Uluru</a> (last visited June 22, 2009).</p>‘ +
           '</div>‘ + '</div>';

     var infowindow = new google.maps.InfoWindow({
            content: contentString
     });
     var marker = new google.maps.Marker({
            position: myLatlng,
            map: map,
            title:"Uluru (Ayers Rock)«
     });
     google.maps.event.addListener(marker, 'click', function() {
           infowindow.open(map,marker);
     });


                                         Web Information Systems
Layers

• Collections of information that are overlaid on top of
  base map tiles and modify its appearance. They are
  treated as a single object.

• Existing layers:
   – KmlLayer, GeoRSS
      • Presentation of feed of geotagged posts on a map.
   – TrafficLayer
   – BicyclingLayer


                          Web Information Systems
Geocoding


• Service for converting between street names and
  geographical coordinates.
• Rate limit: 2500 calls / day
• Base URL
   http://maps.googleapis.com/maps/api/geocode/output?par
   output: json / xml
   par 
     required: address OR latlng, sensor
     optional: bounds, region, language


                      Web Information Systems
Use Cases:
Geographical Data Clustering + Heat Maps
Geo-Clustering (1)


• Represent each location p = (lat, lon) as a feature
  vector: x = [x1 x2]

• Apply some density-based clustering method.
   – Prefer methods that do not require setting the number of
     clusters as parameter (e.g. k-means is not really
     appropriate). Recommend methods: BIRCH and DBSCAN
     (they require setting other parameters).




                       Web Information Systems
Geo-Clustering (2)

• Example of using the BIRCH algorithm on a set of
  geotagged Flickr photos in the vicinity of
  Thessaloniki.


                                                       CENTER +
                                                       40 ΕKKLISIES +
                                                       ANO POLI

                                                     NEA PARALIA
                                                  KARABOURNAKI
                                                      ARETSOU



        d = 0.0015                             d = 0.0030


                     Web Information Systems
Geo-Clustering (3)

 • Example of using clustering in a web application.




http://www.clusttour.gr

markers + polygons + infowindow

                                  Web Information Systems
Heat maps

• A visual analytics tool.

• Useful for visualizing location-dependent quantities,
  e.g. population density, housing prices, crime rates,
  pollution, etc.

• Requires knowledge or inference of quantity
  distribution over space.

• Overlay on top of the map.


                      Web Information Systems
Heat maps

• Example: geotagged photos collected from Flickr based on the
  query “sziget festival”)




                              GRID




                       Web Information Systems
Heat maps

• Counting frequency in each cell + smoothing based on
  adjacent cells + color coding




                       Web Information Systems
Heat maps (1)




Languages in Twitter




                       http://flowingdata.com/2011/10/27/language-communities-of-twitter/

                                                   Web Information Systems
http://flowingdata.com/2011/12/07/every-death-on-the-


     Heat maps (2)
                                 road-in-great-britain/




Deadly car accidents in the UK

                                               Web Information Systems
http://www.flickr.com/photos/walkingsf/5925793753/in/set-72157627140310742



     Heat maps (3)

                                           La Grande Arche de la Defense




                                       Versailles




                                                                                              Orly Airport
Twitter (blue) +
             Flickr (orange)

                                                 Web Information Systems
http://tagmaps.research.yahoo.com/worldexplorer.php


Tag maps   Rome




           London




                                Web Information Systems
Thank you!
Symeon Papadopoulos
papadop@iti.gr

Weitere ähnliche Inhalte

Ähnlich wie Geographical Data Management for Web Applications

Brewing the Ultimate Data Fusion
Brewing the Ultimate Data FusionBrewing the Ultimate Data Fusion
Brewing the Ultimate Data FusionSafe Software
 
NAPSG 2010 Fire/EMS Conference - Data Sharing Basics
NAPSG 2010 Fire/EMS Conference - Data Sharing BasicsNAPSG 2010 Fire/EMS Conference - Data Sharing Basics
NAPSG 2010 Fire/EMS Conference - Data Sharing Basicspdituri
 
The state of geo in ElasticSearch
The state of geo in ElasticSearchThe state of geo in ElasticSearch
The state of geo in ElasticSearchFan Robbin
 
dashDB: the GIS professional’s bridge to mainstream IT systems
dashDB: the GIS professional’s bridge to mainstream IT systemsdashDB: the GIS professional’s bridge to mainstream IT systems
dashDB: the GIS professional’s bridge to mainstream IT systemsIBM Cloud Data Services
 
Fed Geo Day - GeoTrellis Intro
Fed Geo Day - GeoTrellis IntroFed Geo Day - GeoTrellis Intro
Fed Geo Day - GeoTrellis IntroAzavea
 
Dsd int 2014 - data science symposium - application 1 - point clouds, prof. p...
Dsd int 2014 - data science symposium - application 1 - point clouds, prof. p...Dsd int 2014 - data science symposium - application 1 - point clouds, prof. p...
Dsd int 2014 - data science symposium - application 1 - point clouds, prof. p...Deltares
 
Gis capabilities on Big Data Systems
Gis capabilities on Big Data SystemsGis capabilities on Big Data Systems
Gis capabilities on Big Data SystemsAhmad Jawwad
 
Lambda Data Grid: An Agile Optical Platform for Grid Computing and Data-inten...
Lambda Data Grid: An Agile Optical Platform for Grid Computing and Data-inten...Lambda Data Grid: An Agile Optical Platform for Grid Computing and Data-inten...
Lambda Data Grid: An Agile Optical Platform for Grid Computing and Data-inten...Tal Lavian Ph.D.
 
Spatially enabled open source BI (GeoBI) with GeoKettle, GeoMondrian & SOLAPL...
Spatially enabled open source BI (GeoBI) with GeoKettle, GeoMondrian & SOLAPL...Spatially enabled open source BI (GeoBI) with GeoKettle, GeoMondrian & SOLAPL...
Spatially enabled open source BI (GeoBI) with GeoKettle, GeoMondrian & SOLAPL...Thierry Badard
 
Using postgre sql for 3d cms
Using postgre sql for 3d cmsUsing postgre sql for 3d cms
Using postgre sql for 3d cmsTim Child
 
Pg intro part1-theory_slides
Pg intro part1-theory_slidesPg intro part1-theory_slides
Pg intro part1-theory_slideslasmasi
 
Building Location Aware Apps - Get Started with PostGIS, PART I
Building Location Aware Apps - Get Started with PostGIS, PART IBuilding Location Aware Apps - Get Started with PostGIS, PART I
Building Location Aware Apps - Get Started with PostGIS, PART Ilasmasi
 
Unleashing the Power of Data Interoperability - UMass Amherst
Unleashing the Power of Data Interoperability - UMass AmherstUnleashing the Power of Data Interoperability - UMass Amherst
Unleashing the Power of Data Interoperability - UMass AmherstSafe Software
 
Geographic information system
Geographic information systemGeographic information system
Geographic information systemSumanta Das
 
Giving MongoDB a Way to Play with the GIS Community
Giving MongoDB a Way to Play with the GIS CommunityGiving MongoDB a Way to Play with the GIS Community
Giving MongoDB a Way to Play with the GIS CommunityMongoDB
 
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...Safe Software
 
Evolution of Esri Data Formats Seminar
Evolution of Esri Data Formats SeminarEvolution of Esri Data Formats Seminar
Evolution of Esri Data Formats SeminarEsri South Africa
 

Ähnlich wie Geographical Data Management for Web Applications (20)

Geoservices Activities at EDINA
Geoservices Activities at EDINAGeoservices Activities at EDINA
Geoservices Activities at EDINA
 
Brewing the Ultimate Data Fusion
Brewing the Ultimate Data FusionBrewing the Ultimate Data Fusion
Brewing the Ultimate Data Fusion
 
NAPSG 2010 Fire/EMS Conference - Data Sharing Basics
NAPSG 2010 Fire/EMS Conference - Data Sharing BasicsNAPSG 2010 Fire/EMS Conference - Data Sharing Basics
NAPSG 2010 Fire/EMS Conference - Data Sharing Basics
 
The state of geo in ElasticSearch
The state of geo in ElasticSearchThe state of geo in ElasticSearch
The state of geo in ElasticSearch
 
dashDB: the GIS professional’s bridge to mainstream IT systems
dashDB: the GIS professional’s bridge to mainstream IT systemsdashDB: the GIS professional’s bridge to mainstream IT systems
dashDB: the GIS professional’s bridge to mainstream IT systems
 
ArcGIS and Multi-D: Tools & Roadmap
ArcGIS and Multi-D: Tools & RoadmapArcGIS and Multi-D: Tools & Roadmap
ArcGIS and Multi-D: Tools & Roadmap
 
Fed Geo Day - GeoTrellis Intro
Fed Geo Day - GeoTrellis IntroFed Geo Day - GeoTrellis Intro
Fed Geo Day - GeoTrellis Intro
 
Dsd int 2014 - data science symposium - application 1 - point clouds, prof. p...
Dsd int 2014 - data science symposium - application 1 - point clouds, prof. p...Dsd int 2014 - data science symposium - application 1 - point clouds, prof. p...
Dsd int 2014 - data science symposium - application 1 - point clouds, prof. p...
 
Gis capabilities on Big Data Systems
Gis capabilities on Big Data SystemsGis capabilities on Big Data Systems
Gis capabilities on Big Data Systems
 
Lambda Data Grid: An Agile Optical Platform for Grid Computing and Data-inten...
Lambda Data Grid: An Agile Optical Platform for Grid Computing and Data-inten...Lambda Data Grid: An Agile Optical Platform for Grid Computing and Data-inten...
Lambda Data Grid: An Agile Optical Platform for Grid Computing and Data-inten...
 
Spatially enabled open source BI (GeoBI) with GeoKettle, GeoMondrian & SOLAPL...
Spatially enabled open source BI (GeoBI) with GeoKettle, GeoMondrian & SOLAPL...Spatially enabled open source BI (GeoBI) with GeoKettle, GeoMondrian & SOLAPL...
Spatially enabled open source BI (GeoBI) with GeoKettle, GeoMondrian & SOLAPL...
 
Day 6 - PostGIS
Day 6 - PostGISDay 6 - PostGIS
Day 6 - PostGIS
 
Using postgre sql for 3d cms
Using postgre sql for 3d cmsUsing postgre sql for 3d cms
Using postgre sql for 3d cms
 
Pg intro part1-theory_slides
Pg intro part1-theory_slidesPg intro part1-theory_slides
Pg intro part1-theory_slides
 
Building Location Aware Apps - Get Started with PostGIS, PART I
Building Location Aware Apps - Get Started with PostGIS, PART IBuilding Location Aware Apps - Get Started with PostGIS, PART I
Building Location Aware Apps - Get Started with PostGIS, PART I
 
Unleashing the Power of Data Interoperability - UMass Amherst
Unleashing the Power of Data Interoperability - UMass AmherstUnleashing the Power of Data Interoperability - UMass Amherst
Unleashing the Power of Data Interoperability - UMass Amherst
 
Geographic information system
Geographic information systemGeographic information system
Geographic information system
 
Giving MongoDB a Way to Play with the GIS Community
Giving MongoDB a Way to Play with the GIS CommunityGiving MongoDB a Way to Play with the GIS Community
Giving MongoDB a Way to Play with the GIS Community
 
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...
Spatial decision support and analytics on a campus scale: bringing GIS, CAD, ...
 
Evolution of Esri Data Formats Seminar
Evolution of Esri Data Formats SeminarEvolution of Esri Data Formats Seminar
Evolution of Esri Data Formats Seminar
 

Mehr von Symeon Papadopoulos

DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...
DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...
DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...Symeon Papadopoulos
 
Deepfakes: An Emerging Internet Threat and their Detection
Deepfakes: An Emerging Internet Threat and their DetectionDeepfakes: An Emerging Internet Threat and their Detection
Deepfakes: An Emerging Internet Threat and their DetectionSymeon Papadopoulos
 
Knowledge-based Fusion for Image Tampering Localization
Knowledge-based Fusion for Image Tampering LocalizationKnowledge-based Fusion for Image Tampering Localization
Knowledge-based Fusion for Image Tampering LocalizationSymeon Papadopoulos
 
Deepfake Detection: The Importance of Training Data Preprocessing and Practic...
Deepfake Detection: The Importance of Training Data Preprocessing and Practic...Deepfake Detection: The Importance of Training Data Preprocessing and Practic...
Deepfake Detection: The Importance of Training Data Preprocessing and Practic...Symeon Papadopoulos
 
COVID-19 Infodemic vs Contact Tracing
COVID-19 Infodemic vs Contact TracingCOVID-19 Infodemic vs Contact Tracing
COVID-19 Infodemic vs Contact TracingSymeon Papadopoulos
 
Similarity-based retrieval of multimedia content
Similarity-based retrieval of multimedia contentSimilarity-based retrieval of multimedia content
Similarity-based retrieval of multimedia contentSymeon Papadopoulos
 
Twitter-based Sensing of City-level Air Quality
Twitter-based Sensing of City-level Air QualityTwitter-based Sensing of City-level Air Quality
Twitter-based Sensing of City-level Air QualitySymeon Papadopoulos
 
Aggregating and Analyzing the Context of Social Media Content
Aggregating and Analyzing the Context of Social Media ContentAggregating and Analyzing the Context of Social Media Content
Aggregating and Analyzing the Context of Social Media ContentSymeon Papadopoulos
 
Verifying Multimedia Content on the Internet
Verifying Multimedia Content on the InternetVerifying Multimedia Content on the Internet
Verifying Multimedia Content on the InternetSymeon Papadopoulos
 
A Web-based Service for Image Tampering Detection
A Web-based Service for Image Tampering DetectionA Web-based Service for Image Tampering Detection
A Web-based Service for Image Tampering DetectionSymeon Papadopoulos
 
Learning to detect Misleading Content on Twitter
Learning to detect Misleading Content on TwitterLearning to detect Misleading Content on Twitter
Learning to detect Misleading Content on TwitterSymeon Papadopoulos
 
Near-Duplicate Video Retrieval by Aggregating Intermediate CNN Layers
Near-Duplicate Video Retrieval by Aggregating Intermediate CNN LayersNear-Duplicate Video Retrieval by Aggregating Intermediate CNN Layers
Near-Duplicate Video Retrieval by Aggregating Intermediate CNN LayersSymeon Papadopoulos
 
Verifying Multimedia Use at MediaEval 2016
Verifying Multimedia Use at MediaEval 2016Verifying Multimedia Use at MediaEval 2016
Verifying Multimedia Use at MediaEval 2016Symeon Papadopoulos
 
Placing Images with Refined Language Models and Similarity Search with PCA-re...
Placing Images with Refined Language Models and Similarity Search with PCA-re...Placing Images with Refined Language Models and Similarity Search with PCA-re...
Placing Images with Refined Language Models and Similarity Search with PCA-re...Symeon Papadopoulos
 
In-depth Exploration of Geotagging Performance
In-depth Exploration of Geotagging PerformanceIn-depth Exploration of Geotagging Performance
In-depth Exploration of Geotagging PerformanceSymeon Papadopoulos
 
Perceived versus Actual Predictability of Personal Information in Social Netw...
Perceived versus Actual Predictability of Personal Information in Social Netw...Perceived versus Actual Predictability of Personal Information in Social Netw...
Perceived versus Actual Predictability of Personal Information in Social Netw...Symeon Papadopoulos
 
Web and Social Media Image Forensics for News Professionals
Web and Social Media Image Forensics for News ProfessionalsWeb and Social Media Image Forensics for News Professionals
Web and Social Media Image Forensics for News ProfessionalsSymeon Papadopoulos
 
Predicting News Popularity by Mining Online Discussions
Predicting News Popularity by Mining Online DiscussionsPredicting News Popularity by Mining Online Discussions
Predicting News Popularity by Mining Online DiscussionsSymeon Papadopoulos
 
Finding Diverse Social Images at MediaEval 2015
Finding Diverse Social Images at MediaEval 2015Finding Diverse Social Images at MediaEval 2015
Finding Diverse Social Images at MediaEval 2015Symeon Papadopoulos
 

Mehr von Symeon Papadopoulos (20)

DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...
DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...
DeepFake Detection: Challenges, Progress and Hands-on Demonstration of Techno...
 
Deepfakes: An Emerging Internet Threat and their Detection
Deepfakes: An Emerging Internet Threat and their DetectionDeepfakes: An Emerging Internet Threat and their Detection
Deepfakes: An Emerging Internet Threat and their Detection
 
Knowledge-based Fusion for Image Tampering Localization
Knowledge-based Fusion for Image Tampering LocalizationKnowledge-based Fusion for Image Tampering Localization
Knowledge-based Fusion for Image Tampering Localization
 
Deepfake Detection: The Importance of Training Data Preprocessing and Practic...
Deepfake Detection: The Importance of Training Data Preprocessing and Practic...Deepfake Detection: The Importance of Training Data Preprocessing and Practic...
Deepfake Detection: The Importance of Training Data Preprocessing and Practic...
 
COVID-19 Infodemic vs Contact Tracing
COVID-19 Infodemic vs Contact TracingCOVID-19 Infodemic vs Contact Tracing
COVID-19 Infodemic vs Contact Tracing
 
Similarity-based retrieval of multimedia content
Similarity-based retrieval of multimedia contentSimilarity-based retrieval of multimedia content
Similarity-based retrieval of multimedia content
 
Twitter-based Sensing of City-level Air Quality
Twitter-based Sensing of City-level Air QualityTwitter-based Sensing of City-level Air Quality
Twitter-based Sensing of City-level Air Quality
 
Aggregating and Analyzing the Context of Social Media Content
Aggregating and Analyzing the Context of Social Media ContentAggregating and Analyzing the Context of Social Media Content
Aggregating and Analyzing the Context of Social Media Content
 
Verifying Multimedia Content on the Internet
Verifying Multimedia Content on the InternetVerifying Multimedia Content on the Internet
Verifying Multimedia Content on the Internet
 
A Web-based Service for Image Tampering Detection
A Web-based Service for Image Tampering DetectionA Web-based Service for Image Tampering Detection
A Web-based Service for Image Tampering Detection
 
Learning to detect Misleading Content on Twitter
Learning to detect Misleading Content on TwitterLearning to detect Misleading Content on Twitter
Learning to detect Misleading Content on Twitter
 
Near-Duplicate Video Retrieval by Aggregating Intermediate CNN Layers
Near-Duplicate Video Retrieval by Aggregating Intermediate CNN LayersNear-Duplicate Video Retrieval by Aggregating Intermediate CNN Layers
Near-Duplicate Video Retrieval by Aggregating Intermediate CNN Layers
 
Verifying Multimedia Use at MediaEval 2016
Verifying Multimedia Use at MediaEval 2016Verifying Multimedia Use at MediaEval 2016
Verifying Multimedia Use at MediaEval 2016
 
Multimedia Privacy
Multimedia PrivacyMultimedia Privacy
Multimedia Privacy
 
Placing Images with Refined Language Models and Similarity Search with PCA-re...
Placing Images with Refined Language Models and Similarity Search with PCA-re...Placing Images with Refined Language Models and Similarity Search with PCA-re...
Placing Images with Refined Language Models and Similarity Search with PCA-re...
 
In-depth Exploration of Geotagging Performance
In-depth Exploration of Geotagging PerformanceIn-depth Exploration of Geotagging Performance
In-depth Exploration of Geotagging Performance
 
Perceived versus Actual Predictability of Personal Information in Social Netw...
Perceived versus Actual Predictability of Personal Information in Social Netw...Perceived versus Actual Predictability of Personal Information in Social Netw...
Perceived versus Actual Predictability of Personal Information in Social Netw...
 
Web and Social Media Image Forensics for News Professionals
Web and Social Media Image Forensics for News ProfessionalsWeb and Social Media Image Forensics for News Professionals
Web and Social Media Image Forensics for News Professionals
 
Predicting News Popularity by Mining Online Discussions
Predicting News Popularity by Mining Online DiscussionsPredicting News Popularity by Mining Online Discussions
Predicting News Popularity by Mining Online Discussions
 
Finding Diverse Social Images at MediaEval 2015
Finding Diverse Social Images at MediaEval 2015Finding Diverse Social Images at MediaEval 2015
Finding Diverse Social Images at MediaEval 2015
 

Kürzlich hochgeladen

Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machinePadma Pradeep
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Mattias Andersson
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024BookNet Canada
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Enterprise Knowledge
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024Lorenzo Miniero
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationSafe Software
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Wonjun Hwang
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek SchlawackFwdays
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Commit University
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Mark Simos
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesZilliz
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii SoldatenkoFwdays
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):comworks
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embeddingZilliz
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfRankYa
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsSergiu Bodiu
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024The Digital Insurer
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...Fwdays
 

Kürzlich hochgeladen (20)

Install Stable Diffusion in windows machine
Install Stable Diffusion in windows machineInstall Stable Diffusion in windows machine
Install Stable Diffusion in windows machine
 
Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?Are Multi-Cloud and Serverless Good or Bad?
Are Multi-Cloud and Serverless Good or Bad?
 
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
New from BookNet Canada for 2024: BNC CataList - Tech Forum 2024
 
Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024Designing IA for AI - Information Architecture Conference 2024
Designing IA for AI - Information Architecture Conference 2024
 
SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024SIP trunking in Janus @ Kamailio World 2024
SIP trunking in Janus @ Kamailio World 2024
 
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptxE-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
E-Vehicle_Hacking_by_Parul Sharma_null_owasp.pptx
 
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry InnovationBeyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
Beyond Boundaries: Leveraging No-Code Solutions for Industry Innovation
 
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
Bun (KitWorks Team Study 노별마루 발표 2024.4.22)
 
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
"Subclassing and Composition – A Pythonic Tour of Trade-Offs", Hynek Schlawack
 
Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!Nell’iperspazio con Rocket: il Framework Web di Rust!
Nell’iperspazio con Rocket: il Framework Web di Rust!
 
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
Tampa BSides - Chef's Tour of Microsoft Security Adoption Framework (SAF)
 
Vector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector DatabasesVector Databases 101 - An introduction to the world of Vector Databases
Vector Databases 101 - An introduction to the world of Vector Databases
 
DMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special EditionDMCC Future of Trade Web3 - Special Edition
DMCC Future of Trade Web3 - Special Edition
 
"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko"Debugging python applications inside k8s environment", Andrii Soldatenko
"Debugging python applications inside k8s environment", Andrii Soldatenko
 
CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):CloudStudio User manual (basic edition):
CloudStudio User manual (basic edition):
 
Training state-of-the-art general text embedding
Training state-of-the-art general text embeddingTraining state-of-the-art general text embedding
Training state-of-the-art general text embedding
 
Search Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdfSearch Engine Optimization SEO PDF for 2024.pdf
Search Engine Optimization SEO PDF for 2024.pdf
 
DevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platformsDevEX - reference for building teams, processes, and platforms
DevEX - reference for building teams, processes, and platforms
 
My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024My INSURER PTE LTD - Insurtech Innovation Award 2024
My INSURER PTE LTD - Insurtech Innovation Award 2024
 
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks..."LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
"LLMs for Python Engineers: Advanced Data Analysis and Semantic Kernel",Oleks...
 

Geographical Data Management for Web Applications

  • 1. Web Information Systems prof. Athena Vakali Geographical Data Management for Web Applications Symeon Papadopoulos Informatics Department, AUTh 13 Dec 2011
  • 2. Overview • Introduction • Representation & storage of geographical data • Geographical data management with Java (JDBC) • Text Indexing for Geographical Data • Google Maps API • Use Cases: Geographical Data Clustering + Heat maps Web Information Systems
  • 3. The Era of Geo • 350K websites make use of Google Maps API (2010) • GPS-enabled mobile phones > 200M (2010) • Foursquare users: 5M (2010)  15M (2011) • The HotPotato check-in service was acquired by Facebook for $10Μ (2010) Web Information Systems
  • 4. Categories of Geo-Services • Primary services (providers) – Google Maps, Bing Maps, Yahoo Maps, MapQuest, OpenStreetMap • Derivative services (mash-ups) • Location-based social networks – Foursquare, Gowalla (acquired by Facebook), Brightkite • Intelligent geo-based services – Analytics, Route planning, Real estate Web Information Systems
  • 5. GIS • GIS offer sophisticated software for geographical data management and processing. • They offer many means of representation, search and rendering. • They mostly target geo-centered applications: – Urban planning, environmental studies, etc. • Popular tools: ArcGIS, MapInfo, GeoMedia, κλπ. • GIS will not be discussed in this lecture. Web Information Systems
  • 6. Representation and Storage of Geographical Data
  • 7. Lat, Lon • Geo coordinates constitute the basic element of any geo-enabled service. • A coordinate element is defined by the following: Latitude Longitude Web Information Systems
  • 8. Lat, Lon (ΙΙ) • Traditional writing: 40° 38′ 0″ N, 22° 57′ 0″ E • Decimal representation: 40.633333, 22.95 • Important note for storage: Precision of 0.00001 in the decimal representation corresponds to approximately 1m distance (near the equator) Web Information Systems
  • 9. Altitude, Bearing • Two additional variables related to geographical points (still not very common): – Altitude: vertical distance of point from the level of sea – Bearing: direction to which the user/device is pointed • The increasing use of GPS-enabled devices is expected to increase the importance of these variables, especially in Augmented Reality (AR) applications. Web Information Systems
  • 10. Distance calculation (lat1, lon1) d • Haversine formula: – Δlat = lat2 – lat1, Δlon = lon2 – lon1 – a = sin2(Δlat/2) + cos(lat1)  cos(lat2)  sin2(Δlon/2) (lat2, lon2) – c = 2  atan2(sqrt(a), sqrt(1-a)) – d=Rc [R: earth radius (6 371 km)] • In JavaScript: More accurate and computational expensive – var R = 6371; // km alternative: Vincenty formula – var dLat = (lat2-lat1).toRad(); – var dLon = (lon2-lon1).toRad(); – var a = Math.sin(dLat/2) * Math.sin(dLat/2) + Math.cos(lat1.toRad()) * Math.cos(lat2.toRad()) * Math.sin(dLon/2) * Math.sin(dLon/2); – var c = 2 * Math.atan2(Math.sqrt(a), Math.sqrt(1-a)); – var d = R * c; Web Information Systems
  • 11. Geo in DB • Representation in table form • e.g. List of Points of interest  Table of Pois • double  slightly faster • DECIMAL  higher accuracy Field type: double ή DECIMAL(13,9) Web Information Systems
  • 12. Use cases • Query 1: What points of interest can I see near point of interest X? – Online city exploration in cities (e.g. recommend new Pois based on proximity) • Query 2: What points of interest are there near me? – Valuable within location-based applications (e.g. I need recommendations for sightseeing, eating, etc. based on my current location) Web Information Systems
  • 13. Geo in DB – Proximity Queries (I) • Implementation 1 – Compute distance by use of SQL and return ordered results: • set @lat0 = 40.645466; set @lon0 = 22.858342; set @dist = 10; • SELECT *, 6371  2  ASIN ( SQRT( POWER( SIN(@lat0 – abs(dest.lat))  pi()/180 / 2), 2) + COS (@lat0  pi()/180 )  COS(abs(dest.lat)  pi()/180)  POWER( SIN(@lon0 – dest.lon)  pi()/180 / 2), 2) )) as distance FROM pois dest HAVING distance < @dist Haversine formula ORDER BY distance LIMIT 10; Web Information Systems
  • 14. Geo in DB – Proximity Queries (II) • Problems: – Painfully slow! For tables of even modest sizes (~1000), response might take as long as 8-10 seconds. – Distance computations are not reused in successive invocations of the same query, since caching computations is not supported – Not an option for web applications. Web Information Systems
  • 15. Geo in DB – Proximity Queries (IIΙ) • Alternative implementation: – If, instead of nearby Pois, we ask for Pois that lie within a certain radius from a fixed point (e.g. current location)? 300m Aghia 300m Sophia Web Information Systems
  • 16. Geo in DB – Proximity Queries (IV) • Bounding box query: – SELECT * FROM pois WHERE pois.latitude > @min_lat AND pois.latitude < @max_lat AND pois.longitude > @min_lon AND pois.longitude < @max_lon; (@max_lat, @max_lon) – The question now is how to determine min_lat, min_lon, max_lat, max_lon. (@min_lat, @min_lon) Web Information Systems
  • 17. Geo in DB – Proximity Queries (Vα) • Bounding box computation (top and bottom): • 1°difference in latitude ~ 111km (110.567 near the equator, 111.699 near the poles)  – 300m  ~0.0027° latitude (since our fixed point is located at approximately 45°) @max_lat – @min_lat = @lat0 – 0.0027 – @max_lat = @lat0 + 0.0027 e.g. for lat0 = 40.632844 (Aghia Sophia) @lat0 @min_lat ~ 40.630144 @max_lat ~ 40.635544 @min_lat Web Information Systems
  • 18. Geo in DB – Proximity Queries (Vβ) • Bounding box computation (left and right): • 1°difference in longitude  depends a lot on the latitude of the fixed point, e.g ~111.320 km near the equator and 0 near the poles. Distance (km) for 1° – Table with distance correspondences difference in longitude (the more lines, the better the achieved approximation – for intermediate values one needs to use interpolation) – 300m  ~0.003805° latitude since the fixed point lies at latitude~45° – @min_lon = @lon0 – 0.003805 – @max_lon = @lon0 + 0.003805 – e.g. for lon0 = 22.947094 (Aghia Sophia) @max_lon @min_lon ~ 22.943289 @max_lon ~ 22.950899 @min_lon @lon0 Web Information Systems
  • 19. Geo in DB – Proximity Queries (VI) • In summary: – During the table creation, we make sure to index the latitude and longitude columns (conventional B-tree). – We convert the proximity query to a bounding box query by use of geographical approximations (that can be made sufficiently precise if needed). – In this form, the SQL query can be executed really fast thanks to the indexes. Web Information Systems
  • 20. Geo-enabled DBs • PostGIS (Extension for PostgreSQL) – Support for “geographical objects”: • POINT (X Y) • LINESTRING (0 0, 1 1, 1 2) • POLYGON (0 0, 4 0, 4 4, 0 4, 0 0) • MULTIPOINT, MULTIPOLYGON, GEOMETRYCOLLECTION – Support for tables and calculations (e.g. proximity queries) directly through the query engine. • GeoServer (GIS) – Complete Java framework for search of geographical objects and map rendering based on a server-client architecture. Web Information Systems
  • 22. Typical lifecycle of geographical data Relational model specification JDBC Geo data source Transformation to RDBMS storage (e.g. Wikimapia) records JDBC UPDATE READ Processing JAVA Web Information Systems
  • 23. Data model specification • Very simple example: table with id + 3 columns – Name: VARCHAR (100) – Latitude: DOUBLE – Longitude: DOUBLE Pois Web Information Systems
  • 24. Input data + transformation parsing <name>Benaki museum</name>  “Benaki Museum” <lon>23.70825</lon>  23.70825 <lat>37.974515</lat>  37.974515 Record name latitude longitude Benaki museum 23.70825 37.974515 Web Information Systems
  • 25. Record insertion with JDBC (1) • Step 1 Connection Creation (A) Load the appropriate JDBC driver E.g. in the following example: JDBC_DRIVER = “com.mysql.jdbc.Driver” Class.forName(JDBC_DRIVER); (B) Connection initialization Τhe variable dbURL has the form: protocol + vendor + server + port nr + dbName + params e.g. jdbc:mysql//myserver.gr:3306/testdb/? useUnicode=true&characterEncoding=UTF-8 Connection con = DriverManager.getConnection(dbURL, username, password); throws exceptions (InstantiationException, IllegalAccessException, ClassNotFoundException, SQLException) ATTENTION: Connection objects should be reused! Web Information Systems
  • 26. Record insertion with JDBC (2) • Step 2 Creation and execution of statement (A) Statement creation String sqlInsert = “INSERT INTO Pois VALUES (‘Benaki museum’, 23.70825, 37.974515)”; Statement insertStmt = con.createStatement(); (B) Statement execution (throws SQLException) insertStmt.executeUpdate(sqlInsert); Alternatively PreparedStatement prepInsert = con.prepareStatement(sqlInsert, Statement.RETURN_GENERATED_KEYS); prepInsertStmt.executeUpdate(); ResultSet rs = prepInsertStmt.getGeneratedKeys(); long recordId = -1; Record database key  could if (rs.next()) recordId = rs.getLong(1); be useful later (better keep it than query for it) Web Information Systems
  • 27. Wrapper method creation • Encapsulation of process for easy reuse public long insertPoi(Connection con, String poiName, double lat, double lon) { String sqlInsert = “INSERT INTO Pois VALUES (‘” + poiName + “’,” String.valueof(lat) + “,” + String.valueof(lon) + “)”; PreparedStatement prepInsert = con.prepareStatement(sqlInsert, Statement.RETURN_GENERATED_KEYS); prepInsertStmt.executeUpdate(); ResultSet rs = prepInsertStmt.getGeneratedKeys(); long recordId = -1; if (rs.next()) recordId = rs.getLong(1); return recordId; } Web Information Systems
  • 28. Batch insertion • In case of massive insertions, it is better to make use of the batch mode of insertion in order to significantly speed up the process. con.setAutoCommit(false); Very important!! Statement batchInsert = con.createStatement(); for (int i = 0; i < 10,000; i++){ batchInsert.addBatch(“INSERT INTO Pois VALUES (‘“ + names*i+ + “’,” + String.valueof(lats*i+) + “,” + String.valueof(lons*i+) + “)”; } int[] updateCount = batchInsert.executeBatch(); con.commit(); con.setAutoCommit(true); Web Information Systems
  • 29. ResultSet structure Initial cursor position C1 C2 C3 C4 C5 C6 C7 …. R1 22 ab 0.1 R2 27 cd 0.5 rs.next() R3 28 gc 1.2 R4 40 ob 3.5 R5 51 gz 3.2 rs.absolute(3) rs.getInt(1) rs.getDouble(3) rs.getInt(“C1”) rs.getDouble(“C3”) rs.getString(2) (faster) rs.getString(“C2”) Web Information Systems
  • 30. Record reading • Use SELECT statement and iterate through ResultSet (A) SELECT statement creation Statement selectStmt = con.createStatement(); (B) Form and execute SELECT statement List<Poi> pois = new ArrayList<Poi>(); String sqlSelect = “SELECT * FROM Pois”; ResultSet rs = stmt.executeQuery(sqlSelect); while ( rs.next() ) { long poiId = rs.getLong(“id”); String poiName = rs.getString(“name”); double lat = rs.getDouble(“latitude”); Candidate wrapper method Poi extractPoiFromResultSet(ResultSet rs) double lon = rs.getDouble(“longitude”); Poi poi = new Poi(poiId, poiName, lat, lon); pois.add(poi); } Web Information Systems
  • 31. Record updates • Suppose I have collected data about the popularity of a Poi (e.g. count the number of results from a search engine) and I have its id at my disposal: public void updatePoiPopularity(Connection con, long poiId, int popularity) { PreparedStatement updateStatement = con.prepareStatement( “UPDATE Pois SET poi_popularity = ? WHERE id = ?”); updateStatement.setInt(1, popularity); updateStatement.setLong(2, poiId); int n = updateStatement.executeUpdate(); } NEW COLUMN name latitude longitude poi_popularity …. … … … Web Information Systems
  • 32. Example Design WikimapiaWrapper PopWrapper PoiDbWrapper • Task separation List<Poi> pois = WikimapiaWrapper.collectPois(“Thessaloniki”); for (int i = 0; i < pois.size(); i++){ long poiId = PoiDbWrapper.insertPoi(con, pois.get(i).getName(), pois.get(i).getLatitude(), pois.get(i).getLongitude()); pois.get(i).setId(poiId); } …. Map<Poi,Integer> poiPopMap = PopWrapper.collectPoiPopularities(pois); for (Entry<Poi,Integer> poiPopEntry : poiPopMap.entrySet()){ PoiDbWrapper.updatePoiPopularity(con, poiPopEntry.getKey().getId(), poiPopEntry.getValue()); } Web Information Systems
  • 33. Text Indexing for Geographical Data
  • 34. Full-text Indexing • Extremely valuable technology for applications requiring full- text search features. • Classical DB systems do not offer a standard means of indexing and searching large amounts of text. • Established technology: Inverted indices • Popular implementations: – Lucene, Solr (Java) – Sphinx (C++) – OpenFTS (PostgreSQL) – Lemur (C++) Web Information Systems
  • 35. Text queries in RDBMS • Operators LIKE and RLIKE – SELECT * FROM pois WHERE name LIKE ‘%museum%’ – SELECT * FROM pois WHERE name RLIKE ‘museum’ • Conventional indexing structures in RDBMS are not particularly helpful in quickly responding to such queries (might require visiting every record  too slow for large tables) • Each vendor offers own solution, e.g. FULLTEXT INDEX από MySQL. Web Information Systems
  • 36. Inverted index (I) • Starting point: Document collection (corpus) • Text processing  Dictionary: set of unique terms appearing in the documents of the collection + additional information (stats, stems, stop words) • Index: Document retrieval mechanism based on the terms contained in the documents Web Information Systems
  • 37. Inverted index (II) • Inverted index: Set of inverted lists. • Inverted list: A list containing information about the occurrence of a term within the documents of the collection – represented as a list of postings. • Posting: Occurrence of a term in a certain position of a document. Apart from the document, where the term occurs, the posting also contains its position in the text  support for phrase search (increases the size of the index and complexity of the query mechanism). Web Information Systems
  • 38. Inverted index (IIΙ) Example • Document collection: – Document 1: “White Tower” – Document 2: “Byzantine Museum” – Document 3: “Archeological Museum” • Inverted lists of postings: – “Archeological”: {(3,1)} – “Byzantine”: {(2,1)} – “Museum”: {(2,2), (3,2)} – “White”: {(1,1)} – “Tower”: {(1,2)} Document id Term position within the document Web Information Systems
  • 39. Inverted index (IV) Query example • Query: “Museum”  documents 2 and 3 • Query: “Byzantine Museum”  – Result 1: document 2 (contains both terms) – Result 2: document 3 (contains only the term “museum”) • “Archeological”: {(3,1)} • Document 1: “White Tower” • “Byzantine”: {(2,1)} • Document 2: “Byzantine Museum” • “Museum”: {(2,2), (3,2)} • Document 3: “Archeological Museum” • “White”: {(1,1)} • “Tower”: {(1,2)} Web Information Systems
  • 40. Solr (text search engine) • Complete framework for indexing and searching massive document collections. • Based on the Lucene library. • Flexible document representation structure. • Flexible query structure. • Can be installed as a servlet in an appropriate container (e.g. Tomcat, Jetty)  Possibility for use over HTTP Web Information Systems
  • 41. Solr basics (Ι) • Every document consists of a set of fields (similar to Lucene) • Every field can be of different type (e.g. text, number, etc.) • Every field can processed (e.g. tokenized, normalized) and indexed in a different way. • Every field or combination of fields can be queried in a different way  flexible query system Web Information Systems
  • 42. Solr basics (ΙΙ) Example: – id: 13424 – name: White Tower of Thessaloniki – latitude: 40.626369 – longitude: 22.948428 – history: The tower, which once guarded the eastern end of the city's sea walls, was for many years attributed to Venice, to which the Byzantines ceded Thessaloniki in 1423. – wikipedia_link: http://en.wikipedia.org/wiki/White_Tower_of_Thessaloniki Xml representation suitable fοr use by Solr: <add><doc> <field name=“id”>13424</field> <field name=“name”>White Tower of Thessaloniki</field> ... </doc></add> Web Information Systems
  • 43. Solr: index management • Addition of documents: java -jar post.jar lefkos_pirgos.xml – The xml file should follow the structure of the example in the previous example. – A list of files can be provided as input (whitespace separated). – If a document with the same id (unique key) is provided as input, then the new document will replace the existing. • Deletion of documents: java -Ddata=args -Dcommit=no -jar post.jar "<delete><id>13424</id></delete>" • Committing changes: java –jar post.jar • The above operations (and more) are available through a Java (not only) API as well. Web Information Systems
  • 44. Solr: Queries HTTP API: Base URL: http://server:8983/solr/select/? Return fields “name”, “id” of all entries containing ‘museum’. • q=museum&fl=name,id As above but also return relevance score for each entry. • q=museum&fl=name,id,score As above but return all stored field. • q=museum&fl=*,score Return entries in JSON format. • q=museum&wt=json Return all entries containing the keyword ‘pirgos’ in their field “name”. • q=name:pirgos Web Information Systems
  • 45. Challenges • Text indexing and search frequently encounters the following two challenges: – Synonyms, writing variations: users search for something using a form different than the “reference” form. e.g. White Tower, Λευκός Πύργος, λευκός πύργος, lefkos pyrgos, ...  Solution: Attempt to collect and index many different lexical forms of the same term. – Polysemy/Ambiguity: A single term refers to more than one concepts. e.g. the term museum may refer to a number of real-world museums Solution: Appropriate interfaces are necessary to make the user aware of the different senses/contexts of a term and ask for query reformulation/specification (could also come up with suggestions). Web Information Systems
  • 46. Synonyms • Strategy 1 (simple but not always available): – Use dictionary/thesaurus/gazetteer (e.g. Geonames) Contains alternative names and variations for a large number of Pois and locations. – Addition of a “synonyms” field in the index – Include the additional field in the search …but give a higher weight to the “name” field. Web Information Systems
  • 47. Related terms • Strategy 2 (more complex but generally applicable): – Issue query to search engine or other content application (e.g. Flickr) • Use Poi name as query. – Process search results in two ways: • Find terms or term sequences that frequently occur in the same context (e.g. web page, title of photo, etc.). • Find terms that are lexically similar to the query term (e.g. based on the Levenshtein distance) . – Addition of a “similars” field in the index. – Include the additional field in the search • but give higher weight to the “name” field Web Information Systems
  • 49. Licensing • The Google Maps service is free for web pages that are freely available. Charges apply for commercial usage of the API. • Since V3, the API does not require a key. Web Information Systems
  • 50. Google Maps API (I) • The map service is based on the notion of map tiles, which, when stitched together, provide an impression of a continuous map. Web Information Systems
  • 51. Google Maps API (II) • There are different types of map types, e.g. ROADMAP, SATELLITE, HYBRID, TERRAIN. Those are called base map tiles. ROADMAP HYBRID HYBRID TERRAIN Web Information Systems
  • 52. Google Maps API – Coordinates (I) Coordinates: • World: Point 0,0  lat ~ 85° lon 180° • Pixel: pixelCoord = worldCoord  2zoomLevel The API first determines the visible area of the map on the client (viewport) based on the zoom level, the map center and the size (in pixels) of the DOM element in the client web page. Web Information Systems
  • 53. Google Maps API – Coordinates (II) Coordinates: • Tile coordinates: – With every scroll or zoom to a new viewport, the API needs to render a new area on the map. For this to happen, it needs to translate the pixel coordinates into appropriate tile coordinates in order to load the appropriate tiles. At each zoom level the tile coordinates are different. – e.g. at zoom level 2 (beside) 16 tiles need to be indexed. The tile containing Greece has coordinates (2,1). At zoom level 3, 16 x 4 = 64 tiles need to to be indexed, and so on. Web Information Systems
  • 54. The “hello world” of Google Maps • The map object is integrated to any web page as a DOM element. <html> <head> <meta name="viewport" content="initial-scale=1.0, user-scalable=no" /> <style type="text/css"> html { height: 100% } body { height: 100%; margin: 0px; padding: 0px } #map_canvas { height: 100% }</style> Library import <script type="text/javascript" src="http://maps.google.com/maps/api/js?sensor=false"></script> <script type="text/javascript"> function initialize() { Map initialization and centering var latlng = new google.maps.LatLng(40.66, 22.95); var myOptions = { zoom: 12, center: latlng, mapTypeId: google.maps.MapTypeId.ROADMAP }; var map = new google.maps.Map(document.getElementById("map_canvas"), myOptions); } </script> </head> <body onload="initialize()"> Map canvas DOM element <div id="map_canvas" style="width:100%; height:100%"></div> </body> </html> Spans the whole page body Web Information Systems
  • 55. Basic objects • google.maps.LatLng – Initialization: var latlng = new google.maps.LatLng(lat_p, lon_p); – Very useful object, e.g. for marker placement. – In case we don’t know the geo coordinates, but know the street name, we could make use of a Geocoding service. • google.maps.Map – Initialization: var mapx = new google.maps.Map(domElement, options); – Basic parameters: zoom level, center, mapType (ROADMAP, HYBRID, etc.) – It is possible to have more than one map objects on the same page. Web Information Systems
  • 56. Zoom level • Very important to select the proper zoom level depending on the application needs. zoom = 4 zoom = 8 zoom = 12 zoom = 16 zoom = 18 zoom = 20 Web Information Systems
  • 57. More basic objects • google.maps.Marker – Initialization: var latlng = new google.maps.Marker({ position: latlng, map: mapx, title: “POI Title” }); • Very useful object! • Use it for marking Pois. Web Information Systems
  • 58. Events on the map UI • Interacting with the map, as in every UI, triggers a variety of events. • There are two basic event categories: – User/UI events: Objects on the map respond to certain user actions. – MVC events: Triggered by changes in the state of the map, e.g. change of value for some parameter (zoom level). Web Information Systems
  • 59. Map events • UI events Look like common DOM events but are separately triggered in order to prevent incompatibility with different browsers. e.g. the Marker object responds to the following events: – “click”, – “dblclick”, – “mouseup”, – “mousedown”, – “mouseover”, – “mouseout”. Web Information Systems
  • 60. Event listening (I) • addListener() – Object that “listens” – event that is “listened” MVC event – Method that is invoked as soon as the event occurs • Example: google.maps.event.addListener(map, 'zoom_changed', function() { setTimeout(moveToDarwin, 3000); // fire moveToDarwin() after 3 secs }); google.maps.event.addListener(marker, 'click', function() { map.setZoom(8); }); UI event function moveToDarwin() { var darwin = new google.maps.LatLng(-12.461334, 130.841904); map.setCenter(darwin); } Web Information Systems
  • 61. Event listening (II) • Argument passing – Only for UI events – Access properties and their values as in any JavaScript object • Example: google.maps.event.addListener(map, 'click', function(event) { placeMarker(event.latLng); }); function placeMarker(location) { var marker = new google.maps.Marker({ position: location, Marker creation at the point clicked by map: map the user. }); map.setCenter(location); Map centering on the same point. } Web Information Systems
  • 62. Event listening (IΙI) • Read and write properties within MVC events – The value that we read may be different than the value at the time of event triggering (usually we care only for the current value). • Example: google.maps.event.addListener(map, 'zoom_changed', function() { zoomLevel = map.getZoom(); alert("Zoom: " + zoomLevel); if (zoomLevel == 0) { map.setZoom(10); } }); Web Information Systems
  • 63. Event listening (IV) • Listen to common DOM events – addDomListener() window, document.body, etc. DOM element • Example: named elements <script> event name function initialize() { // map initialization method to be invoked } google.maps.event.addDomListener(window, 'load', initialize); </script> <body> <div id="map_canvas"></div> </body> Web Information Systems
  • 64. Control elements (I) • UI elements that enable users to interact with the viewport on the map. • Navigation control: pan & zoom • Scale control • MapType control Web Information Systems
  • 65. Control elements (II) • Determine active controls by use of approprate arguments in the map object options (only at create time): { navigationControl: boolean, mapTypeControl: boolean, scaleControl: boolean } • Example: function initialize() { var myOptions = { zoom: 4, center: new google.maps.LatLng(-33, 151), navigationControl: false, scaleControl: true, mapTypeId: google.maps.MapTypeId.ROADMAP } var map = new google.maps.Map(document.getElementById("map_canvas"), myOptions); } Web Information Systems
  • 66. Control elements (IΙI) • Control elements offer additional options: – google.maps.NavigationControlStyle.SMALL  mini-zoom form – google.maps.NavigationControlStyle.ZOOM_PAN  standard form of zoom & pan – google.maps.MapTypeControlStyle.DROPDOWN_MENU  select map types from dropdown menu • Example: var myOptions = { ... mapTypeControl: true, mapTypeControlOptions: { style: google.maps.MapTypeControlStyle.DROPDOWN_MENU }, navigationControl: true, navigationControlOptions: { style: google.maps.NavigationControlStyle.SMALL }, .... } var map = new google.maps.Map(document.getElementById("map_canvas"), myOptions); Web Information Systems
  • 67. Control elements (IV) • Control elements can be placed in different positions on the map: navigationControl: true, navigationControlOptions: { style: google.maps.NavigationControlStyle.ZOOM_PAN, position: google.maps.ControlPosition.TOP_RIGHT } Web Information Systems
  • 68. Control elements (V) custom control • It is possible to create custom control elements. • Control elements are essentially div elements. • In order to create them, you need: – Define their appearance by use of CSS. – Determine their interactions by use of event handlers. – Create div element and add it to controls property of the Map object. Web Information Systems
  • 69. Overlays • Objects with specific coordinates. Markers Icons Polylines Polygons InfoWindow Web Information Systems
  • 70. Overlays - Markers • Markers are initialized with their position, the map, and a title as arguments (the title appears in a tooltip upon mouse hover). • It is also possible to animate markers with two predefined animation schemes by use of the method marker.setAnimation(method): – google.maps.Animation.DROP: Drops and stops. – google.maps.Animation.BOUNCE: Bounces. – null: Static. Web Information Systems
  • 71. Overlays - Icons • Markers with a different look. • Set the icon property. • Example: var image = `beachflag.png’; var myLatLng = new google.maps.LatLng(44.890, 22.27); var beachMarker = new google.maps.Marker({ position: myLatLng, map: map, icon: image }); • Options for more complex appearance (e.g. by use of shadow and stacking). Web Information Systems
  • 72. Polylines • Useful for presenting routes, directions, etc. • Example: var flightPlanCoordinates = [ new google.maps.LatLng(37.772323, -122.214897), new google.maps.LatLng(21.291982, -157.821856), new google.maps.LatLng(-18.142599, 178.431), new google.maps.LatLng(-27.46758, 153.027892) ]; var flightPath = new google.maps.Polyline({ path: flightPlanCoordinates, strokeColor: "#FF0000", strokeOpacity: 1.0, strokeWeight: 2 }); flightPath.setMap(map); Web Information Systems
  • 73. Polylines - Processing • It is possible to dynamically modify a polyline. • Read and modify the points in a line: – var path = poly.getPath(); – var latlng_idx = path.getAt(idx)  read element on position idx – path.insertAt(idx)  insert element in position idx – path.removeAt(idx)  delete element on position idx – path.push(latlng)  add element at the end of the line Web Information Systems
  • 74. Polygons • Polygons cover an area. • Example: var triangleCoords = [ new google.maps.LatLng(25.774252, -80.190262), Same start and new google.maps.LatLng(18.466465, -66.118292), new google.maps.LatLng(32.321384, -64.75737), end coordinate new google.maps.LatLng(25.774252, -80.190262) (optional) ]; bermudaTriangle = new google.maps.Polygon({ paths: triangleCoords, strokeColor: "#FF0000", strokeOpacity: 0.8, strokeWeight: 2, fillColor: "#FF0000", fillOpacity: 0.3 }); bermudaTriangle.setMap(map); • Same behaviour as Polylines with respect to processing their elements (e.g. getPath(), etc.). Web Information Systems
  • 75. Info Window (Ι) • Very important for providing information for a Poi upon click. • May contain: (a) simple text, (b) HTML code, or (c) DOM element. • Initialized by use of the following elements: – content: String (simple text/HTML), DOM node – position: If the Info Window is to be opened on top of a marker, then the marker position is provided. – maxWidth: Maximum width in pixels (autowraps text). Web Information Systems
  • 76. Info Window (ΙΙ) • Example: var contentString = '<div id="content">'+ '<div id="siteNotice">'+ '</div>'+ '<h1 id="firstHeading" class="firstHeading">Uluru</h1>'+ '<div id="bodyContent">'+ '<p><b>Uluru</b>, also referred to as <b>Ayers Rock</b>, is a large ' + 'sandstone rock formation in the southern part of the '+ 'Northern Territory, central Australia. It lies 335 km (208 mi) '+ .... + </p>'+ '<p>Attribution: Uluru, <a href="http://en.wikipedia.org/w/index.php?title=Uluru&oldid=297882194">‘ + 'http://en.wikipedia.org/w/index.php?title=Uluru</a> (last visited June 22, 2009).</p>‘ + '</div>‘ + '</div>'; var infowindow = new google.maps.InfoWindow({ content: contentString }); var marker = new google.maps.Marker({ position: myLatlng, map: map, title:"Uluru (Ayers Rock)« }); google.maps.event.addListener(marker, 'click', function() { infowindow.open(map,marker); }); Web Information Systems
  • 77. Layers • Collections of information that are overlaid on top of base map tiles and modify its appearance. They are treated as a single object. • Existing layers: – KmlLayer, GeoRSS • Presentation of feed of geotagged posts on a map. – TrafficLayer – BicyclingLayer Web Information Systems
  • 78. Geocoding • Service for converting between street names and geographical coordinates. • Rate limit: 2500 calls / day • Base URL http://maps.googleapis.com/maps/api/geocode/output?par output: json / xml par  required: address OR latlng, sensor optional: bounds, region, language Web Information Systems
  • 79. Use Cases: Geographical Data Clustering + Heat Maps
  • 80. Geo-Clustering (1) • Represent each location p = (lat, lon) as a feature vector: x = [x1 x2] • Apply some density-based clustering method. – Prefer methods that do not require setting the number of clusters as parameter (e.g. k-means is not really appropriate). Recommend methods: BIRCH and DBSCAN (they require setting other parameters). Web Information Systems
  • 81. Geo-Clustering (2) • Example of using the BIRCH algorithm on a set of geotagged Flickr photos in the vicinity of Thessaloniki. CENTER + 40 ΕKKLISIES + ANO POLI NEA PARALIA KARABOURNAKI ARETSOU d = 0.0015 d = 0.0030 Web Information Systems
  • 82. Geo-Clustering (3) • Example of using clustering in a web application. http://www.clusttour.gr markers + polygons + infowindow Web Information Systems
  • 83. Heat maps • A visual analytics tool. • Useful for visualizing location-dependent quantities, e.g. population density, housing prices, crime rates, pollution, etc. • Requires knowledge or inference of quantity distribution over space. • Overlay on top of the map. Web Information Systems
  • 84. Heat maps • Example: geotagged photos collected from Flickr based on the query “sziget festival”) GRID Web Information Systems
  • 85. Heat maps • Counting frequency in each cell + smoothing based on adjacent cells + color coding Web Information Systems
  • 86. Heat maps (1) Languages in Twitter http://flowingdata.com/2011/10/27/language-communities-of-twitter/ Web Information Systems
  • 87. http://flowingdata.com/2011/12/07/every-death-on-the- Heat maps (2) road-in-great-britain/ Deadly car accidents in the UK Web Information Systems
  • 88. http://www.flickr.com/photos/walkingsf/5925793753/in/set-72157627140310742 Heat maps (3) La Grande Arche de la Defense Versailles Orly Airport Twitter (blue) + Flickr (orange) Web Information Systems