[abstract]
This project is about modelling and visualizing maritime paths of ships from the XVIIIth Century. In this report, we explain the steps of how from a filemaker database containing historical information, we have extracted the data, created a model, stored the information in a postgresql database, computed and visualized the multiple paths from harbours to harbours along the French coast of the Atlantic Ocean.
Introduction
Digital maps have progressed a lot in the last few years and they have reached a level of detail that has never been reached before. Google, Bing, Nokia and others have made a fantastic job mapping all the world, sometimes not just by providing simple maps, but also providing a rich 3D experience, where a user can even have a street level view almost anywhere in the world. In parallel, with the advent of digital maps there has been an on-going effort to try to bring this experience in many different fields, for example with the web 2.0 contributions from projects like OpenStreetMaps and others.
People have built and used different types of maps, including maritime maps for centuries, but recent progress have been made with the apparition of digital mapping, adding a new dimension and many possibilities. Currently, people can find maritime maps and use them for navigation, but sometimes, research is not only about discovering the new, but also about a deep dive in history.
While digital maritime maps are not something of a novelty, providing a glimpse in the past is. How did people navigate in the past? What routes did they choose? This is a question that lied at the very bottom of our research. Though there are many existing projects that try to provide an insight to mapping historical maritime routes, there is no standard, or there is no common database that people can use in order to easily have access to information.
Our goal in this project was to try to provide a way in which people can view information about historical maritime maps. For research purposes, our project is based on data from the 18th century, but this does not make the project loose its general purpose. The framework that we have used and proposed in our research can easily be adapted to any period in history and people can use it to build on top of it. Our research project does not stop with these results. The results that we have obtained so far (which will be presented later in this report) are just to show the huge research potential such projects have.
Before describing our actual solution, we have done some research to see what current research projects exist and what topics do they approach.
State of the art
Since there are many interesting projects that deal with historical maritime routes, historical information about ships and historical maps, we will only present some that are relevant to the work that we have done.
One of the biggest databases on historical ships can be obtained from Navigocorpus. Navigocorpus is a database that provides information from different sources about shipping and maritime trade and their movements through time and space. The information provided is stored chronologically and can be accessed through the use of the software called Filemaker. There is also an online website where data can be accessed from, but it is not as complete as the one accessed through Filemaker 12. The way in which data from Navigocorpus looks can be seen in the image below.
Another very interesting project that deals with maritime maps and routes is ORBIS. ORBIS is a project from Stanford University and it represents a Geospatial Network Model of the Roman World. On this model it tries to reconstruct different types of travel in antiquity, including maritime travel. The model that they provide for mapping sea routes takes into account a lot of different factors, including wind conditions, currents and possible waves. The model that they have is made of 900 sea routes that is based on the book written by Pascal Arnaud. An interesting aspect of ORBIS is that they even provide different costs for travelling, depending on which month the journey has taken place in.
A question one might ask is why do we need another project that deals with historical maritime routes. The reason behind our project is that, first of all, the data provided is easily accessible, but not so easily accessible for future development. Data can be accessed through their map and it provides an interface with different parameters and it produces routes. Data can even be exported, but this does not help much.
How can we add different data? How can we use their model and build on top of it? Our project involves two aspects in order to produce maritime routes. One is data collection and one is data visualisation. Since there is no real access to the database ORBIS is using, we cannot make much use of their approach in our project. However, we can use it to compare some of the results that we have.
Further on, we will describe our solution, the challenges that we have faced, how we have solved them and the future research questions that still remain open.
Project Goal
The goal of our project is to propose a data model for maritime routes, by storing information about ships, harbours, dates when ships have travelled to those harbours etc., and to create a visualisation method that others can use for future research or even for educational purposes. This visualisation imposes finding an algorithm to map paths between two harbours.
Our project has two major components:
– Data collection
– Data visualisation & Route planning
Data Collection
One of the biggest challenges for such a project is getting data. Since our project is small with limited resources, we have decided to try to find and use an already existing database. The work done by Navigocorpus was exactly what we were looking for. The data that they provide contains information about ships, captains, harbours, shipment, tonnage and so on. This data was perfect for the kind of research we are doing.
Since we have found a source for our data, we have created a database using PostgreSQL and the spatial extension PostGIS. Having our own database structure is much more flexible to work with, we can alter data however we want, we can create, merge, query tables without having to deal with the limitations imposed by other projects (e.g. Navigocorpus allows only basic querying, we cannot build complicated queries).
From our research, we have reached the conclusion that our data model will consist of 5 tables. These tables are: “ships”, “captains”, “ships_captains”, “maritime_routes” and “harbours”.
The “ships” table is used to store information about each ship. For the moment we only record the id and the name of the ship. The “captains” table is similar to the ships table, holding the id and the name of captains. In a very similar way we have designed the tables “harbours” which has an id field and a name. The “harbours” table is a bit different for the rest because it also contains a filed for geo referencing, the location. The “location” field is a specific data type used in PostGIS; it is a specific data type called “geometry”. This data type allows storing both the latitude and the longitude of a point’s coordinates into a single field. A more important table is the “ships_captains” table. It stores specific combinations of captains and ships. The last and the most important table from our database schema is the “maritime_routes” one. This entity has a composite key, composed of an id and a sequence_id, which is being used to identify all the routes that ships took. The sequence_id field is perticulary useful when we want to track the specific route a ship took while delivering the goods.
The database model can also be viewed in the diagram below:
Database model together with auxiliary tables
Our database has been populated with several harbours and ships, as it can be seen below.
List of ships and the harbours they have visited
“Destin” routes
The database also contains another table, which stores the coastline of the map. This is not mentioned in the db model because it is not really part of the actual data, but will be further used to find maritime paths.
Data Visualisation & Route Planning
In this section we will describe the way in which we perform route planning and some results of our experiments using a data visualisation tool (Quantum GIS).
For the route-planning problem we have created two different approaches:
1. Use buffers to identify potential paths
2. Use an algorithm, designed and implemented by us, that can identify routes based on the starting harbour, ending harbour and by knowing where the coastline is.
1. Buffers
In geographic science, buffers are areas of influence around a point which can include specific properties. For our case, this was the first idea of how to map these routes. The goal was to define a minimal and a maximal distance from the shore.
We defined the minimal distance as the range of traditional artillery from the XVIIIth century (maritime and coast) like 36-pounders long guns which had a range of a nautical mile (1’852m).
We defined the maximal distance as the usual “cabotage” (short shipping travels for a same country) distance from the shore as a distance of 20 nautical miles (~40km)
This leads us to the following result:
2. Route planning algorithm
While trying to identify a way in which we can create maritime paths between harbours, we have notice that not much information about making such routes is available online.
As a result, we have decided to try and build our own algorithm and compare it with our other approach (using buffers) and with some historical based routing algorithms (those available on ORBIS’s website).
In this approach, we have not used a graph based implementation (have a set of predefined edges/paths and then just perform a minimum distance algorithm that will return a minimum path). A graph based implementation involves a huge amount of a priori work to define the possible paths. ORBIS works in a similar way and its edges/paths are based on historical information. In our approach, we have tried to see if there is an automatic way to generate routes knowing only the position of the harbours (start and finish) and the coastline.
Based on the description provided, below, our algorithm was designed as a function in PostgreSQL. The function takes as input the ids of the harbours and it returns a geometry field that represents the path between the two harbours.
Input: (h1:harbour, h2:harbour, coastline)
Output: (result_path: geometry)
getMaritimePath()
{
create output table path_points
currentPoint = h1
line = st_makeline(currentPoint, h2)
while st_intersects(line, coastline)
{
intersection = st_intersection(line, coastline)
closestPoint = st_closestpoint(currentPoint, intersection);
newline = st_makeline(currentPoint, closestPoint);
newPoint = st_line_interpolate_point(newLine);
// also use a counter as a safety to end the loop
while st_intersects(st_makeline(currentPoint, newPoint),coastline)
{
// try to place point right
newPoint = st_line_interpolate_point(newline, right);
}
// also use a counter as a safety to end the loop
while st_intersects(st_makeline(currentPoint, newPoint),coastline)
{
// try to place point left
newPoint = st_line_interpolate_point(newline, left);
}
insert(newPoint, path_points);
currentPoint = newPoint;
}
// Create a geometry out of all the intermediary points
return st_makeline(path_points);
The algorithm above is described in a very rough pseudo code. The actual code is provided as an attachment. The basic idea is the following. If there is no direct link between the start and finish, then we find where the intersection is. We get the closest point of the intersection and create a line between this point and start. We try to position the new point at a certain length from the start, either to the left or to the right of the newly created line. If the new point is validated, we set this as the start and start over. The algorithm finished when there is no intersection with the coastline, so it means that the path has been found.
The function described earlier uses some of the following PostGIS specific functions: (the code box below allows scrolling to view the whole content)
st_makeline(geom g1, geom g2) : create a new segment based on 2 points expressed as a geometry st_intersects(geom g1, geom g2) : returns a Boolean which tells if there is an intersection between the two geometries st_intersection(geom g1, geom g2) : returns a set of points as a geometry which represents the actual intersection between the two geometries st_closestpoint(geom g1, geom g2) : returns a point from g2 which is the closest to g1 st_line_interpolate_point (geom g, m_offset, m_distance) : returns a new point that is at m_offset% from the segment and at m_distance from the segment (g).
Depending on the m_offset and m_distance parameter, the results can very a bit. A small m_distance will lead to more point, thus a smoother path, since the distance covered until the destination (end harbour) is less.
Below, you can see some of the results that we have obtained. Each picture represents a path that is being computer based on the algorithm described previously. Depending on the parameters of the st_interpolate_line function, the path can be smoother or “edgier”.
Saint Pierre
Saint Pierre
Saint Pierre
Destin
Destin
Antoinette
The algorithm is far from perfect. As one can see, it does generate quite good paths, but it does not take into account complex factors like wind or maybe depth of the water. The algorithm can be further improved and we believe that it can give much better results if we keep working on it. This demo was just to show that with minimal information available (just harbours and coastline) one can generate possible routes and can reveal the paths of ships.
Conclusion
The goal of this project was to realize some visualization of maritime ships from the XVIIIth century, using the data from the Navigocorpus database, to design and implement a postgresql database with its postgis extension to store the geographical component and to reuse it then to compute and map the maritime paths. We georeferenced the harbours using google earth and with help of some GIS programs and we achieved to draw some pathlines using postgis functions.
We have seen that mapping these paths is very difficult, the main informations which are used in other projects are the streams, the historical sailing points and of course the shortest / quickest paths which were the best arguments to assess whether or not a ship was travelling along a specific path.
There’s a big potential of development in this domain, with this project we tried to go as far as possible.
Special Thanks to
Prof. Frédéric Kaplan (DH Lab EPFL)
Mélanie Fournier (DH Lab EPFL)
Prof. Silvia Marzagalli (Université de Nice et responsable Navigocorpus)
_____________________________________________________________________
SQL source code: https://dl.dropboxusercontent.com/u/70900477/source_code.zip
_____________________________________________________________________
References:
1. Navigocorpus: http://navigocorpus.org/Presentation/Database
2. ORBIS: http://orbis.stanford.edu/#
3. PostGIS documentation: http://postgis.net/documentation/
4. PostgreSQL: http://www.postgresql.org/docs/9.3/static/index.html
5. Quantum GIS Buffers: http://docs.qgis.org/2.0/html/en/docs/gentle_gis_introduction/09_vector_spatial_analysis_buffers.html
6. PostGIS interpolate point along a line
http://trac.osgeo.org/postgis/wiki/UsersWikiExamplesInterpolateWithOffset
