Get me outta here!

Mapping European Navy

They were sailors

Menu

Skip to content
  • Research question
  • State of the art
  • Methodology
  • Plan and milestones

Author Archives

Arnaud Poncet-Montangeshttp://arnaudponcetmontanges.wordpress.comStudent @ EPFL in Environmental Sciences and Engineering

Final Report

15 May 2013 by Arnaud Poncet-Montanges

[abstract]

This project is about modelling and visualizing maritime paths of ships from the XVIIIth Century. In this report, we explain the steps of how from a filemaker database containing historical information, we have extracted the data, created a model, stored the information in a postgresql database, computed and visualized the multiple paths from harbours to harbours along the French coast of the Atlantic Ocean.

Introduction

Digital maps have progressed a lot in the last few years and they have reached a level of detail that has never been reached before. Google, Bing, Nokia and others have made a fantastic job mapping all the world, sometimes not just by providing simple maps, but also providing a rich 3D experience, where a user can even have a street level view almost anywhere in the world. In parallel, with the advent of digital maps there has been an on-going effort to try to bring this experience in many different fields, for example with the web 2.0 contributions from projects like OpenStreetMaps and others.

People have built and used different types of maps, including maritime maps for centuries, but recent progress have been made with the apparition of digital mapping, adding a new dimension and many possibilities. Currently, people can find maritime maps and use them for navigation, but sometimes, research is not only about discovering the new, but also about a deep dive in history.

While digital maritime maps are not something of a novelty, providing a glimpse in the past is. How did people navigate in the past? What routes did they choose? This is a question that lied at the very bottom of our research. Though there are many existing projects that try to provide an insight to mapping historical maritime routes, there is no standard, or there is no common database that people can use in order to easily have access to information.

Our goal in this project was to try to provide a way in which people can view information about historical maritime maps. For research purposes, our project is based on data from the 18th century, but this does not make the project loose its general purpose. The framework that we have used and proposed in our research can easily be adapted to any period in history and people can use it to build on top of it. Our research project does not stop with these results. The results that we have obtained so far (which will be presented later in this report) are just to show the huge research potential such projects have.

Before describing our actual solution, we have done some research to see what current research projects exist and what topics do they approach.

State of the art

Since there are many interesting projects that deal with historical maritime routes, historical information about ships and historical maps, we will only present some that are relevant to the work that we have done.

One of the biggest databases on historical ships can be obtained from Navigocorpus. Navigocorpus is a database that provides information from different sources about shipping and maritime trade and their movements through time and space. The information provided is stored chronologically and can be accessed through the use of the software called Filemaker. There is also an online website where data can be accessed from, but it is not as complete as the one accessed through Filemaker 12. The way in which data from Navigocorpus looks can be seen in the image below.

Another very interesting project that deals with maritime maps and routes is ORBIS. ORBIS is a project from Stanford University and it represents a Geospatial Network Model of the Roman World. On this model it tries to reconstruct different types of travel in antiquity, including maritime travel. The model that they provide for mapping sea routes takes into account a lot of different factors, including wind conditions, currents and possible waves. The model that they have is made of 900 sea routes that is based on the book written by Pascal Arnaud. An interesting aspect of ORBIS is that they even provide different costs for travelling, depending on which month the journey has taken place in.

A question one might ask is why do we need another project that deals with historical maritime routes. The reason behind our project is that, first of all, the data provided is easily accessible, but not so easily accessible for future development. Data can be accessed through their map and it provides an interface with different parameters and it produces routes. Data can even be exported, but this does not help much.

How can we add different data? How can we use their model and build on top of it? Our project involves two aspects in order to produce maritime routes. One is data collection and one is data visualisation. Since there is no real access to the database ORBIS is using, we cannot make much use of their approach in our project. However, we can use it to compare some of the results that we have.

Further on, we will describe our solution, the challenges that we have faced, how we have solved them and the future research questions that still remain open.

Project Goal

The goal of our project is to propose a data model for maritime routes, by storing information about ships, harbours, dates when ships have travelled to those harbours etc., and to create a visualisation method that others can use for future research or even for educational purposes. This visualisation imposes finding an algorithm to map paths between two harbours.

Our project has two major components:

–          Data collection

–          Data visualisation & Route planning

Data Collection

One of the biggest challenges for such a project is getting data. Since our project is small with limited resources, we have decided to try to find and use an already existing database. The work done by Navigocorpus was exactly what we were looking for. The data that they provide contains information about ships, captains, harbours, shipment, tonnage and so on. This data was perfect for the kind of research we are doing.

Since we have found a source for our data, we have created a database using PostgreSQL and the spatial extension PostGIS. Having our own database structure is much more flexible to work with, we can alter data however we want, we can create, merge, query tables without having to deal with the limitations imposed by other projects (e.g. Navigocorpus allows only basic querying, we cannot build complicated queries).

From our research, we have reached the conclusion that our data model will consist of 5 tables. These tables are: “ships”, “captains”, “ships_captains”, “maritime_routes” and “harbours”.

The “ships” table is used to store information about each ship. For the moment we only record the id and the name of the ship. The “captains” table is similar to the ships table, holding the id and the name of captains. In a very similar way we have designed the tables “harbours” which has an id field and a name.  The “harbours” table is a bit different for the rest because it also contains a filed for geo referencing, the location. The “location” field is a specific data type used in PostGIS; it is a specific data type called “geometry”. This data type allows storing both the latitude and the longitude of a point’s coordinates into a single field. A more important table is the “ships_captains” table. It stores specific combinations of captains and ships. The last and the most important table from our database schema is the “maritime_routes” one. This entity has a composite key, composed of an id and a sequence_id, which is being used to identify all the routes that ships took. The sequence_id field is perticulary useful when we want to track the specific route a ship took while delivering the goods.

The database model can also be viewed in the diagram below:

Database model together with auxiliary tables

Database model together with auxiliary tables

Our database has been populated with several harbours and ships, as it can be seen below.

List of ships and the harbours they have visited

List of ships and the harbours they have visited

"Destin" routes

“Destin” routes

The database also contains another table, which stores the coastline of the map. This is not mentioned in the db model because it is not really part of the actual data, but will be further used to find maritime paths.

Data Visualisation & Route Planning

In this section we will describe the way in which we perform route planning and some results of our experiments using a data visualisation tool (Quantum GIS).

For the route-planning problem we have created two different approaches:

1.      Use buffers to identify potential paths

2.    Use an algorithm, designed and implemented by us, that can identify routes based on the starting harbour, ending harbour and by knowing where the coastline is.

1.      Buffers

In geographic science, buffers are areas of influence around a point which can include specific properties. For our case, this was the first idea of how to map these routes. The goal was to define a minimal and a maximal distance from the shore.

We defined the minimal distance as the range of traditional artillery from the XVIIIth century (maritime and coast) like 36-pounders long guns which had a range of a nautical mile (1’852m).

We defined the maximal distance as the usual “cabotage” (short shipping travels for a same country) distance from the shore as a distance of 20 nautical miles (~40km)

This leads us to the following result:

2.      Route planning algorithm

While trying to identify a way in which we can create maritime paths between harbours, we have notice that not much information about making such routes is available online.

As a result, we have decided to try and build our own algorithm and compare it with our other approach (using buffers) and with some historical based routing algorithms (those available on ORBIS’s website).

In this approach, we have not used a graph based implementation (have a set of predefined edges/paths and then just perform a minimum distance algorithm that will return a minimum path). A graph based implementation involves a huge amount of a priori work to define the possible paths. ORBIS works in a similar way and its edges/paths are based on historical information.  In our approach, we have tried to see if there is an automatic way to generate routes knowing only the position of the harbours (start and finish) and the coastline.

Based on the description provided, below, our algorithm was designed as a function in PostgreSQL. The function takes as input the ids of the harbours and it returns a geometry field that represents the path between the two harbours.

Input: (h1:harbour, h2:harbour, coastline)
Output: (result_path: geometry)
getMaritimePath()
{
create output table path_points
currentPoint = h1
line = st_makeline(currentPoint, h2)
while st_intersects(line, coastline)
{
intersection = st_intersection(line, coastline)
closestPoint = st_closestpoint(currentPoint, intersection);
newline = st_makeline(currentPoint, closestPoint);
newPoint = st_line_interpolate_point(newLine);
// also use a counter as a safety to end the loop
           while st_intersects(st_makeline(currentPoint, newPoint),coastline)
{
// try to place point right
newPoint = st_line_interpolate_point(newline, right);
}
// also use a counter as a safety to end the loop
while st_intersects(st_makeline(currentPoint, newPoint),coastline)
{
// try to place point left
newPoint = st_line_interpolate_point(newline, left);
}
insert(newPoint, path_points);
currentPoint = newPoint;
}
// Create a geometry out of all the intermediary points
return st_makeline(path_points);

The algorithm above is described in a very rough pseudo code. The actual code is provided as an attachment. The basic idea is the following. If there is no direct link between the start and finish, then we find where the intersection is. We get the closest point of the intersection and create a line between this point and start. We try to position the new point at a certain length from the start, either to the left or to the right of the newly created line. If the new point is validated, we set this as the start and start over. The algorithm finished when there is no intersection with the coastline, so it means that the path has been found.

The function described earlier uses some of the following PostGIS specific functions: (the code box below allows scrolling to view the whole content)

st_makeline(geom g1, geom g2) : create a new segment based on 2 points expressed as a geometry
st_intersects(geom g1, geom g2) : returns a Boolean which tells if there is an intersection between the two geometries
st_intersection(geom g1, geom g2) : returns a set of points as a geometry which represents the actual intersection between the two geometries
st_closestpoint(geom g1, geom g2) : returns a point from g2 which is the closest to g1
st_line_interpolate_point           (geom g, m_offset, m_distance) : returns a new point that is at m_offset% from the segment and at m_distance from the segment (g).

Depending on the m_offset and m_distance parameter, the results can very a bit. A small m_distance will lead to more point, thus a smoother path, since the distance covered until the destination (end harbour) is less.

Below, you can see some of the results that we have obtained. Each picture represents a path that is being computer based on the algorithm described previously. Depending on the parameters of the st_interpolate_line function, the path can be smoother or “edgier”.

Saint Pierre

Saint Pierre

Saint Pierre

Saint Pierre

Saint Pierre

Saint Pierre

Destin

Destin

Destin

Destin

Antoinette

Antoinette

The algorithm is far from perfect. As one can see, it does generate quite good paths, but it does not take into account complex factors like wind or maybe depth of the water. The algorithm can be further improved and we believe that it can give much better results if we keep working on it. This demo was just to show that with minimal information available (just harbours and coastline) one can generate possible routes and can reveal the paths of ships.

Conclusion

The goal of this project was to realize some visualization of maritime ships from the XVIIIth century, using the data from the Navigocorpus database, to design and implement a postgresql database with its postgis extension to store the geographical component and to reuse it then to compute and map the maritime paths. We georeferenced the harbours using google earth and with help of some GIS programs and we achieved to draw some pathlines using postgis functions.

We have seen that mapping these paths is very difficult, the main informations which are used in other projects are the streams, the historical sailing points and of course the shortest / quickest paths which were the best arguments to assess whether or not a ship was travelling along a specific path.

There’s a big potential of development in this domain, with this project we tried to go as far as possible.

Special Thanks to

Prof. Frédéric Kaplan (DH Lab EPFL)

Mélanie Fournier (DH Lab EPFL)

Prof. Silvia Marzagalli (Université de Nice et responsable Navigocorpus)

_____________________________________________________________________

SQL source code: https://dl.dropboxusercontent.com/u/70900477/source_code.zip

_____________________________________________________________________

References:

1. Navigocorpus: http://navigocorpus.org/Presentation/Database

2. ORBIS: http://orbis.stanford.edu/#

3. PostGIS documentation: http://postgis.net/documentation/

4. PostgreSQL: http://www.postgresql.org/docs/9.3/static/index.html

5. Quantum GIS Buffers: http://docs.qgis.org/2.0/html/en/docs/gentle_gis_introduction/09_vector_spatial_analysis_buffers.html

6. PostGIS interpolate point along a line

http://gis.stackexchange.com/questions/33478/how-to-offset-point-perpendicular-to-line-direction-in-postgis

http://trac.osgeo.org/postgis/wiki/UsersWikiExamplesInterpolateWithOffset

Advertisement
database modelDigital HumanitiesDigitalizationEuropean NavyFileMakerGISMappingMaritime MapsNavigocorpusPostGISPostgreSQL 2 Comments

Blog Post No. 3

3 May 2013 by Arnaud Poncet-Montanges

The purpose of this blog post is to show the progress on our project. First of all, as we have said in the 2nd blog post our main goal has shifted. The original timetable was designed in a way that the final objective of the project is data visualization.

 Data visualization is still a goal of our project, but while doing some in depth research on maritime routes we have seen that the automatic generation of route plans is not so well developed. As a conclusion, our project will have 2 goals, the second being based on the first on:

 – Create a way in which routes can be automatically generated based on a list of visited harbors while knowing the boundaries of sea/land

 – Visualize the routes generated from the goal above. This is also a way to validate the algorithm proposed in the previous goal.

 Since our second report, we have focused our research towards generating maritime routes. We have already found some software that can create routes, but most of them are manual (e.g. we can use OpenSeaMap.org to manually create a route and the export it in KML format or other common used formats). We have also found some professional navigation devices made by Garmin or Magellan (e.g. Garmin eTrex Venture HC GPS Receiver), but they cannot be used for the purpose that we are interested in.

 These aspects have made us try and design our own “maritime route planner” that will be used for the precise purpose of our project and with possible extensions to any future maritime route planning projects. The biggest problem that we are currently facing is how to avoid “crossing” land. This situation has been described in the previous blog post. In order to solve this problem we have conceived the following algorithm (which will be later improved):

 – If the route between A and B does not intersect the land => everything is OK

 – Else: take the point that intersects the segment between A and B and the land. Create a new point (A’) that lies on the perpendicular of AB, starting from the intersection point, at a certain distance (best distance can be determined from experiments)

 – After the new point has been added, take this as the starting point and redo the algorithm until no more conflicts.

 – The list of resulting points will be the intermediary points for the maritime route.

algorithm

 We have also done some experiments using GIS software. For example determine buffers to illustrate the distances to the shore. This was our first idea to draw semi automatically our pathlines. Here we visualise the distances between 2000m and 3000m to the shore, which corresponds more or less to the range of a 36-pounder long gun of the XVIIIth century (practical 1600m, max 3700m, see “Jean Boudriot et Hubert Berti, L’Artillerie de mer : marine française 1650-1850, Paris, éditions Ancre, 1992” for more details) .

Antoine_Morel-Fatio_pl10

Using the openstreetmap data for the shore, we faced some problems like the non completion of the polylines, and to create multiple buffers, the number of polylines created per multiple users/contributors was so high that it was too expensive to compute buffers for each segment. The solution to this problem was to merge these segments and then draw only one buffer  for the entire polyline.

map

We are actually in the final part of our project, the last steps will be the following:

  • code and run our algorithm,

  • set a form to visualise our data

Leave a comment

Project presentation

19 December 2012 by Arnaud Poncet-Montanges
European NavyMappingPrroject Presentation Leave a comment

State of the art: Mapping seas around the world

28 November 2012 by Arnaud Poncet-Montanges

[Abstract]

Maritine maps can be Nautical Charts, graphical / historical representations or “virtual” via the logbooks. The projection of the maps is quite important because it makes the difference between a simple painting of the globe and a map with spatial references. Actually, the OpenSeaMap project is focusing on mapping the nautical charts around the globe! The Navigocorpus project is focusing on creating a Geographical Information System  (GIS) Database from old archives. The CLIWOC project is focusing on extracting the meteorological information to a geographical database.

Introduction

Our subject focuses on old maritime maps, that’s why we investigate to gather some information about what has already been done and what are the actual projects in this domain.

Firstly, we must differentiate the maritime maps. There are nautical charts containing all relevant information for sailing and the graphical historical representations on large paintings. To that, we can add all the old navigation and commercial journals (logbooks) which linked together can form virtual maps for all the maritime cruises.

Nautical Charts

“A nautical chart, also known as a maritime map, is a representation of a maritime area and its adjacent costal regions.” (Wikipedia, Nautical Chart, 2012) The chart can display a lot of information depending on its scale. You may find information about the depth of the water, details of the coastline, navigational hazards, information about tides, the structure of harbors and so on. Charts are very important for maritime navigation, it is compulsory in a lot of countries for vessels to carry maritime maps. You can see more on Wikipedia, Nautical Chart, 2012 (website)

Nautical Chart
Figure 1 Nautical Chart Example from Porto Rico

Graphical / Historical Representations

Maritime maps are not something very common in day-to-day life. There are few people that actually know how to use a map. This is because a maritime map is quite different from street navigation maps that everyone uses. Old maritime maps are even harder to read since maps weren’t as exact, some maps were even a bit different from the maps that we have today. Below you can see an example of a maritime map of the Mediterranean see. The picture has been taken from the BNF Cartes Marines Exposition. More old maritime maps can be found on the same website.

Europe Atlas Martines
Figure 2 Europe issue de l’Atlas nautique du monde par Joan Martines, Messine, 1583.

It is quite obvious to see that the above map is quite different from a street navigation map, so how can one read such a map? School of sailing, Powerboat or Maptech provide some guides to reading a nautical chart. Traditional maritime maps are made from paper and have a large format. Nowadays, you can also find maritime maps on electronic devices like computers, tablets, smartphones, but the information that they contain is basically the same. If you have a look at a nautical map you might be surprised that there are a lot of symbols and abbreviations that you do not understand. “A typical nautical chart contains a lot of information about the depth of the water, buoys, lights, dangerous areas, hazards, quality of the bottom for anchoring” (School of sailing, 2012, website). The map also contains information about “chart datum, the projection method, tides and currents, and who made the chart” (School of sailing, 2012, website) and, maybe the most important part, the traffic lanes. Once a boat gets near a harbor, it is quite common that the maritime map contains traffic lanes. These lanes are usually for very large ships which can help other sailors to find them when the weather conditions and the visibility are very bad. For a more detailed insight on how to read maritime maps, the links provided earlier are a good start.

Logbooks

“A logbook was originally a book for recording readings from the chip log, and is used to determine the distance a ship traveled within a certain amount of time. The readings of the log have been recorded in equal times to give the distance traveled with respect to a given start position.” (Wikipedia, Logbook, 2012) (website) These books were used to estimate the position of the ship and all specific information about the cruise were stored inside. This source of information was also gathered from the first cartographers to try to map representations of the coasts and of the world. There are also the logs from the commercial harbors which can be an amazing source of information because we find the name of the ships and the date of their arrival and departure. Gathering all these sources of data, this creates the possibility of mapping virtually the traffic of all ships.

The Mercator Projection

“The Mercator projection is a cylindrical map projection presented by the Flemish geographer and cartographer Gerardus Mercator in 1569. It became the standard map projection for nautical purposes because of its ability to represent lines of constant course, known as rhumb lines or loxodromes, as straight segments which conserve the angles with the meridians. While the linear scale is equal in all directions around any point, thus preserving the angles and the shapes of small objects (which makes the projection conformal), the Mercator projection distorts the size and shape of large objects, as the scale increases from the Equator to the poles, where it becomes infinite.” (Wikipedia, Mercator Projection, 2012) (website) This projection is an important discovery because all the maps that were designed before this projection system are not adapted to be fit on the globe for a visual representation. The deformations would be too large and too irregular to be fitted. The map won’t be well projected and will look ugly.

Projection Mercator
Figure 3 Mercator Projection between 82°S and 82°N

Projects : OpenSeaMap

An interesting project about maritime maps is Open Sea Map. “Open Sea Map is a worldwide open source project for creation of a free nautical database.” (Open Sea Map Wiki, 2012) (website) This project is a sub-project of the other well-known mapping project, Open Street Map.  The Open Sea Map will display lighthouses, lateral buoys, cardinal marks and many other navigational aids. Ports are described in much more detail; the map contains all the facilities like port wall, pier, walkways, docks, fueling stations and so on. This map is based on the Open Street Map and is provided as an overlay to the standard navigation map. Below you can see a picture of how the map looks like.

OpenSeaMap
Figure 4 OpenSeaMap

Projects : Navigocorpus

So how does one get from an old maritime map to a modern computer aided representation? Just looking at an old maritime map and trying to convert it to a modern map is a very difficult and tedious job. It requires a lot of experts in history, Geographic Information Systems (GIS), cartographers and maybe others. Navigocorpus is a research project founded by the French National Research Agency. Some of the goals of this project are: creation of a fully searchable on-line database of European maritime shipping in the seventeenth to early nineteenth centuries, creation of a standard for data entry, editing and analysis of historical maritime shipping records. This project has appeared due to research in the following directions: collecting and editing existing databases on historical maritime shipping, a general structure for historical maritime shipping data, GIS database features for maritime maps and training on how to use Navigocorpus for future research. A more detailed description of the project can be found on the Navigocorpus website.

The Navigocorpus database has a structure that was built from scratch; it is designed to accommodate historical maritime data. The main components of the database structure are Points, Cargoes, Taxes and Actions, which are all linked together by means of some unique identifiers. The idea is that a ship movement registration contains information about the route of a ship. The ship movement is connected to a point of observation and if not specified otherwise the registration of the ship will be strongly connected to the point of observation. A very important element in the database is the uniform point definition fields, which are described in detail. The uniform point is used to describe a geographical location.

As we have seen so far, translating old maritime maps to modern nautical charts require a lot of work and a good understanding of the map itself. Luckily, a few existing projects have already started doing research in this direction so one has a starting point from which future research can begin.

Projects : CLIWOC

An amazing project is the CLIWOC project, which uses the meteorological data extracted from the logbooks to create an International Maritime Meteorological Database. These logbooks are giving such elements as “wind direction, wind force, precipitation, fog, ice cover and state of sea and sky” (CLIWOK, 2012, website)

CLIWOC Results
Figure 5 Visualisations from the CLIWOK results ( from left to the right: single cruise informations about windforce and direction, multiple data becoming vector fields for wind, average precipitations estimations)

“The Climatological database for the world’s oceans (CLIWOC) was a research project to convert ships’ logbooks into a computerised database. It was funded by the European Union, and the bulk of the work was done between 2001 and 2003. The database draws on British, Dutch, French and Spanish ships’ logbook records for the immediate pre-instrumental period, 1750 to 1850. (Wikipedia, Cliwok, 2012)”(website)

CLIWOClogbooksMappingMaritime Mapsmercator projectionNautical ChartsNavigocorpusOpenSeaMap Leave a comment

[Site Description]

12 November 2012 by Arnaud Poncet-Montanges
HMS Bounty

Le HMS Bounty en 2003 au large de Chicago
Image: AFP

This blog has for purpose to show the progression and the steps on our dh101 Project 19 : Google maps for old maritime maps.

This project has 3 aims or steps:

1. Digitalization
2. Transcription
3. Modelling

The Digitalization phase has been already processed by http://expositions.bnf.fr/marine/index.htm

Digital HumanitiesDigitalizationMappingModellingNavyTranscription Leave a comment

Post navigation

Recent Posts

  • Final Report
  • Blog Post No. 3
  • Progress Report No. 2
  • Progress Report No. 1
  • Project presentation

Archives

  • May 2013
  • April 2013
  • March 2013
  • December 2012
  • November 2012

Categories

  • Non classé

Meta

  • Register
  • Log in
  • Entries feed
  • Comments feed
  • WordPress.com
Create a free website or blog at WordPress.com.
Privacy & Cookies: This site uses cookies. By continuing to use this website, you agree to their use.
To find out more, including how to control cookies, see here: Cookie Policy
  • Follow Following
    • Mapping European Navy
    • Already have a WordPress.com account? Log in now.
    • Mapping European Navy
    • Customize
    • Follow Following
    • Sign up
    • Log in
    • Report this content
    • View site in Reader
    • Manage subscriptions
    • Collapse this bar
 

Loading Comments...