Import-Export

Import-Export#

The Polaris Supply (network) file is built on open data formats, so in general there is no need to export the data to other formats in order to read it in most available software.

However, there are two genuine needs for importing/exporting models from/to simpler (text-based) formats:

  1. Creating new Polaris models from supply data available from

  2. Converting network data to a format adequate for version-control

Text-based files#

A custom text-based format for Polaris was created to represent complete Polaris supply files on a set of text-based files, where each data table is represented by two separate files:

  1. A *.schema file with the table structure, including data type and SQL constraints such as whether which field should be not null, have a default value or be a unique identifier

  2. A *.csv file with the contents of the table, where geometry fields are converted to Well-Known-Text (WKT)

The last file part of the text-based representation of Polaris supply files is the list of all SRIDs (projections) for each of the WKT geometries included in each one of the data tables.

This general format allows for the export of supply files with an arbitrary set of data tables, as there is no pre-defined structure for the text-based data.

The capability of importing supply files in this format is also built into polaris-studio, and extends the Polaris datamodel to include all additional tables exported to the format and any extra fields on existing tables.

GMNS#

The General Modeling Network Specification (GMNS) is proposed to become the industry standard for network data exchange, being the network-equivalent to what GTFS is for transit data.

Importer#

The GMNS importer in polaris-studio is currently capable of importing data for the following GMNS tables:

Although the ideal scenario is one where we the data imported remains unchanged during the import process, a few changes are required to adapt the GMNS data into the Polaris model structure.

Besides the table-specific changes required during the import process, all coordinates for geometry fields are rounded to 6 digits, making sure that they snap to each other when necessary. Just for reference, the rounding process happens after conversion to the model’s project coordinate system in meters, and therefore the precision is equal to 1 micron, and therefore immaterial.

link#

  • Links are converted to bi-directional links when they exist, as that’s the default for Polaris.

  • The minimum number of lanes for an existing link direction is set to 1. This is necessary because OSM imports to GMNS may bring the number of lanes directly from OSM, which is often missing.

  • allowed_uses is imported into use.

  • facility_type is imported into type, and all non-existing types are added to the Link_Type table with a rank of 10 and permission for pockets, and a default pocket length of 50m. The use_codes for each added link type are all of those allowed for links in the input data.

  • link length is automatically re-calculated by Polaris based on the link’s geometry

  • node_a/node_b are re-calculated if there are any discrepancies between node and link geometries and the information for node_a/node_b

location#

  • Importing locations without coordinates (based on linear referencing only) is not supported

  • loc_type is imported as land_use types. Absence of this field results in types being set to “ALL”

  • link and zone for all locations are re-computed, as tight geo-correspondence for those fields is critical for Polaris

  • Because of the above, absence of data on zones or links table will result in the location import to be skipped

Open-Street Maps#

Parsing OSM data is a tedious task already tackled by numerous Python packages, such as OSMNX and AequilibraE. There is one package, however, dedicated to the conversion of OSM data into GMNS networks. With this package, OSM2GMNS, and the GMNS importer fully implemented, one can create Polaris networks from OSM through the intermediary step of creating the network in the GMNS format.

Exporter#

At this point there is no plan to implement a GMNS exporter for the Polaris Supply file.

API#

import_export.ImportExport

This class allows for importing and exporting to the GMNS format.