collapse

M4OPS

For our prototype mapping system try the Demo at mapping4ops.org/M4OPS.

You can also see an overview of M4OPS on YouTube.

* Forum Staff

PeterC admin PeterC
Administrator
admin webmasterOPS
Administrator

Author Topic: The input CSV files-1  (Read 832 times)

PeterC

  • Administrator
  • Member
  • *****
  • Posts: 480
  • Passionate about my one-place study
    • View Profile
    • Holywell-cum-Needingworth History
The input CSV files-1
« on: 15 January 2017, 11:59:19 »
Note that this is mostly for technical information.
It is the first part of the document about the input csv files.

For each OPS in M4OPS

Input files
All files are in csv format using semi-colon ";", rather than comma, as the separator. No fields can have semi-colons within them, but they can have commas. (On Peter's PC all csv files are within eg \OPS\HcN Holywell-cum-Needingworth\FLG\.)

Every file has field names in its first row, and these field names must be as specified (in the correct case). No quotes are needed for field names or values. Any null or blank values should be left null, ie represented as ;;.

Any blank lines, any starting with the word “Comment”, and any starting with the first field name (thereby allowing a repeat of the first row of field names), are ignored. If you use Excel for accuracy then ensure that the delimiter is set to ";".

These csv files are used to generate geojson and json files, which are what M4OPS to use directly as its input.

The required files are:
  • OPS.csv - a single row with the parameters of the OPS
    • It must have the following fields
      • OPSCode - short code (this is often 3 upper case letters)
      • OPSName - the name of the OPS (usually mixed case)
      • Area – of the OPS, eg ENG or SCO (if otherwise then Peter will let you know what to use)
      • Lon – longitude of the centre of the OPS (in decimal degrees - known as EPSG:4326)
      • Lat – latitude of the centre of the OPS (ditto)
      • password - requires to be entered before any updating
    • It can have the following fields
      • OPSDescription – appears when you click on the heading, can include abbreviations (default is “No description available”)
      • ContactEmail – ditto (no default), and where the Contact button email goes to
      • Website – ditto (no default)
      • Zoom – starting zoom (default is 16)
      • Rotation – starting rotation in radians clockwise, default is 0 (ie North at the top)
      • nFeatureLayers – number of feature layers to allow/show (default 0, maximum 3)
      • minYearBound - minimum year to allow (default 700AD)
      • maxYearBound - maximum year to allow (default the current year)
      • minYear -  the starting lower value for the range of the lower time slider (default minYearBound)
      • maxYear -  the starting upper value for the range of the lower time slider (default maxYearBound)
      • IconsFolder – the icons folder to use if specific to this OPS (M4OPS default is icons/default-icons/)
      • minx (Not implemented yet - becomes extent, with miny, maxx, maxy - must all be there or all absent)
The optional files are:
  • LayerDefs_Feature.csv - a list of all feature (vector) layers specific to this OPS
    • It must have the following fields
      • layertype - always Vector
      • category - used to group the layers, the basis of the category drop-downs
      • title - the name of the layer (and is used when specifying a parameter for preloading a layer)
      • url - specification of where to get the layer from:
        • a file name for a GeoJSON file
      • attribution – to be shown when the layer is displayed, can be an abbreviation
    • It can have the following fields
      • csvname - the name of the csv file (without the .csv) if it is to be included in any compiling
      • candownload - either TRUE if the layer can be downloaded eg for printing, or null if not (default is null)
      • mapkey – used when the key button is clicked, can be either:
        • the url of a web page (with or without http:), or
        • the file name for a jpg file in the OPS's Images folder, or
        • the file name for a jpg file in M4OPS's images folder
        • (Note that either jpg filename can be followed immediately by "(WithText)" and then text to show with the image)
      • minx (Not implemented yet - becomes extent, with miny, maxx, maxy - must all be there or all absent)
      • shorttext - defines the name of the field in this feature layer that is the equivalent to "shorttext", eg when hovering over features (no default)
      • description - defines the name of the field in this feature layer that is the equivalent to "description", eg when displaying in the panel (no default)
      • fl_Ncols – the number of columns in the feature list (default is 1)
      • flhead_col1/2/3 – the heading for the first/second/third column (default is blank)
      • fl_col1/2/3 - the definition of the first/second/third column (default is a field called fl_col1/2/3 or, if undefined, shorttext)
      • textforsort – used for sorting the feature list, if undefined use fl_col1
      • layerdescription - html text shown when you click on the Description button (?)
      • Note that some of these fields can include abbreviations, equivalencies, constants, concatenation, defaults [see below]
      • Note also that any placeholder text (identical to the fieldname) is ignored, as is any "fieldname:" construction
  • LayerDefs_Other.csv - a list of all other layers specific to this OPS, including raster Tiles
    • (Note that the general DefinedLayerDefs_Other.csv file has exactly the same specification, except that Defined layertype is not allowed)
    • It must have the following fields
      • layertype - one of
        • Tile - either from a map server, or as prepared by maptiler and served by TileServer
        • VectorTile - from a map server (not implemented yet)
        • WMTS - from a Web Map Tile Service/url]
        • WMS - from a Web Map Service/url]
        • Group (made up of any number of other layers)
        • Defined (picked up from DefinedLayerDefs_Other.csv)
      • category - used to group the layers, the basis of the category drop-downs
      • title - the name of the layer
        • (this is used when referring to a layer eg for specifying a parameter for preloading, or picking up a (pre-)Defined layer)
      • Note that Defined layertypes need only these three fields, and not even nulls and separators for the other fields
    • It must also have the following fields (unless it is a Defined layertype)
      • url - specification of where to get the layer from, either:
        • a full url (if served by a map server), or
        • a folder name (if served by TileServer), or
        • (for WMTS layertypes) a full url of a Capabilities file, or
        • (for WMS layertypes) a full url of the server, or
        • (for Group layertypes) a comma separated list of layer titles that are to be included in the group, eg Lidar Coldhams Field,Lidar Holywell Centre 1,Lidar Holywell Centre 2
        • note that if preferred this can be provided as an array of quoted layer titles, eg ["Lidar Coldhams Field","Lidar Holywell Centre 1","Lidar Holywell Centre 2"]
      • attribution – to be shown when the layer is displayed, can include abbreviations (default null)
      • folder - either:
        • (for Tile layertypes) where the url folder is within the TileServer folder (default is null), or
        • (for WMTS layertypes) the WMTS layer name
        • (for WMS layertypes) the WMS layer name
      • storageName, where the files are - either:
        • AWSS3 - use m4opsprod bucket on AWS S3
        • AWSS3DEV - use m4opsdev bucket on AWS S3 (not used yet)
        • ShowMaps - use folders in ../ShowMaps/
        • ShowMapsDev - use folders in ../ShowMapsDev/ [default if null]
    • It can have the following fields
      • donotshow - TRUE if the layer should NOT be listed as itself in the drop-down (usually because the layer is part of a layer Group) (default is false -  it will be listed)
      • candownload - TRUE if the layer can be downloaded eg for printing (default is false)
      • mapkey – used when the key button is clicked, can be either:
        • the url of a web page (with or without http:), or
        • the file name for a jpg file in the OPS's Images folder, or
        • the file name for a jpg file in M4OPS's images folder
        • (Note that either jpg filename can be followed immediately by "(WithText)" and then text to show with the image)
      • projection – not used yet (EPSG:3857 or EPSG:4326 at present)
      • minZoom – the minimum zoom level that tiles are available for (default is 0)
      • maxZoom – the maximum zoom level that tiles are available for (default is 18)
      • tilePixelRatio – usually 1 (not implemented yet)
      • origin - (not implemented yet) the tiling origin (topleft = OpenGIS, M4OPS default for tileserver, and MapTiler default)
        • (bottomleft was tried, but tileserver can only do top left)
      • minx (with miny, maxx, maxy this becomes extent – must all be there or all absent)
      • date - the year of the map, needed for ordering series (THISYEAR can be used for the current year, ie if the map is known to be kept uptodate)
      • shiftEastM and shiftNorthM - the metres the layer should be shifted east and north (E/N) (usually only needed for very fine adjustments of rasters georeferenced by elsewhere)
        • one (and only one) of the layers is designated as the 'base layer', and this is indicated by a 'B' in front of the number for its shiftEastM
        • the base layer itself can be given shifts (E/N) relative to its given georeferenced location
        • each other layer can be given shifts (E/N) relative to the base layer [thus in effect this shift is added to the shift for the base layer]
        • (Note that georeferencing of other rasters should be done if possible against the base layer, and there should then be no need to shift the resulting map)
        • if you want to see what the layers would be like NOT shifted east or north in M4OPS then use the parameter &NoShift
        • a convenient way of determining the E/N shift for a raster is to:
          • set up M4OPS with the 'base layer' as the Base and the raster of interest as Overlay
          • create a new MFL
          • add a single one-segment line joining identical points on the two layers
            • with the start point on  the 'base layer'
            • the end point on the raster of interest
            • (remember you can move slider to show each of the two layers better)
          • set Shorttext to be anything, Save, Close
          • click on the E/N button and copy (Ctl/C) the result, click OK
          • close the MFL, click OK to lose it
          • when editing LayerDefs_Other overwrite the last 0;0 (shiftEastM;shiftNorthM) by pasting what you have just copied (the East/North shifts)
            • (Note that, depending on the direction you draw the line, you may need to change the signs of both numbers)
            • (You may also want to take an average of 2-4 such E/N measures to get the best fit over the whole OPS)
          • save LayerDefs_Other, Upload it and Compile
          • clear your browser cache, and refload M4OPS
      • layerdescription - html text shown when you click on the Description button (?), can include abbreviations
For more on specifying WMTS and WMS, see:abbreviations
equivalencies
constants
concatenation
defaults


For more input csv files see The input CSV files-2 and The input CSV files-3
« Last Edit: 9 July 2018, 16:02:01 by PeterC »

 

mapping4ops.org is a Society for One-Place Studies project supported by Grassroots Giving from Skipton Building Society
Glossary | BBCodes | Feedback