collapse

M4OPS

For our prototype mapping system try the Demo at mapping4ops.org/M4OPS.

You can also view a demo and discussion about M4OPS on YouTube.

* Forum Staff

PeterC admin PeterC
Administrator
admin webmasterOPS
Administrator

Show Posts

You can view here all posts made by this member. Note that you can only see posts made in areas to which you currently have access.


Topics - PeterC

Pages: [1] 2 3 ... 30
1
Other Mapping things / Google Arts and Culture - maps
« on: 17 May 2018, 10:28:05 »
Google Arts and Culture curates many map collections, as well as many other artifacts in museums etc across the world.

2
Mapping Technologies we can probably ignore / OpenAddresses
« on: 17 May 2018, 06:36:07 »
OpenAddresses.io is "The free and open global address collection." Anyone can Parse & import into a database, put on a map, or use for geocoding.

Some countries (eg France) are much better covered than others (eg UK).

3
Current Examples / Mapping a cemetery
« on: 17 May 2018, 06:27:30 »
New GIS mapping project is chronicling genealogical history through Waxahachie (Texas) Cemetery.

“This is just for the benefit of our citizens so they can better search our cemetery. Genealogy is becoming a very popular hobby and a very popular thing for families to do,” Smith explained. “It will help in that hobby and help our citizens better learn our cemetery because our cemetery is large and kind of confusing if you are not familiar with how it is laid out.”

4
Other Mapping things / Morphology
« on: 17 May 2018, 06:22:03 »
Morphology is "an exploratory cartographic tool based on the Mapzen platform that eschews common communicative elements like color and symbols. Instead, it seeks to show the world, and all its constituent parts, as a series of carefully chosen lines."

5
This post from Rory Biggadike of webmapsolutions says "We are today seeing an explosion in the number and type of mapping platforms. In this post we will discuss the differences between these mapping platforms, and why you might choose one over the other."

6
Other Mapping things / True Size Maps
« on: 17 May 2018, 06:11:40 »
The “True Size” Maps Shows You the Real Size of Every Country (and Will Change Your Mental Picture of the World).

The True Size is "a website that lets you compare the size of any nation or US state to other land masses, by allowing you to move them around to anywhere else on the map."

7
Commercial Mapping Companies / Foursquare
« on: 17 May 2018, 06:04:57 »
Foursquare "is a technology company that uses location intelligence to build meaningful consumer experiences and business solutions.

For consumers, we believe the world is full of amazing experiences. We make two apps to help guide you to them:
  • The Foursquare City Guide app helps you discover new places, with recommendations from a community you trust. Find a better experience, anywhere in the world.
  • For people who have a lust for life, Foursquare Swarm is a lifelog that keeps track of every place you go. Although life moves fast and is full of fleeting moments, Swarm is always with you motivating you to take a richer and more interesting journey.
For commerce our location intelligence technology helps brands to locate, message and measure their own consumers."

8
Other Mapping things / All the Buildings in Manhattan
« on: 16 May 2018, 06:56:38 »
(From curbed New York) "There are more than 1 million buildings in New York City, and many of those are clustered on the island of Manhattan, spanning architectural styles, hundreds of years, and all manner of types. And now, a nifty new tool, called All the Buildings in Manhattan, takes information about those myriad structures and puts it in one colorful data visualization."

The tool was created by Taylor Baldwin, a software engineer at BuzzFeed (“One thing I learned is that there is an unbelievable amount of data the city makes available.”)

(Note that I had difficulty getting it to work.)

9
Other Mapping things / Map Books of 2018
« on: 16 May 2018, 06:46:14 »
The Map Room (Jonathan Crowe blogging about maps since 2003) has many interesting things about maps - for example this list of Map Books which have been published or are scheduled to be published in 2018.

10
Other Mapping things / Viz Palette
« on: 16 May 2018, 06:37:10 »
Viz Palette is a colour picker tool to help with selecting a palette for data visualizations, including maps.

11
Commercial Mapping Companies / Latitude-Cartagena
« on: 16 May 2018, 06:32:58 »
Latitude-Cartagena, a cooperative society based in Lyon, France, is "much more than a cartographic communication agency. It's a living network of cartographers, consultants, designers and developers working together to improve your everyday life ... without you noticing."

12
An example of one of Peter Watson's Valuation transcripts for Herne Bay in Word document format is attached to this post.  It comprises separate tables, one for each property in the 1910 Valuation, interspersed by some images to help remind about the property. For convenience each document contains a limited number of properties (eg 100), and this example shows just a few.

Each table has the following data for each property:
  • ValuationID (when preceded by V this is used as featureid)
  • Situation
  • Description (which for technical reasons internally we call Designation)
  • Gross Value (£ and optionally shillings and pence)
  • Occupier
  • Former Sale - the date the property was last sold, and hopefully the amount of sale, + additional cost (eg if it was bought as land, then a house built in it)
    • [Note this is displayed/stored after Owner in M4OPS]
  • Owner
  • Particulars

Broadly speaking the stages of the process are:
  • Manipulate the Word Documents to prepare a single table of data
  • Convert this table into a list of features for M4OPS
  • Add the All Features layer
  • Upload and compile
  • Georeference the Features

Manipulate the Word Documents to prepare a single table of data:
  • In Word, create a new landscape document
  • Empty the document, headers and footers of all content
  • For each transcript document: Select all, copy and paste onto the end of this landscape document (selecting the default - destination theme)
  • To remove header lines: Select all of style Title and delete
  • Use Replace All various times:
    • Images: ^g to null
    • Manual line breaks: ^l to space
    • Tabs: ^t to space
    • Semicolons: ; to #semicolon#
    • Description or description to #Dscn# or #dscn# (technical problem!)
    • Other special characters/words may need converting:
      • (These are all defined in the CSV2HTMLConvsArray in M4OPS.json)
      • non-breaking space to #nbsp#
      • less than or greater than to #lt# or #gt#
      • ampersand to #amp#
      • single or double quotation mark to #apos# or #quot#
      • cent, pound, yen, euro to #cent#, #pound#, #yen#, #euro#
      • copyright or registered trademark to #copy# or #reg#
    • [(unnecessary?)Extra paras: ^p^p to ^p – repeat until 0 (any left delete manually)]
    • Single paras: ^p to null
  • Select all, cut and paste as text only
  • Select all and insert table (Convert: paragraphs, 5 columns, Autofit contents)
  • Check for changes in pattern, and for each: Undo, adjust, redo
    • [eg in the attached document there is an issue after id 63]
  • Delete the blank column (must be narrow if it is empty)
  • Convert to text (separate text with tabs)
  • Select all and insert table (Convert: tabs, 8 columns, Autofit contents) – if not 8 then undo, resolve issue, and redo
  • Save the document

Convert this table into a list of features for M4OPS:
  • Insert 4 (blank) columns to the right of the table (these fields will be filled by geometries)
  • (At this point, if relevant, transfer any geometry cells from an existing features.csv file)
  • Check there are no semicolons in the document
  • Convert to text (separate text with semicolons)
  • Replace ;space by ;
  • Replace ^p by ^pV (thereby preceding each id by the letter V)
    • ensure the first line has a V at the start
    • remove any last line(s) with just a V on them
  • Select all, copy and paste into a blank Features.csv text file
  • Copy the following string and paste it as a new line at the beginning of this file
    • featureid;Situation;Designation;GrossValue;Occupier;FormerSale;Owner;Particulars;Lon(X);Lat(Y);GeomType;GeomCoords
  • Under this header line enter a blank line and a line starting Comment, with the date etc
  • Save the file as Features.csv

See Compiling Features for the next stage.

13
M4OPS - for those responsible for an OPS / Compiling Features
« on: 12 April 2018, 12:37:27 »
Once a list of un-georeferenced features has been prepared in a Features.csv file (eg from The Journal or transcripts), then the next stages of the process are:
  • Add the All Features layer
  • Upload and compile
  • Georeference the Features
(If not already done) Add the All Features layer to the LayerDefs_Feature.csv file:
  • Whereas other processes may be done more than once, this one is done just once
  • Where the fieldnames are
    layertype;category;title;csvname;candownload;url;attribution;mapkey;minx;miny;maxx;maxy;shorttext;description;fl_Ncols;flhead_col1;layerdescription
  • the All Features layer entry is
    Vector;Local;All Features;AllFeatures;TRUE;AllFeatures.geojson;#PDC#;;;;;;;;1;;These are all the features as we have recorded them.
Upload and compile:
  • In M4OPS for your OPS (first Dev then, when tested, Production):
  • Click on the Upload Tab
  • Click on Choose Files, Browse to the folder where your csv files are located
  • If you have updated the LayerDefs_Feature.csv file:
    • Select the LayerDefs_Feature.csv file and click on Open
    • Click on Upload, and (if necessary) provide the password
    • When the Uploaded OK message appears click on OK
  • As you have updated the Features.csv file:
    • Select the Features.csv file and click on Open
    • Click on Upload, and (if necessary) provide the password
    • When the Uploaded OK message appears click on OK
  • Click on Compile
  • When the (detailed) compilation results message appears click on OK or Cancel
  • (As after every Compile) Clear your browser cache, and reload M4OPS

See Georeferencing Features for the next stage.

14
To process a new version of the SOPS' Studies spreadsheet:
  • Go to https://drive.google.com/drive/shared-with-me and right-click/download the latest StudiesYYYYMMDD.xlsx file to the Mapping\Software\M4OPS folder
  • Open the downloaded file in Excel, and enable editing
  • File/Save As Text (Tab delimited) (*.txt) for the Active Sheet only
  • Exit Excel
  • Rename the file as StudiesYYYYMMDD.xlsx - Active.tsv (tsv is our name for Tab Separated Values)
  • Open this file in an editor and
    • delete all the empty rows at the end of the file
    • check there are no semi-colons in the file (and if there are change them)
    • change all tabs to semi-colons
    • delete all the excess semi-colons at the end of lines
    • Encoding - click on Convert to UTF-8, save and reload to check it is still encoded as UTF-8 (as json needs this)
  • FTP the .tsv to /ShowMapsDev/
  • On /ShowMapsDev/ rename the old Studies.csv as StudiesOLD.csv (having deleted any previous OLD versions)
  • On /ShowMapsDev/ rename the .tsv file to Studies.csv
  • Open mapping4ops.org/ShowMapsDev/CompileStudies.php?BRs&Test to precompile the Studies.json file
  • Make sure that the only OPS listed as "Location not set from Studies.csv" are those we know of (eg BAL, BRI, POR, USG, EDI, LGIWG), or for which we have not yet compiled the OPS.json file
  • (If there are problems later you may want to check the resulting Studies.json file)
  • On our local M4OPS rename the old Studies.json as StudiesOLD.json (having deleted any previous OLD versions)
  • FTP Studies.json from /ShowMapsDev/ to our local M4OPS
  • Test M4OPS, and when happy all is OK, FTP Studies.json from our local M4OPS to /ShowMaps/ (ie the production M4OPS system)
  • (Note that there is no need to repeat the precompile process on the production system, as all that M4OPS needs is the Studies.json file)
  • Test the production M4OPS

15
Feature Layer Technicalities / What is AllFeatures?
« on: 5 March 2018, 07:34:40 »
This is a technical note about the use of AllFeatures.geojson and Features.csv within M4OPS.

The Features.csv file contains detail of Features, each of which has a featureid. These can be referenced just by their featureid within other feature csv files (eg Pubs.csv).

If there is an AllFeatures.geojson line within the LayerDefs_Feature.csv file that specifies AllFeatures as the csvname, then FLG.php will compile all the features it can find (that are georeferenced and have a featureid) into an AllFeatures.geojson file. If no such features are found then one point at the centre of the box defined in the OPS, with featureid=F_OPS, is generated - and this should be deleted when there are other points.

To compile AllFeatures.geojson FLG.php will look in Features.csv as well as all the other feature csv files it is told about. Any features in Features.csv that have no geometries are called 'Features still to be Georeferenced'.

AllFeatures.geojson can be opened within M4OPS either as:
  • a Feature Layer - where it cannot be changed
  • a Modifiable Feature Layer - where it can be changed
As an MFL AllFeatures.geojson can be altered in the usual ways using Alter/Move, but it can also be added to by georeferencing 'Features still to be Georeferenced'.  Note that AllFeatures.geojson is stored in the main OPS folder, not in SavedFiles where the normal MFL geojsons are stored.

When the MFL AllFeatures is saved then besides being saved as AllFeatures.geojson a new version of Features.csv is saved - comprising both georeferenced and non-georeferenced features. This can be downloaded onto the user's computer if wanted.

Pages: [1] 2 3 ... 30
mapping4ops.org is a Society for One-Place Studies project supported by Grassroots Giving from Skipton Building Society
Glossary | BBCodes | Feedback