Provides a reference for NT Government (NTG) spatial data custodians to make their data available via the NTG Spatial Data Infrastructure (SDI), the framework for accessing spatial information across the NT government.
The aim of this project is to improve the visibility, accessibility and usability of Spatial Datasets made available by the Northern Territory Government (NTG).
SCOPE OF PROJECT:
- To assist Data Custodians improve their understanding of the Spatial Data Infrastructure and Data Access Methods existing within the NTG.
- To assist Data Custodians in making the necessary decisions to integrate their dataset with the available Spatial Data Infrastructure.
- To publish Spatial Dataset’s Accessibility Process on a public forum so as to be consistent with transparency requirements of NTG.
- To assist Data Custodians with the necessary preparation of the Spatial Dataset for its integration within the Spatial Data Infrastructure.
- To include all known Data Custodians within DPI and NRETA in the process.
- To assist/include interested Data Custodians from other government departments who express interest in the process.
- To log a project request for the development of a Custom Viewing Application.
OUTSIDE PROJECT SCOPE:
- The content and quality of the spatial data remains the responsibility of the Data Custodians. Such things will only be considered within this project as far as is necessary to make the dataset available within the Infrastructure.
- Data correctness issues after the dataset is loaded into a dissemination database, are out of the scope of this project. These errors should be mentioned by the custodian in the metadata and fixed by the custodian as time allows.
- Raster data will not be considered within this project.
- The development of Custom Viewing Applications is not within the scope of this project.
An electronic collection of related data that includes a geographical aspect.
Government group or personnel that have responsibility for a particular spatial dataset.
Spatial Data Infrastructure:
Includes: Dissemination databases; Authentication processes; Metadata storage area; and the Client Applications being developed by NT Land Information System (NTLIS) group.
Department of Planning and Infrastructure
Department of Natural Resources, Environment and the Arts
A database that is considered to be the official or "authorative" source for users (people and applications) to access the datasets held within.
Metadata is information about data, describing the "what", "when", "who" and "how" of data. Spatial metadata incorporates additional information about the "where" component of data.
Custom Viewing Application
An application that increases the accessibility of spatial datasets by applying available technology in a new manner.
Previous initiatives to capture and make available standards-based metadata have been successful, with some 400 records currently available on the Northern Territory Spatial Data Directory (NTSDD). A natural progression after improving data discoverability is to improve accessibility of these exposed datasets. With GIS and non-GIS applications increasingly able to connect directly to spatial data via network protocols, there is an increasing ability for data providers to reduce the time and effort required to connect to, and use, the spatial data once it has been discovered via existing mechanisms.
This document identifies the existing components of the Northern Territory Land Information System (NTLIS) Spatial Data Infrastructure (SDI) and provides a framework for how they can be best applied to datasets by data custodians. The document is intended to be used as a reference by data custodians, to provide a consistent, clear methodology for making spatial data available to end users inside and outside of the NTG.
In effect, it forms a checklist describing the steps required in the “life cycle” of a dataset. During the publishing phase, this checklist will enable the status of an individual dataset to be easily identified.
The document defines a procedure for addressing the following elements:
- Data model design and documentation
- Data hosting and update regime
- Data usage guidelines including thematics and scale
- Metadata creation and discovery
- Data access interfaces
For each dataset considered complete under the checklist, custodians can be confident that all appropriate services available through the NTG SDI can be applied with minimal user effort in a sustainable manner. This is not to say that all services will be applied, the custodian will always remain in control of who accesses the data and how, it simply outlines the options available and makes recommendations where appropriate.
Once a dataset is compete under the checklist, it can be said to be NTG SDI ‘certified’, in that it is compatible and accessible via the SDI.
The components of the NTG SDI including the relationship of data custodians, dissemination databases, access mechanisms and end users are depicted in Figure 1: The Northern Territory Government Spatial Data Infrastructure (NTG SDI).
Figure 1: The Northern Territory Government Spatial Data Infrastructure (NTG SDI)
Figure 1 suggests that information flows from the bottom up, and that all ‘ideal use’ type end user access is through established interfaces, reducing the number of interfaces that users need to have experience using to connect to a greater range of spatial datasets.
This model aims to maximise the value delivered by the NTG SDI to data custodians and end users.
STEPS FOR NTG SDI COMPLIANCE
Items on the checklist (and the SDI !) are of course open to comment and modification by the data custodians through discussions with NTLIS.
If custodians need help either interpreting or implementing these steps NTLIS will be glad to offer assistance where possible.
1 The data exists in digital, spatial form
Given that a priority objective of the NTG SDI Accessibility Procedure is to make data easier to use by end users, there is high importance in making the data model and representation as clear and logical as possible. Achieving this step in a sustainable way such that the data to be disseminated is current may require a redesign of the data management application and procedures.
The data needs to be available in a traditional GIS form, i.e. features associated with geometry and attributes in a tabular sense. For those datasets comprising of more than one ‘table’, the data model needs to be defined and documented. In these cases, a ‘summary table’ pulling together key fields to produce a flat table may be useful.
The custodian needs to be satisfied with the choice of attributes and spatial representation that is to be disseminated to different classes of users, (e.g. department division, government, general public etc) and structure the tables exposed accordingly.
2 The data is exclusively available via a single point of truth
Key to the use of spatial data through the NTG SDI is the concept that each dataset is available through a ‘single point of truth’. This phrase defines an interface as being the authorative source for the data in terms of currency, accuracy and completeness as detailed in the dataset metadata. As with any authorative source, it is important that all access be through this interface to allow for authentication, metering and to ensure that the metadata description matches the dataset, providing for more accurate end usage of the data. This extends not just to human users but also to applications accessing the data, such as mapping engines or other web services. There is only one ‘single’ point of truth. For performance or other internal reasons certain groups may wish to download information from the authorative source and share it internally from a local point, however this practice should not extend beyond the immediate work area to avoid ‘stale data’ or modifications being passed along so as to render the metadata document inaccurate.
Figure 1 provides a representation of how data is accessed via the NTG SDI. It is up to the custodian whether data is maintained internally and published periodically to the dissemination ‘point of truth’, or alternatively if the dissemination point of truth is a read-only view of the source data.
data model should be described in here somwhere establish layer in dissemination database schema
For example, the cadastre dataset is maintained in the Digital Cadastre Database (DCDB) environment using a legacy application rich in business rules and based on older (CAD) technology. To provide external users with current cadastre information, DPI provide a read-only view of the data, refreshed nightly on the Oracle-based Spatial Database 1 (SDB1). All use of the cadastre other than maintenance operations are sourced from SDB1 to ensure currency and compliance to the metadata documentation. The mechanism for update using this model is very important and must be documented in the metadata. In contrast, the Administrative Boundaries dataset is maintained by DPI stored directly on SDB1 in a special authenticated schema, with a read-only view forming the dissemination point of truth. Either approach is acceptable, it is up to the custodian to decide which is more appropriate for each dataset.
Both DPI and NRETA currently run databases nominated as ‘dissemination databases’, i.e. databases with publicly available read-only logins through which a range of datasets are available. There is nothing to stop custodians establishing their own dissemination databases, if this is to occur maximum value will be achieved if they are available directly via a TCP/IP connection; preferably Oracle Spatial, PostGIS and to a lesser degree ESRI SDE. There is benefit however from grouping databases that share information on the same physical hardware to allow for better integration and normalisation.
Note that users need not be given direct access to the database itself. If the custodian wishes the data can be added to the database for the benefit of the NTG SDI applications, but no users can connect to the data itself.
Note that reliance on windows file sharing mechanisms and proprietary file-based solutions such as Microsoft Access or Excel is strongly discouraged for use in the NTG SDI for network performance and application accessibility reasons.
3 The data has a metadata document published in the NTSDD.
Spatial data should always be distributed such that the metadata document is either directly attached or is easily accessible. This can be either as an attached file or a hyperlink directly to the relevant metadata record in the NTSDD,
The custodian is responsible for maintaining the associated metadata through the use of the Metadata Management Tool (MMT), access to which can be provided by contacting NTLIS.
Custodians with responsibility for multiple datasets should have ready access to the MMT; those with only one or two records can provide their metadata in XML format, according to the ANZLIC Metadata Guidelines, to NTLIS for entering. An online metadata entry tool is currently being developed by NTLIS which will replace the MMT.
The metadata information required to be captured has changed as part of this NTG SDI accessibility project. In order to both reduce the amount of prior knowledge and experience the user is expected to have with the dataset; and the amount of contact they will need to have with the data custodian, more detailed information is now required.
3.1 The metadata record includes a data dictionary and thematic hints
The metadata record must include a data dictionary that explains attribute column names and the respective attribute code values to assist with the interpretation of the data. To date, this has not been a required component in metadata records. It is currently suggested that this information be made available in the existing “Supplementary Information” field. An example of a data dictionary table is shown below, using an extract of Town Planning Zone information:
Data Dictionary: Column descriptions
The zone code of the feature. Used for thematic styling. See the table below relating to the Zone Code descriptions.
The descriptive value of the planning zone of the feature. Used for labelling, identifying etc.
Data Dictionary: Column ‘CODE’ attribute descriptions (note thematic hints are optional)
Recommended Colour Code
All methods that assist in describing the data to enable a user to identify if a particular dataset is “fit for purpose” should be included in the data dictionary, if it is not already part of the essential metadata descriptors. This could include preferred layer name (e.g. how the dataset does / should appear in a viewing application) and thematic hints such as appropriate scale range and colours.
4 The data in the database is linked to its ANZLIC Identifier
A table should exist within the dissemination database alongside the dataset linking the table name to the ANZLIC identifier for the metadata. This table should always be called “METADATA”, and it should have at least the column names ‘TABLE_NAME’ and ‘ANZLIC_ID’. This will provide users browsing the dissemination database with a list of the datasets available to them, helpful when the dissemination database holds a large number of datasets.
Both NRDB1 and SDB1 include a table containing this information.
5 The data is added to the appropriate access services
It is important to restate that this checklist does not prescribe that data be added to all services, it is expected that certain services will be formally rejected for various reasons. Instead, it prompts the custodian to consider all data publishing options and be satisfied that end users will be able to make best use of the data.
There are a number of different mechanisms currently in place as part of the NTG SDI that permit the viewing and sharing of spatial data within the NTG and the public. Custodians should consider making their data available through these existing services, all of which are ready to accept NTG SDI ‘certified’ data. Currently NTLIS will do the work of adding a dataset to the services, this work is minimal once the data has passed the other items on the checklist.
5.1 NT Atlas
NT Atlas (http://www.ntlis.nt.gov.au/atlas/) is a web browser application available on the public internet providing users with the ability to view and identify on the more commonly used NTG spatial datasets. The NT Atlas provides an interface to the NTSDD and is a good way to advertise the availability of a dataset to the public to promote its use.
5.2 ILISMaps / FME Spatial Data Download
ILISMaps (https://www.ilis.nt.gov.au/imf/imf.jsp?site=ilismap) is a more functional version of the NT Atlas, able to external registered ILIS users as well as general NTG staff. ILISMaps provides the interface with FME Spatial Data Download allowing users to extract vector data in any format and projection.
5.3 OGC Web Mapping Service (WMS) and Web Feature Service (WFS)
Offering a dataset via WMS and WFS allows a range of GIS applications including ESRI ArcGIS and MapInfo Pro to connect read-only to the data. In addition, it is relatively easy for 3rd parties to design applications to use data offered using these mechanisms in a sustainable, live-access type way.
WMS stands for 'Web Mapping Server' and provides the end user with non-georeferenced images (usually PNG or JPeg) containing the layers and extent provided in the request. More information about WMS in the NTG can be found here: Web Mapping Servers (WMS) available from the NT Government. All layers added to ILISMaps are currently made available on the public internet via WMS.
WFS stands for "Web Feature Service" and provides the end user with an XML representation of the data, including all attributes and coordinates. Functionally, this is the equivalent to providing the user with a shape file or tab file, in that it contains all the information required to perform attribute and geometry queries and processing. Adding a layer to the WFS server allows programmers to write code able to query that layer easily.. for instance the address search in NT Visualiser makes a live query to the WFS server to return the geometry of a parcel from its address attributes. NTLIS does not currently offer a WFS server outside the government firewall.
5.4 Google Earth Enterprise (NT Visualiser)
The NTG has purchased the server version of Google Earth and branded it ‘NT Visualiser’. NT Visualiser provides a intuitive visualisation interface able to integrate a range of datasets and services.
5.5 NRETA Maps
Appropriate to certain ‘data communities’. NRETA Maps (insert the link here) is a web browser application available on the government intranet for general NTG staff. It contains datasets relating to natural resourses.
NTLIS will be working with custodians and this document to populate the dissemination databases with government datasets. The progress of this will be available in the form of a checklist of ‘scheduled’ and ‘completed’ datasets as they are made accessible within the NTG SDI.
In addition, we will be working with custodians from other departments to achieve similar concepts in their areas.
If there are any questions or suggestions please don’t hesitate to contact us at: email@example.com