The NTLIS Imagery Catalogue offers improved file storage and metadata management. The design and functionality features are described in the attached document.

Introduction


There is a requirement to better capture the metadata relating to satellite and ortho-rectified imagery assets held by the NTG, in particular by DPI and NRETA. This will make it easier for the data custodians and end users to find and use the assets, and will reduce duplication of storage and procurement across the agencies.

The capture process will require the development of a management application and database into which custodians can input the metadata information. The scope of this document is to describe the management application and database, and requirements of custodians for management of the data.

Currently the Department of NRETA has a Microsoft Access database that contains a catalogue of remotely sensed data aquired by the Department. The new system is being designed in the Oracle / web environment, complying with the NT government spatial dissemination database paradigm.

Along with the development of the metadata management application and database is the establishment of online storage for the imagery itself, allowing for easy extraction of data once the desired image asset has been identified. Imagery extraction allows users to download an image to their machine if the various online linkage methods are not sufficient.

A metadata harvester has also been developed, able to extract georeferencing details such as extent and coordinate system and create a metadata record including all online linkage fields, minimising the amount work required of the custodian.


Data Model


The fields of metadata able to be captured in the Imagery Catalogue have been determined by NRETA and DPI image custodians and are listed below. Completion of these fields will ensure each image asset is able to be described in the ISO 19115/19139 metadata standard. For further information regarding NT spatial metadata go to: http://www.ntlis.nt.gov.au/forum/ntlis_documents/ntlis-metadata-tool/

Table 1 (below) is an outline of the fields and expected data input for the main database table. Fields in bold are able to be automatically harvested from the imagery itself, as described in the next section.


CatalogueDescription
UFIUnique Feature Identification: auto generated
START DATEDate of capture for the data or earliest date for collective dataset (mosaic)
END DATEEnd date for collective dataset ie mosaic, time series dataset
CUSTODIANCustodian name, sourced from a list
SENSORLandsat, SPOT, MODIS, etc
SENSOR TYPEPanchromatic, Multispectral, Hyperspectral
BANDSSpecific to sensor or what was purchased ie 1-7 or Infra-red
RESOLUTIONPixel size represented in meters, e.g: 0.25
PROCESS LEVELRaw, projected, rectified, mosaic, calibrated
PROCESS LEVEL COMMENTSBrief comments regarding process steps ie calibrated to yr2000 data
FILE FORMAT.ers, .img, .dat, .jp2, .bil, .udf
FILE NAMEName of the file where it is stored
STORAGE LOCATIONWhere cd/dvd physically located i.e. Berrimah
STORAGE NOCD, DVD or Tape reference number
LAYER NAMELink to image data file, used for online linkages. Usually same as the file name
WMS SERVER URLLink to WMS GetCapabilities document
ECWP SERVER URLLink to image via ECWP protocol
DATUMAGD66, GDA94, AGD84
PROJECTIONMGA50-54, Lat Long
USAGE RESTRICTIONSCoded i.e. 1=Unrestricted, 2=NTG restricted, 3=In-confidence, 4=Protected, 5= Highly protected
COSTPurchasing cost
PROJECT NAMEDeptartment project name acquiring remotely sensed data
USAGE RESTRICTIONS TEXTDescribe copyright limitations
PROVIDERCompany the image was sourced from, e.g: GeoImage, SKM, ACRES
PREVIEW
Link to a quicklook image, used as a thumbnail for users search through images
PURCHASER NAMENominated officer purchasing data
DATE GOOGLE FUSEDDate the image was fused into the NT Visualiser world
DATE GOOGLE RETIREDDate the image was removed from the NT Visualiser world



Table 1: Imagery Catalogue Fields


Automatic Harvesting of Metadata


NTLIS has developed a procedure whereby custodians are able to copy imagery into a specific network location and have it harvested and inserted into the Imagery Catalogue, assuming the file meets certain minimum criteria. The bolded rows in Table 1 are able to be automatically harvested. This process automates the extraction of spatial properties and online linkage information, as well as copies the file itself to a secure, accessible location. At this time the automatic harvesting only works if the imagery is georeferenced and stored as a JPEG 2000 file.

The information not able to harvested can be filled in after the image has been inserted by the Harvester.


Technical Architecture


Figure 1 (below) describes the structure of the Imagery Catalogue and its associated components. As illustrated, the metadata capture in the Management Application is stored as a regular Oracle Spatial table on the NRDB dissemination database, allowing a range of clients including MapInfo, ArcGIS, FME and custom web services to query and display the information.

Importantly, the Management Application of the Imagery Catalogue is only used by custodians, in the future there will be a Discovery Application developed allowing internal and external users to search the catalogue and optionally extract imagery data.






Figure 1: Technical Architecture



Discovery Application


As per the architectural diagram, the data will be stored and made available in the NRDB dissemination database so that 3rd party discovery applications can easily be built. In addition, NTLIS will develop an 'official' Discovery Application, as is the case with the current NTLIS Aerial Photography Index. The discovery application will allow users to search on any field stored in the metadata, as well as its spatial extent.

Database search criteria will include:

  • Drawing a boundary on a map;  e.g. NRETA maps integration
  • Path / row searches – Landsat, SPOT
  • 1:250K Map Sheets
  • Sensor
  • Date or date range
  • Spatial datasets i.e. Bioregion, pastoral or parcel name/number, district
  • A combination of all or some of the above


Imagery Extraction


Images that have been made available on an online data storage point will be able to be extracted in a range of formats and coordinate systems for use in their 3rd party program, similar to the Spatial Data Download function in ILISMaps (NT Government only). In addition, users will be able to connect to online imagery using the WMS and ECWP protocols, removing the need to store the image locally.

A number of free plug-ins supporting output formats to assist clients in utilising the images for a variety of purposes or software they utilise are available.


Imagery Catalogue Migration and Management


Content from the old MS Access Satellite Imagery Database will be migrated to the new online system. The remainder of the database entry is the responsibility of the respective custodians, for the data they are responsible for at the divisional level;  i.e. Biodiversity, Bushfires, Resource Management, Land & Water etc. Custodians need to check the data transferred from the Access database and fill in all the appropriate fields relevant to the individual dataset.

Custodians are to have edit access to only datasets that they are responsible for, all other files will be view/read only.

Custodians are the persons best placed to update and maintain the database, due to their knowledge of the data, requests for the data by clients, and physical location of the data.

  • Custodians will be responsible for;
    • Data correctness within the database to ensure integrity
    • Loading required data files onto the server
    • Generate metadata for all data sets available in the database. This will in time allow the database to have web based delivery similar to DPI Aerial Photography Index or through NRETA Maps. Besides this requirement Metadata needs to be up to standard for the purposes of correct data management procedures.
    • Generating JPEG2000 (lossless) files with the most appropriate bands and stretch applied to generate quicklook images for use by clients. Unless this can be automatically generated.



Procedure for Updating Database and Acquiring Imagery Data


Custodians will be responsible for updating the database with any new data acquired, filling required fields in the database, generating quicklook and metadata files and linking these files and the original data files to the database as described previously. Before purchasing new data it is advisable to adhere to the following steps:

  • Ascertain the appropriate satellite sensor to meet the requirements of the project/s.
  • Check the database for similar data types previously purchased for the area of interest and possibly adjacent areas. This could help determine what type of data, time of year, and who else may benefit from a new purchase.
  • Check with other projects or personnel in regards to the purchase and whether mutual benefits from a shared purchase can be achieved. This may require either an email system to all custodial managers regarding the proposed purchase and gain feedback or a centralised forum by which to communicate.
  • These steps will hopefully generate a greater understanding of what is to be purchased (sensor, process level, date, area etc) by various groups or projects and possibly maximising benefits for other projects through sharing of data and costs, and knowledge of other projects in operation.


After purchasing new data or creating an improved version of the purchased data (eg rectified, calibrated, mosaic) the following steps need to be adhered to maintain and update the database:
Metadata generated for the data – linked to the database.

  • A copy of the image data file to the storage facility – linked to the database.
  • A copy of the quicklook generated and placed on the server – linked to the database.
  • Geometry for the image data generated for search capabilities linked to the database. This maybe the same as a previous purchase covering the same area, hence the same geometry can be used.
  • All other fields need to be filled in for the new data, a proportion of which can be accessed from the metadata.
  • Check that all links to image data, quicklooks and metadata are active and correct.
  • No labels

feedback|contact us|interpreter services|disclaimer / copyright / privacy|email site manager|sitemap

© Northern Territory Government of Australia 2006-2020