NEANIAS Gitlab

Commit b2d3fd6c authored by Carlos H. Brandt's avatar Carlos H. Brandt
Browse files

Add archive/db sections to data-store doc

parent c50995d5
## Data Store
In this document we discuss the structure of the data archive and metadata database
necessary not only to keep queries and access to data efficient but also to store
all the _history_ data products have gone through since the original/source retrieval.
The bulk data storage in our archives/database is meant to support the services being
provided, the _reduced_ data.
Before it, _upstream_ data and intermediary states are temporary, kept only
enough to succeed in creating the _reduced_ version.
Service _products_ are kept for the long term either for reuse by our services
-- _e.g._, mosaic products used on landind sites analysis --
or for direct access and download.
### Archive
The data (files) are stored in a filesystem (or object store) according to their
processing level and their file format; We can (and do) have multiple file formats
for the same (processing) level (_e.g._, GeoTIFF and ISIS cubes)
Processing level can be seen as _versions_ of the same (original) data product, and
represent the time (processing) evolution (history) of that data product.
Whereas multiple file formats represent different views (or viewing modes) for a
given data product (level).
### DB
Databses to store spatial data considered: MondoDB or PostGIS
MongoDB is a NoSQL, document-based database with support for WGS84 spatial index.
The index can hold polugons representing products footprint, allowing _intersection_
queries. Spatial data points allowed by MongoDB spatial index are _planetocentric east
positive_ centered on zero ([-180:+180] longitude, [-90:+90] latitude).
PostGIS database is a Postgresql database with WGS84 index support. As a relational
database it is, table fields must be pre-defined (different from document-based
databases) and groups/blocks of fields are represented by tables that relate through a
_primary_ key/field.
#### Store data and metadata
Store final products in archive and insert respective metadata to DB
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment