Welcome to the SnowEx Database

Welcome to the SnowEx Database#

Documentation Status Testing Status code coverage

Database creation and management software for SnowEx data. The goal is to create a single source (citeable) dataset that is cross queriable for snow researchers.

WARNING - This is under active development in preparation for SnowEx Hackweek. Use at your own risk. Data will change as it is QA/QC’d and the end goal is for all data in this database to be pulled from NSIDC. The goal is for this to become a community database open to all.

DOI#

  • SnowEx Hackweek 2024 - HW24

  • SnowEx Hackweek 2022 - HW22

Features#

  • Auto parse header information and include it in upload

  • Interpret and upload csv, rasters, uavsar files,

  • Full provenance for download and upload

  • Manage Site, Point, Profile and Raster Data

  • Postgres SQL Database end point for researchers

Installing#

If you are are database maintainers or installing it for the first time follow the instructions below completely.

Mac OS#

First ensure you have following prerequisites:

  • Python3.8 +

  • HomeBrew

Then to install the postgres database with postgis functionality run:

cd scripts/install && sh install_mac.sh

Ubuntu#

First ensure you have following prerequisites:

  • Python3.8 +

  • wget

Then to install the postgres database with postgis functionality run:

cd scripts/install && sh install_ubuntu.sh

Docker#

Alternatively if you have docker install on either os, the simplest way to spin up the db is to run:

docker-compose up -d

To tear it down (which will delete the data!)

docker-compose down

Python#

Install the python package by:

python3 setup.py install

If you are planning on running the tests or building the docs below also run:

pip install -r requirements_dev.txt

If you are using conda you may need to reinstall the following using conda:

  • Jupyter notebook

  • nbconvert

Tests#

Quickly test your installation by running:

pytest

This will run a series of tests that create a small database and confirm that samples of the data sets references in ./scripts/upload folder can be uploaded seamlessly. These tests can serve as a nice way to see how to interact with the database but also serve to confirm our reproducibility.

The goal of this project is to have high fidelity in data interpretation/submission to the database. To see the current test coverage run:

make coverage

Documentation#

There is a whole host of resources for users in the documentation. It has been setup for you to preview in your browser.

In there you will find:

  • Database structure

  • API to the python package snowex_db

  • Links to other resources

  • Notes about the data uploaded

  • Info on populating the database

  • And more!

To see the documentation in your browser:

Warning: To see the examples/gallery you will need to populate the database before running this command. Otherwise they will be left with the last image submitted to GitHub.

make docs
Contents